# openai/gpt-3 architecture

> Architecture diagram of the GitHub repository openai/gpt-3, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/openai/gpt-3
- Repository: https://github.com/openai/gpt-3

## Overview

##### Project type and purpose
- This repository is a **research artifact bundle** for the paper “GPT-3: Language Models are Few-Shot Learners.” It is *not* an application with servers or UIs, but a **data-centric project** that collects:
- Model outputs (unfiltered GPT-3 samples)
- Synthetic datasets for tasks like arithmetic and word scrambles
- Statistics on training-set languages
- Analytical reports (overlap frequency, model card)
- The goal is to *deliver* all of these artifacts in a well-organized, discoverable structure for researchers to inspect, reuse, or build upon.

##### File structure
- Top level:
- 175b_samples.jsonl → raw GPT-3 text generations
- README.md, model-card.md, overlap_frequency.md → documentation
- data/ → synthetic task datasets (.jsonl, .jsonl.gz)
- dataset_statistics/ → CSV statistics and its own README
- No “src” or compile scripts: this is a **static repository** of data + docs.

##### README for additional insights
- Describes the paper, points to the sample file and directories.

##### Detail
- That dataset_statistics/CSV files correspond to specific portions of data (e.g., languages_by_word_count.csv ← training-corpus language files).
- README files as separate doc nodes that link back to the artifacts they describe.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Sources
    Inference["GPT-3 Inference"]:::source
    Generator["Synthetic Data Generator"]:::source

    %% Storage Repository
    subgraph "Repository (Storage)" 
        direction TB
        Outputs["175b_samples.jsonl"]:::storage
        DataDir["data/"]:::storage
        subgraph "data/" 
            direction TB
            CLW["cycle_letters_in_word.jsonl.gz"]:::storage
            FDAdd["five_digit_addition.jsonl"]:::storage
            FDSub["five_digit_subtraction.jsonl"]:::storage
            FoDAdd["four_digit_addition.jsonl"]:::storage
            FoDSub["four_digit_subtraction.jsonl"]:::storage
            MWA1["mid_word_1_anagrams.jsonl.gz"]:::storage
            MWA2["mid_word_2_anagrams.jsonl.gz"]:::storage
            RIW["random_insertion_in_word.jsonl.gz"]:::storage
            RW["reversed_words.jsonl.gz"]:::storage
            SD3O["single_digit_three_ops.jsonl"]:::storage
            SD6Add["six_digit_addition.jsonl"]:::storage
            SD6Sub["six_digit_subtraction.jsonl"]:::storage
            SOD["sum_of_digits.jsonl"]:::storage
            TDAdd["three_digit_addition.jsonl"]:::storage
            TDSub["three_digit_subtraction.jsonl"]:::storage
            TwDAdd["two_digit_addition.jsonl"]:::storage
            TwDMul["two_digit_multiplication.jsonl"]:::storage
            TwDSub["two_digit_subtraction.jsonl"]:::storage
        end
        StatsDir["dataset_statistics/"]:::storage
        subgraph "dataset_statistics/" 
            direction TB
            StatsReadme["README.md"]:::doc
            LangChar["languages_by_character_count.csv"]:::storage
            LangDoc["languages_by_document_count.csv"]:::storage
            LangWord["languages_by_word_count.csv"]:::storage
        end
    end

    %% Processing Pipeline
    StatsPipeline["Statistics Pipeline"]:::process

    %% Documentation
    DocReadme["README.md"]:::doc
    DocModelCard["model-card.md"]:::doc
    DocOverlap["overlap_frequency.md"]:::doc

    %% Researcher Consumer
    Researcher["Researcher"]:::external

    %% Connections
    Inference -->|writes| Outputs
    Generator -->|populates| DataDir
    Outputs -->|input| StatsPipeline
    DataDir -->|input| StatsPipeline
    StatsPipeline -->|emits| StatsDir

    Outputs -->|described in| DocReadme
    DataDir -->|documented in| DocReadme
    StatsDir -->|detailed in| DocReadme
    StatsReadme -->|context for| StatsDir
    Outputs -->|referenced by| DocOverlap
    StatsDir -->|referenced by| DocOverlap
    StatsDir -->|referenced by| DocModelCard
    DocReadme -->|read by| Researcher
    DocModelCard -->|read by| Researcher
    DocOverlap -->|read by| Researcher
    StatsReadme -->|read by| Researcher

    %% Click Events
    click Outputs "https://github.com/openai/gpt-3/blob/master/175b_samples.jsonl"
    click DataDir "https://github.com/openai/gpt-3/tree/master/data/"
    click CLW "https://github.com/openai/gpt-3/blob/master/data/cycle_letters_in_word.jsonl.gz"
    click FDAdd "https://github.com/openai/gpt-3/blob/master/data/five_digit_addition.jsonl"
    click FDSub "https://github.com/openai/gpt-3/blob/master/data/five_digit_subtraction.jsonl"
    click FoDAdd "https://github.com/openai/gpt-3/blob/master/data/four_digit_addition.jsonl"
    click FoDSub "https://github.com/openai/gpt-3/blob/master/data/four_digit_subtraction.jsonl"
    click MWA1 "https://github.com/openai/gpt-3/blob/master/data/mid_word_1_anagrams.jsonl.gz"
    click MWA2 "https://github.com/openai/gpt-3/blob/master/data/mid_word_2_anagrams.jsonl.gz"
    click RIW "https://github.com/openai/gpt-3/blob/master/data/random_insertion_in_word.jsonl.gz"
    click RW "https://github.com/openai/gpt-3/blob/master/data/reversed_words.jsonl.gz"
    click SD3O "https://github.com/openai/gpt-3/blob/master/data/single_digit_three_ops.jsonl"
    click SD6Add "https://github.com/openai/gpt-3/blob/master/data/six_digit_addition.jsonl"
    click SD6Sub "https://github.com/openai/gpt-3/blob/master/data/six_digit_subtraction.jsonl"
    click SOD "https://github.com/openai/gpt-3/blob/master/data/sum_of_digits.jsonl"
    click TDAdd "https://github.com/openai/gpt-3/blob/master/data/three_digit_addition.jsonl"
    click TDSub "https://github.com/openai/gpt-3/blob/master/data/three_digit_subtraction.jsonl"
    click TwDAdd "https://github.com/openai/gpt-3/blob/master/data/two_digit_addition.jsonl"
    click TwDMul "https://github.com/openai/gpt-3/blob/master/data/two_digit_multiplication.jsonl"
    click TwDSub "https://github.com/openai/gpt-3/blob/master/data/two_digit_subtraction.jsonl"
    click StatsDir "https://github.com/openai/gpt-3/tree/master/dataset_statistics/"
    click StatsReadme "https://github.com/openai/gpt-3/blob/master/dataset_statistics/README.md"
    click LangChar "https://github.com/openai/gpt-3/blob/master/dataset_statistics/languages_by_character_count.csv"
    click LangDoc "https://github.com/openai/gpt-3/blob/master/dataset_statistics/languages_by_document_count.csv"
    click LangWord "https://github.com/openai/gpt-3/blob/master/dataset_statistics/languages_by_word_count.csv"
    click DocReadme "https://github.com/openai/gpt-3/blob/master/README.md"
    click DocModelCard "https://github.com/openai/gpt-3/blob/master/model-card.md"
    click DocOverlap "https://github.com/openai/gpt-3/blob/master/overlap_frequency.md"

    %% Styles
    classDef source fill:#bbdefb,stroke:#2196f3,color:#0d47a1;
    classDef storage fill:#c8e6c9,stroke:#388e3c,color:#1b5e20;
    classDef process fill:#ffe0b2,stroke:#fb8c00,color:#e65100;
    classDef doc fill:#eceff1,stroke:#90a4ae,color:#546e7a;
    classDef external fill:#f3e5f5,stroke:#ab47bc,color:#4a148c;
```
