# timauzi/llm-x-cxg architecture

> Architecture diagram of the GitHub repository timauzi/llm-x-cxg, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/timauzi/llm-x-cxg
- Repository: https://github.com/timauzi/llm-x-cxg

## Overview

##### Project Type and Purpose
- This is a research‐oriented Python project (not a live service) for probing construction grammar (CxG) in large language models (LLMs).
- Two primary tasks:
a. Construction Detection (finding predefined constructions in corpora)
b. Text Generation (few‐shot or zero‐shot example generation guided by construction definitions)

##### High‐Level Architecture
At the highest level, the system comprises:
A. Data Layer
- Raw corpora, construction annotations, pseudoword definitions
- Experiment outputs (detection results, generated examples, similarity measures)
B. Preprocessing & Pseudoword Module
- Scripts to produce pseudoword embeddings (based on BERT tooling)
C. Experiment Pipelines
1. Detection Pipeline (LLM + CxG matcher → detection TSVs)
2. Generation Pipeline (few‐shot templates + LLM → generated TSVs)
3. Similarity Analysis (embedding comparisons → similarity TSVs)
D. Evaluation Layer
- Jupyter notebooks for automated and manual evaluation of outputs
E. Environment & Config
- Conda environment (llm-cxg.txt) for reproducibility

##### Main Components & Interactions
- Data Sources (data/annotations, data/detection, etc.) feed into Preprocessing & Experiment Pipelines.
- Preprocessing & Pseudoword Module (src/pseudowords/) generates augmented embeddings used by both detection and generation pipelines.
- Experiment Pipelines invoke Huggingface LLMs (BERT, Llama 2, mBART) with varying shot settings and constructions (0–5 shots).
- Pipelines write outputs back into data/detection/, data/generation/, data/similarities/.
- Evaluation Notebooks (src/notebooks/) read these outputs, compute metrics, produce figures, and support manual inspection.

##### Architectural Patterns & Design Principles
- Modular Pipeline Pattern: clear separation of data ingestion, model invocation, and evaluation.
- Directory‐as‐Module: each subfolder (e.g., data/generation, src/pseudowords) encapsulates its function.
- Versioned Data Artifacts: TSV files version experiments by model, attempts, and shot counts.

##### Technologies & Frameworks
- Python + Jupyter Notebooks
- Huggingface Transformers (BERT, Llama 2, mBART)
- Pandas/TSV for data interchange
- Conda for environment management
- Bash scripts (e.g., script_bsb.sh) for batch runs

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Store
    subgraph "Data Store"
        direction TB
        DataRaw["Raw & Annotated Corpora"]:::data
        DataDetOut["Detection Outputs (TSVs)"]:::data
        DataGenOut["Generation Outputs (TSVs)"]:::data
        DataSimOut["Similarity Outputs (TSVs)"]:::data
    end

    %% Preprocessing
    Pseudo["Pseudoword Generator Module"]:::processing

    %% Experiment Pipelines
    subgraph "Experiment Pipelines"
        direction TB
        subgraph "Detection Pipeline" 
            direction TB
            BERT_Detect["bert_find_construction.ipynb"]:::processing
            LLAMA_Detect["llama_find_construction.ipynb"]:::processing
        end
        subgraph "Generation Pipeline"
            direction TB
            BERT_Gen1["bert_comapp_generate_examples.ipynb"]:::processing
            LLAMA_Gen1["llama_generate_examples.ipynb"]:::processing
            MBART_Gen1["mbart_comapp_generate_examples.ipynb"]:::processing
            MBART_Gen2["mbart_generate_examples.ipynb"]:::processing
            LLAMAxBERT_Gen["llama_x_bert_comapp_generate_examples.ipynb"]:::processing
        end
        subgraph "Similarity Analysis"
            direction TB
            SimAnalyze["compare_embeddings.ipynb"]:::processing
        end
    end

    %% Evaluation
    subgraph "Evaluation Notebooks"
        direction TB
        EvalBERT["eval_bert.ipynb"]:::evaluation
        EvalLLAMA["eval_llama.ipynb"]:::evaluation
        EvalLLAMA2["evaluate_llama.ipynb"]:::evaluation
        EvalGen["eval_generate.ipynb"]:::evaluation
        EvalGenHuman["eval_generate_human.ipynb"]:::evaluation
        EvalMatches["eval_matches.ipynb"]:::evaluation
        EvalCompare["eval_compare.ipynb"]:::evaluation
        subgraph "Utilities & Setup"
            direction TB
            CommonDeps["get_common_dependencies.ipynb"]:::evaluation
            ReadCons1["read_constructions.ipynb"]:::evaluation
            ReadCons2["read_constructions_bert.ipynb"]:::evaluation
            ReadCons3["read_constructions_deps.ipynb"]:::evaluation
            Readme["readme.md"]:::evaluation
        end
    end

    %% Environment
    Conda["Conda Env / Config (llm-cxg.txt)"]:::env

    %% Connections
    DataRaw -->|feeds| Pseudo
    Pseudo -->|augmented embeddings| BERT_Detect
    Pseudo -->|augmented embeddings| LLAMA_Detect
    Pseudo -->|augmented embeddings| BERT_Gen1
    Pseudo -->|augmented embeddings| LLAMA_Gen1
    Pseudo -->|augmented embeddings| MBART_Gen1
    Pseudo -->|augmented embeddings| MBART_Gen2
    Pseudo -->|augmented embeddings| LLAMAxBERT_Gen
    Pseudo -->|augmented embeddings| SimAnalyze

    BERT_Detect -->|writes| DataDetOut
    LLAMA_Detect -->|writes| DataDetOut

    BERT_Gen1 -->|writes| DataGenOut
    LLAMA_Gen1 -->|writes| DataGenOut
    MBART_Gen1 -->|writes| DataGenOut
    MBART_Gen2 -->|writes| DataGenOut
    LLAMAxBERT_Gen -->|writes| DataGenOut

    SimAnalyze -->|writes| DataSimOut

    DataDetOut -->|inputs| EvalBERT
    DataDetOut -->|inputs| EvalLLAMA
    DataGenOut -->|inputs| EvalGen
    DataGenOut -->|inputs| EvalGenHuman
    DataSimOut -->|inputs| EvalCompare
    DataSimOut -->|inputs| EvalMatches

    %% Conda links
    Conda -.-> Pseudo
    Conda -.-> BERT_Detect
    Conda -.-> LLAMA_Detect
    Conda -.-> BERT_Gen1
    Conda -.-> LLAMA_Gen1
    Conda -.-> MBART_Gen1
    Conda -.-> MBART_Gen2
    Conda -.-> LLAMAxBERT_Gen
    Conda -.-> SimAnalyze
    Conda -.-> EvalBERT
    Conda -.-> EvalLLAMA
    Conda -.-> EvalLLAMA2
    Conda -.-> EvalGen
    Conda -.-> EvalGenHuman
    Conda -.-> EvalMatches
    Conda -.-> EvalCompare
    Conda -.-> CommonDeps
    Conda -.-> ReadCons1
    Conda -.-> ReadCons2
    Conda -.-> ReadCons3
    Conda -.-> Readme

    %% Click Events
    click DataRaw "https://github.com/timauzi/llm-x-cxg/tree/main/data/annotations/"
    click DataDetOut "https://github.com/timauzi/llm-x-cxg/tree/main/data/detection/"
    click DataGenOut "https://github.com/timauzi/llm-x-cxg/tree/main/data/generation/"
    click DataSimOut "https://github.com/timauzi/llm-x-cxg/tree/main/data/similarities/"
    click Conda "https://github.com/timauzi/llm-x-cxg/blob/main/llm-cxg.txt"
    click Pseudo "https://github.com/timauzi/llm-x-cxg/tree/main/src/pseudowords/"
    click BERT_Detect "https://github.com/timauzi/llm-x-cxg/blob/main/src/notebooks/bert_find_construction.ipynb"
    click LLAMA_Detect "https://github.com/timauzi/llm-x-cxg/blob/main/src/notebooks/llama_find_construction.ipynb"
    click BERT_Gen1 "https://github.com/timauzi/llm-x-cxg/blob/main/src/notebooks/bert_comapp_generate_examples.ipynb"
    click LLAMA_Gen1 "https://github.com/timauzi/llm-x-cxg/blob/main/src/notebooks/llama_generate_examples.ipynb"
    click MBART_Gen1 "https://github.com/timauzi/llm-x-cxg/blob/main/src/notebooks/mbart_comapp_generate_examples.ipynb"
    click MBART_Gen2 "https://github.com/timauzi/llm-x-cxg/blob/main/src/notebooks/mbart_generate_examples.ipynb"
    click LLAMAxBERT_Gen "https://github.com/timauzi/llm-x-cxg/blob/main/src/notebooks/llama_x_bert_comapp_generate_examples.ipynb"
    click SimAnalyze "https://github.com/timauzi/llm-x-cxg/blob/main/src/notebooks/compare_embeddings.ipynb"
    click EvalBERT "https://github.com/timauzi/llm-x-cxg/blob/main/src/notebooks/eval_bert.ipynb"
    click EvalLLAMA "https://github.com/timauzi/llm-x-cxg/blob/main/src/notebooks/eval_llama.ipynb"
    click EvalLLAMA2 "https://github.com/timauzi/llm-x-cxg/blob/main/src/notebooks/evaluate_llama.ipynb"
    click EvalGen "https://github.com/timauzi/llm-x-cxg/blob/main/src/notebooks/eval_generate.ipynb"
    click EvalGenHuman "https://github.com/timauzi/llm-x-cxg/blob/main/src/notebooks/eval_generate_human.ipynb"
    click EvalMatches "https://github.com/timauzi/llm-x-cxg/blob/main/src/notebooks/eval_matches.ipynb"
    click EvalCompare "https://github.com/timauzi/llm-x-cxg/blob/main/src/notebooks/eval_compare.ipynb"
    click CommonDeps "https://github.com/timauzi/llm-x-cxg/blob/main/src/notebooks/get_common_dependencies.ipynb"
    click ReadCons1 "https://github.com/timauzi/llm-x-cxg/blob/main/src/notebooks/read_constructions.ipynb"
    click ReadCons2 "https://github.com/timauzi/llm-x-cxg/blob/main/src/notebooks/read_constructions_bert.ipynb"
    click ReadCons3 "https://github.com/timauzi/llm-x-cxg/blob/main/src/notebooks/read_constructions_deps.ipynb"
    click Readme "https://github.com/timauzi/llm-x-cxg/blob/main/src/notebooks/readme.md"

    %% Styles
    classDef data fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef processing fill:#90EE90,stroke:#333,stroke-width:1px
    classDef evaluation fill:#FFA500,stroke:#333,stroke-width:1px
    classDef env fill:#DDA0DD,stroke:#333,stroke-width:1px
```
