# meta-introspector/lang_agent architecture

> Architecture diagram of the GitHub repository meta-introspector/lang_agent, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/meta-introspector/lang_agent
- Repository: https://github.com/meta-introspector/lang_agent

## Overview

##### Project type and purpose
- This is an open-source “language agent” toolkit written in OCaml.
- It provides a collection of command-line tools (under bin/) that orchestrate various prompt-based tasks (dungeon generation, poem writing, grammar parsing, etc.) by calling out to LLM back-ends (OpenAI, Huggingface, Ollama, local LlamaCpp).
- It also bundles Coq files (_CoqProject/) and grammar definitions (grammars/) for proof­-oriented or parsing tasks.

##### High-level components
a. CLI Drivers (bin/)
- Each executable script (chunker.ml, generate_dungeon.ml, scanner.ml, …) is a thin “driver” that invokes core APIs.
b. Core Library (lib/)
- Domain modules (Lang_model, Model_type, Mythos, Embody, CoqOfOCaml, etc.)
- Parsers and grammar runners (Yojson, BNF definitions)
- File-IO and utility modules (folder.ml, unix_handling.ml, StdLib)
c. LLM Adapters
- openai_client.ml – HTTP to OpenAI
- llama_cpp.ml – local C bindings
- ollama.ml / ollama_lwt.ml – Docker or local Ollama interface
d. Data & Grammar Assets
- grammars/ – .gbnf, .mly definitions
- data/huggingface_lang_agent – model weights or snapshot data
e. Build & DevOps
- Dune files, opam (lang_agent.opam), flake.nix, shell.nix
- Dockerfile, Makefile, build scripts (build.sh, build_push.sh)
- CI workflows (.github/workflows/docker-image.yml)
f. Tests
- test/ – test_lang_agent.ml
- script wrappers (test.sh, run_example*.sh)

##### Relationships & data flows
- The CLI Drivers layer calls into the Core Library API.
- Core Library dispatches to one or more LLM Adapters based on configuration (env vars, flags).
- LLM Adapters communicate over HTTP or FFI to External Services (OpenAI, Huggingface Inference API, Ollama server, local LlamaCpp).
- Responses flow back to Core Library, which may pass them through Grammar Modules for parsing or Coq modules for proof verification.
- Results are then written to stdout or to files in the workspace.
- Build & DevOps components orchestrate compilation (Dune/Nix), packaging (Docker), and CI runs.

##### Architectural patterns & principles
- Hexagonal (“ports and adapters”) architecture: separate domain logic (lib/) from IO concerns (LLM adapters, file handling, CLI).
- Modular design: each bin/ script is a composition of reusable library functions and adapters.
- Extensibility: adding a new LLM back-end only requires writing a new adapter module and wiring it into the dispatch logic.
- Declarative build: Dune + Nix + Docker provide consistent environments for reproducible builds and CI.

##### Essential detail checklist
- Bin/ scripts as entry-points
- lib/ modules as core domain
- Adapters to each LLM back-end
- grammars/ and data/ as resource stores
- External LLM services vs. local libraries
- Build & CI (Dune, Nix, Docker, GitHub Actions)
- Testing harness and sample workflows (test/, run_example*.sh)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% CLI Drivers
    subgraph "CLI Drivers" 
        direction TB
        A1["argiope.ml"]:::cli
        A2["chunker.ml"]:::cli
        A3["execute_self.ml"]:::cli
        A4["generate_dungeon.ml"]:::cli
        A5["generate_generic.ml"]:::cli
        A6["la_decode_json.ml"]:::cli
        A7["poem.ml"]:::cli
        A8["recurse.ml"]:::cli
        A9["scanner.ml"]:::cli
        A10["simple.ml"]:::cli
        A11["simple_grammar.ml"]:::cli
        A12["test1.ml"]:::cli
        A13["worker.ml"]:::cli
        A14["dune"]:::cli
    end

    %% Core Library
    subgraph "Core Library" 
        direction TB
        B1["lang_model.ml/.mli/.v/.org"]:::core
        B2["model_type.ml"]:::core
        B3["mythos.ml/.mli/.json"]:::core
        B4["embody.ml/.mli/.v"]:::core
        B5["CoqOfOCaml.v"]:::core
        B6["StdLib.v"]:::core
        B7["Yojson.v"]:::core
        B8["Begin.v"]:::core
        B9["Extract_lang_model.v"]:::core
        B10["Test_archetype_instance.v/.ml/.mli"]:::core
        B11["folder.ml"]:::core
        B12["unix_handling.ml"]:::core
        B13["dune"]:::core
    end

    %% LLM Adapters
    subgraph "LLM Adapters"
        direction TB
        C1["openai_client.ml"]:::adapter
        C2["ollama.ml"]:::adapter
        C3["ollama_lwt.ml"]:::adapter
        C4["llama_cpp.ml"]:::adapter
    end

    %% Data & Grammar Assets
    subgraph "Data & Grammar Assets"
        direction TB
        D1["Grammars"]:::storage
        D2["Huggingface Data"]:::storage
    end

    %% External Services
    subgraph "External LLM Services"
        direction TB
        E1["OpenAI"]:::cloud
        E2["Huggingface API"]:::cloud
        E3["Ollama Server"]:::cloud
        E4["Local LlamaCpp"]:::cloud
    end

    %% Build & CI
    subgraph "Build & CI"
        direction TB
        F1["dune-project"]:::ci
        F2["shell.nix"]:::ci
        F3["flake.nix/flake.lock"]:::ci
        F4["Dockerfile"]:::ci
        F5["Makefile"]:::ci
        F6["build.sh/build2.sh/build_push.sh"]:::ci
        F7[".github/workflows/docker-image.yml"]:::ci
    end

    %% Tests & Examples
    subgraph "Tests & Examples"
        direction TB
        G1["test_lang_agent.ml"]:::test
        G2["test.sh"]:::test
        G3["run_example.sh/2/3"]:::test
        G4["run.sh/run_main.sh"]:::test
    end

    %% Connections
    subgraph interactions
        A1 -->|calls| B1
        A2 --> B1
        A3 --> B1
        A4 --> B1
        A5 --> B1
        A6 --> B1
        A7 --> B1
        A8 --> B1
        A9 --> B1
        A10 --> B1
        A11 --> B1
        A12 --> B1
        A13 --> B1
        A14 --> B1

        B1 -->|dispatch| C1
        B1 --> C2
        B1 --> C3
        B1 --> C4

        C1 --> E1
        C1 <-- E1
        C2 --> E3
        C2 <-- E3
        C3 --> E3
        C3 <-- E3
        C4 --> E4
        C4 <-- E4

        B1 --> D1
        D1 --> B1

        F1 --> A1
        F1 --> B1
        F1 --> C1
        F1 --> D1
        F1 --> G1

        G1 --> B1
    end

    %% Click Events CLI
    click A1 "https://github.com/meta-introspector/lang_agent/blob/main/bin/argiope.ml"
    click A2 "https://github.com/meta-introspector/lang_agent/blob/main/bin/chunker.ml"
    click A3 "https://github.com/meta-introspector/lang_agent/blob/main/bin/execute_self.ml"
    click A4 "https://github.com/meta-introspector/lang_agent/blob/main/bin/generate_dungeon.ml"
    click A5 "https://github.com/meta-introspector/lang_agent/blob/main/bin/generate_generic.ml"
    click A6 "https://github.com/meta-introspector/lang_agent/blob/main/bin/la_decode_json.ml"
    click A7 "https://github.com/meta-introspector/lang_agent/blob/main/bin/poem.ml"
    click A8 "https://github.com/meta-introspector/lang_agent/blob/main/bin/recurse.ml"
    click A9 "https://github.com/meta-introspector/lang_agent/blob/main/bin/scanner.ml"
    click A10 "https://github.com/meta-introspector/lang_agent/blob/main/bin/simple.ml"
    click A11 "https://github.com/meta-introspector/lang_agent/blob/main/bin/simple_grammar.ml"
    click A12 "https://github.com/meta-introspector/lang_agent/blob/main/bin/test1.ml"
    click A13 "https://github.com/meta-introspector/lang_agent/blob/main/bin/worker.ml"
    click A14 "https://github.com/meta-introspector/lang_agent/tree/main/bin/dune"

    %% Click Events Core
    click B1 "https://github.com/meta-introspector/lang_agent/blob/main/lib/lang_model.ml"
    click B2 "https://github.com/meta-introspector/lang_agent/blob/main/lib/model_type.ml"
    click B3 "https://github.com/meta-introspector/lang_agent/blob/main/lib/mythos.ml"
    click B4 "https://github.com/meta-introspector/lang_agent/blob/main/lib/embody.ml"
    click B5 "https://github.com/meta-introspector/lang_agent/blob/main/lib/CoqOfOCaml.v"
    click B6 "https://github.com/meta-introspector/lang_agent/blob/main/lib/StdLib.v"
    click B7 "https://github.com/meta-introspector/lang_agent/blob/main/lib/Yojson.v"
    click B8 "https://github.com/meta-introspector/lang_agent/blob/main/lib/Begin.v"
    click B9 "https://github.com/meta-introspector/lang_agent/blob/main/lib/Extract_lang_model.v"
    click B10 "https://github.com/meta-introspector/lang_agent/blob/main/lib/Test_archetype_instance.ml"
    click B11 "https://github.com/meta-introspector/lang_agent/blob/main/lib/folder.ml"
    click B12 "https://github.com/meta-introspector/lang_agent/blob/main/lib/unix_handling.ml"
    click B13 "https://github.com/meta-introspector/lang_agent/tree/main/lib/dune"

    %% Click Events Adapters
    click C1 "https://github.com/meta-introspector/lang_agent/blob/main/lib/openai_client.ml"
    click C2 "https://github.com/meta-introspector/lang_agent/blob/main/lib/ollama.ml"
    click C3 "https://github.com/meta-introspector/lang_agent/blob/main/lib/ollama_lwt.ml"
    click C4 "https://github.com/meta-introspector/lang_agent/blob/main/lib/llama_cpp.ml"

    %% Click Events Data & Grammar
    click D1 "https://github.com/meta-introspector/lang_agent/tree/main/grammars/"
    click D2 "https://github.com/meta-introspector/lang_agent/tree/main/data/huggingface_lang_agent"

    %% Click Events Build & CI
    click F1 "https://github.com/meta-introspector/lang_agent/tree/main/dune-project"
    click F2 "https://github.com/meta-introspector/lang_agent/blob/main/shell.nix"
    click F3 "https://github.com/meta-introspector/lang_agent/blob/main/flake.nix"
    click F4 "https://github.com/meta-introspector/lang_agent/tree/main/Dockerfile"
    click F5 "https://github.com/meta-introspector/lang_agent/tree/main/Makefile"
    click F6 "https://github.com/meta-introspector/lang_agent/blob/main/build.sh"
    click F7 "https://github.com/meta-introspector/lang_agent/blob/main/.github/workflows/docker-image.yml"

    %% Click Events Tests
    click G1 "https://github.com/meta-introspector/lang_agent/blob/main/test/test_lang_agent.ml"
    click G2 "https://github.com/meta-introspector/lang_agent/blob/main/test.sh"
    click G3 "https://github.com/meta-introspector/lang_agent/blob/main/run_example.sh"
    click G4 "https://github.com/meta-introspector/lang_agent/blob/main/run.sh"

    %% Styles
    classDef cli fill:#FFD580,stroke:#333,stroke-width:1px
    classDef core fill:#80B3FF,stroke:#333,stroke-width:1px
    classDef adapter fill:#FFB3B3,stroke:#333,stroke-width:1px
    classDef storage fill:#C2F0C2,stroke:#333,stroke-width:1px
    classDef cloud fill:#D9D9F3,stroke:#333,stroke-width:1px
    classDef ci fill:#F2CC8F,stroke:#333,stroke-width:1px
    classDef test fill:#E6E6E6,stroke:#333,stroke-width:1px
```
