How fartlabs/turtle-agent works
Project type and purpose
• This is a Deno-based, modular CLI/library “turtle-agent” for generating, disambiguating, linking, storing, and searching RDF/Turtle knowledge graphs from natural-language input. It’s an open-source pipeline tool rather than a web application.
File structure
• Top-level: deno.json (Deno config), README.md, .vscode.
• src/ dissects into six primary domains:
• ner/ – named-entity recognition interface and implementation
• disambiguator/ – entity disambiguation logic plus a CLI entrypoint
• linker/ – linking recognized/disambiguated entities to knowledge bases
• generator/ – context setup, prompt formatting, and graph generation modules
• n3store/ – in-memory/Turtle store with persistence plugins (FS, KV) and SHACL validation, plus interceptor middleware (count, debug, error, Orama sync)
• search/ – full-text (Orama) and SPARQL search adapters, with their own persist and triple-store integrations
• Test files sit alongside their implementations.
README for additional insights
• Purpose statement: “Generate knowledge graphs with natural language.”
• Deno tasks for test and lint hint at a CI pipeline but no deployment files.
a. Main components
• Front-end/CLI: user interface for submitting text (e.g., disambiguator CLI or direct library calls)
• NER Engine: extracts raw entities from text
• Disambiguator: refines entity mentions to canonical identifiers
• Linker: binds entities to external KBs/URIs
• Generator: builds RDF triples using few-shot contexts and formatting rules
• N3 Store: holds triples, applies SHACL validation, persists to FS or KV, and fires interceptors
• Search Layer: supports full-text (Orama) and SPARQL queries against the stored graph
• Configuration/Infra: Deno runtime, deno.json, persistence backends (FS, KV), and test harnesses
b. Relationships and interactions
1. User → CLI/Library API
2. CLI/API calls NER → outputs entity spans
3. Output → Disambiguator → IDs
4. IDs → Linker → external URIs
5. URIs + context → Generator → RDF triples
6. Triples → N3Store → interceptors → persistence
7. End user or other clients → Search Layer → fetch via Orama or SPARQL
c. Architectural patterns and principles
• Modular pipeline: each major concern lives in its own directory with clear inputs/outputs
• Middleware/interceptor pattern in n3store for cross-cutting concerns (counting, logging, error handling, sync)
• Adapter pattern in search (Orama adapter vs. SPARQL adapter) and persistence (FS vs. KV)
• Few-shot prompt contexts for pluggable generator strategies
d. Key technologies/frameworks
• Deno + TypeScript
• Orama (full-text search engine)
• SHACL for schema validation
• Simple file system and key-value persistence layers
• Potential external LLM service (via generator context)
Detail without overthinking
• With its primary responsibility and any sub-modules (e.g., interceptor types)