How extorc/ecoint works
Purpose and entry point
This is a Python pipeline for turning RSS content into a Neo4j knowledge graph. The likely executable entry point is main.py, with workflow composition centralized in graph.py. Runtime configuration, including service credentials and model settings, is expected in config.py and .env.template.
Workflow orchestration
graph.py defines the pipeline/control graph and the ordering of processing stages. main.py should initialize configuration and invoke that graph. The nodes/ package contains independently implemented workflow stages, indicating an explicit node-based orchestration design rather than a monolithic script.
Paths: main.py, graph.py, nodes/__init__.py
RSS ingestion boundary
The ingestion stage retrieves source material from RSS feeds. It is the boundary between external feed/network data and the internal processing pipeline, and likely normalizes feed entries into a document/article representation consumed by downstream nodes.
Path: nodes/fetch_rss.py
Knowledge extraction stage
Extracted RSS text is converted into structured knowledge—such as entities, relationships, claims, or topic metadata—by the knowledge-extraction node. This is the semantic transformation boundary and may call an external LLM or NLP provider according to settings in config.py.
Paths: nodes/extract_knowledge.py, config.py
Entity embedding stage
Entities produced during extraction are embedded into vector representations, enabling semantic similarity, deduplication, or vector-indexed retrieval. Embedding is distinct from extraction, so downstream persistence can store both graph structure and semantic vectors.
Path: nodes/embed_entities.py
Graph persistence
The final storage stage writes knowledge graph records to Neo4j. It is the infrastructure boundary for database connectivity, schema/label conventions, upsert behavior, and relationship persistence. Neo4j is the material external service implied by the repository structure.
Path: nodes/store_neo4j.py
Primary data flow
Control and data should move in this order: RSS feed retrieval → knowledge extraction → entity embedding → Neo4j storage. graph.py coordinates stage transitions; each file under nodes/ owns one transformation or side-effect boundary. Configuration is supplied through environment-derived values rather than hardcoded infrastructure settings.
Paths: graph.py, nodes/fetch_rss.py, nodes/extract_knowledge.py, nodes/embed_entities.py, nodes/store_neo4j.py
Main entry points: extract_knowledge.py Extract knowledge · config.py Runtime config · graph.py Pipeline graph · fetch_rss.py Fetch RSS
Components 11
Workflow
- Main entry point · Initializes runtime and starts the pipeline.
main.py - Pipeline graph · Defines stage ordering and transitions.
graph.py - Node package
nodes/__init__.py - Runtime config · Provides environment-derived credentials and model settings.
config.py
Pipeline stages
- Fetch RSS · Retrieves and normalizes feed entries.
nodes/fetch_rss.py - Extract knowledge · Converts article text into structured knowledge.
nodes/extract_knowledge.py - Embed entities · Creates semantic vectors for extracted entities.
nodes/embed_entities.py - Store in Neo4j · Upserts graph records and entity vectors.
nodes/store_neo4j.py
External services
- RSS feeds
- LLM / NLP provider
- Neo4j · Stores knowledge graph structure and vectors.
Connections 12
- Main entry point → Runtime config: loads
- Main entry point → Pipeline graph: invokes
- Node package → Pipeline graph: provides stages
- Pipeline graph → Fetch RSS: starts
- RSS feeds → Fetch RSS: feed entries
- Fetch RSS → Extract knowledge: normalized articles
- Runtime config → Extract knowledge: model settings
- Extract knowledge → LLM / NLP provider: extracts via
- Extract knowledge → Embed entities: entities and relations
- Embed entities → Store in Neo4j: knowledge with vectors
- Runtime config → Store in Neo4j: database credentials
- Store in Neo4j → Neo4j: upserts