# kamrulhasan12345/nodecommerce-backend architecture

> Architecture diagram of the GitHub repository kamrulhasan12345/nodecommerce-backend, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kamrulhasan12345/nodecommerce-backend
- Repository: https://github.com/kamrulhasan12345/nodecommerce-backend

## Overview

1. **What this repo is**
   This is a Python 3.14 monorepo for a Bangladesh demand-intelligence platform. The main production loop is: transactions and inventory data go into PostgreSQL, feature jobs build typed features, LightGBM produces forecasts, agent signals adjust them, and the API serves dashboard/forecast reads.

2. **Shared platform boundary**
   `packages/core/` is the common foundation: config, logging, DB clients, Redis client, Neo4j client, and the canonical layer-oriented models. Most other packages depend on it rather than talking to infrastructure directly.

3. **Primary storage and infrastructure**
   The runtime infrastructure is PostgreSQL 16, Redis 7, and Neo4j Community, with Prometheus/Grafana for observability. Docker Compose and `infra/` define local service setup, and Alembic migrations under `packages/migrations/` own schema evolution.

4. **Raw-data and seeding path**
   The raw layer includes nodes, products, transactions, customers, and inventory state. `data_generation/` plus `scripts/` produce synthetic/demo datasets and coordinate metadata so the system can be seeded without the full ML pipeline.

5. **Feature engineering subsystem**
   `packages/feature_store/` is the main feature computation layer. It builds calendar, lag, node/product, event, graph, cold-start, and agent-aggregate features, then writes them to the feature store tables that downstream inference consumes.

6. **Forecasting subsystem**
   `packages/forecasting/` owns model training, inference, imputation, trust update logic, and persistence of model artifacts and predictions. The README indicates LightGBM is the main model, and daily inference also emits SHAP-style explanation components for the dashboard.

7. **Agentic signal subsystem**
   `packages/agents/` splits into local and central agents. Local agents ingest or scrape signals, apply guardrails, and write back local signal data; central agents preprocess, detect spikes, and write centralized findings. This layer is partially migrated to the v3 schema and still has some transitional code.

8. **Knowledge-graph subsystem**
   `packages/knowledge_graph/` wraps Neo4j-backed graph data. It has separate EKG and SKG writer/query modules, embedding support, and materialization logic. `feature_store` can consume graph-derived signals, and some pipeline jobs are intended to refresh graph features hourly.

9. **API and dashboard serving boundary**
   `packages/api/` is the HTTP serving layer. It exposes dashboard, forecast, inventory, signal, health, ACO, and HITL routers, and serves the static Leaflet-based investor dashboard from `packages/api/src/api/static/`. The read path is mostly PostgreSQL-backed; no model training happens here.

10. **Pipeline orchestration**
    `packages/pipeline/` contains the scheduled job entry points and Airflow DAG definitions. The active path is job-centric: daily feature generation, daily inference, weekly training, weekly recsys, hourly graph-RAG refresh, and monitoring. The README notes some Airflow DAG files are stale or partially incompatible with v3 schemas.

11. **Recommendations and supporting analytics**
    `packages/recsys/` builds product affinity scores and cold-start support used by the forecasting/feature layers. `packages/monitoring/` computes drift, data quality, health checks, and metrics that feed Prometheus/Grafana dashboards and alerting rules.

12. **Legacy and not-currently-primary subsystems**
    `packages/aco/` and `packages/router/` represent the older fulfillment-routing direction. The repository treats them as v2/legacy and not part of the current v3 production flow, while the README’s “hierarchical fulfillment routing” is described as planned future work rather than an active subsystem.

Main entry points: [`packages/pipeline/src/pipeline/jobs/daily_inference.py`](https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/pipeline/src/pipeline/jobs/daily_inference.py) (Jobs), [`packages/core/src/core/__init__.py`](https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/core/src/core/__init__.py) (Core), [`packages/feature_store/src/feature_store/main.py`](https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/feature_store/src/feature_store/main.py) (Feature store), [`packages/knowledge_graph/src/knowledge_graph/__init__.py`](https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/knowledge_graph/src/knowledge_graph/__init__.py) (Knowledge graph)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_shared["Shared platform"]
  node_core["Core<br/>shared foundation<br/>[__init__.py]"]
end

subgraph group_group_data["Data and storage"]
  node_migrations["Migrations<br/>schema evolution<br/>[env.py]"]
  node_seed_data["Seed data<br/>synthetic data<br/>[README.md]"]
end

subgraph group_group_ml["Feature and forecasting"]
  node_feature_store["Feature store<br/>feature engineering<br/>[main.py]"]
  node_forecasting["Forecasting<br/>modeling<br/>[pipeline.py]"]
  node_recsys["Recsys<br/>recommendation<br/>[pipeline.py]"]
end

subgraph group_group_agents["Agentic and graph"]
  node_agents_local["Local agents<br/>signal ingestion<br/>[agent.py]"]
  node_agents_central["Central agents<br/>signal processing<br/>[agent.py]"]
  node_knowledge_graph[("Knowledge graph<br/>neo4j graph layer<br/>[__init__.py]")]
end

subgraph group_group_serve["Serving"]
  node_api["API<br/>http serving<br/>[main.py]"]
  node_dashboard_router["Dashboard<br/>read router<br/>[dashboard_v3.py]"]
  node_forecast_router["Forecasts<br/>read router<br/>[forecasts.py]"]
end

subgraph group_group_orch["Orchestration"]
  node_pipeline_jobs["Jobs<br/>[daily_inference.py]"]
  node_airflow_dags["Airflow DAGs<br/>workflow definitions"]
end

subgraph group_group_monitor["Monitoring"]
  node_monitoring["Monitoring<br/>metrics and drift<br/>[__init__.py]"]
end

subgraph group_group_legacy["Legacy"]
  node_aco["ACO<br/>legacy routing<br/>[coordinator.py]"]
  node_router["Router<br/>legacy routing<br/>[router.py]"]
end

node_seed_data -->|"seed schema"| node_migrations
node_migrations -.->|"models"| node_core
node_agents_local -->|"write signals"| node_knowledge_graph
node_agents_central -->|"write findings"| node_knowledge_graph
node_knowledge_graph -->|"graph features"| node_feature_store
node_core -->|"db clients"| node_feature_store
node_core -->|"db clients"| node_forecasting
node_core -->|"clients"| node_agents_local
node_core -->|"clients"| node_agents_central
node_core -->|"neo4j client"| node_knowledge_graph
node_recsys -->|"support features"| node_feature_store
node_feature_store -->|"typed features"| node_forecasting
node_feature_store -.->|"quality signals"| node_monitoring
node_forecasting -->|"predictions"| node_api
node_api -->|"routes"| node_dashboard_router
node_api -->|"routes"| node_forecast_router
node_pipeline_jobs -->|"daily features"| node_feature_store
node_pipeline_jobs -->|"daily inference"| node_forecasting
node_pipeline_jobs -->|"central agent job"| node_agents_central
node_pipeline_jobs -->|"graph refresh"| node_knowledge_graph
node_pipeline_jobs -->|"weekly recsys"| node_recsys
node_pipeline_jobs -->|"monitor jobs"| node_monitoring
node_airflow_dags -.->|"orchestrate"| node_pipeline_jobs
node_monitoring -.->|"health data"| node_api
node_aco -.->|"legacy flow"| node_router

click node_core "https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/core/src/core/__init__.py"
click node_migrations "https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/migrations/alembic/env.py"
click node_seed_data "https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/data_generation/README.md"
click node_feature_store "https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/feature_store/src/feature_store/main.py"
click node_forecasting "https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/forecasting/src/forecasting/pipeline.py"
click node_recsys "https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/recsys/src/recsys/pipeline.py"
click node_agents_local "https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/agents/src/agents/local/agent.py"
click node_agents_central "https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/agents/src/agents/central/agent.py"
click node_knowledge_graph "https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/knowledge_graph/src/knowledge_graph/__init__.py"
click node_api "https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/api/src/api/main.py"
click node_dashboard_router "https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/api/src/api/routers/dashboard_v3.py"
click node_forecast_router "https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/api/src/api/routers/forecasts.py"
click node_pipeline_jobs "https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/pipeline/src/pipeline/jobs/daily_inference.py"
click node_airflow_dags "https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/pipeline/src/pipeline/airflow/dag_daily_inference.py"
click node_monitoring "https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/monitoring/src/monitoring/__init__.py"
click node_aco "https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/aco/src/aco/coordinator.py"
click node_router "https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/router/src/router/router.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_core,node_monitoring toneBlue
class node_migrations,node_seed_data,node_aco,node_router toneAmber
class node_feature_store,node_forecasting,node_recsys toneMint
class node_agents_local,node_agents_central,node_knowledge_graph toneRose
class node_api,node_dashboard_router,node_forecast_router toneIndigo
class node_pipeline_jobs,node_airflow_dags toneTeal
```

## Components

### Shared platform

Common foundation and cross-cutting infrastructure clients

- [Core](https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/core/src/core/__init__.py): shared foundation · Config, logging, DB/Redis/Neo4j clients, and canonical models · `packages/core/src/core/__init__.py`

### Data and storage

Raw data seeding and schema evolution

- [Migrations](https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/migrations/alembic/env.py): schema evolution · Alembic-managed PostgreSQL schema changes · `packages/migrations/alembic/env.py`
- [Seed data](https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/data_generation/README.md): synthetic data · Generators and demo dataset creation for raw-layer bootstrapping · `data_generation/README.md`

### Feature and forecasting

Feature computation, forecasting, and recommendation models

- [Feature store](https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/feature_store/src/feature_store/main.py): feature engineering · Builds typed calendar, lag, node, product, event, graph, cold-start, and agent features · `packages/feature_store/src/feature_store/main.py`
- [Forecasting](https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/forecasting/src/forecasting/pipeline.py): modeling · Training, inference, imputation, trust updates, and prediction persistence · `packages/forecasting/src/forecasting/pipeline.py`
- [Recsys](https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/recsys/src/recsys/pipeline.py): recommendation · Product affinity and cold-start support used by downstream layers · `packages/recsys/src/recsys/pipeline.py`

### Agentic and graph

Local/central agents and Neo4j-backed knowledge graph

- [Local agents](https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/agents/src/agents/local/agent.py): signal ingestion · Local ingestion, guardrails, tools, graph-RAG, and writeback · `packages/agents/src/agents/local/agent.py`
- [Central agents](https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/agents/src/agents/central/agent.py): signal processing · Scrape, preprocess, spike detection, and central finding writeback · `packages/agents/src/agents/central/agent.py`
- [Knowledge graph](https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/knowledge_graph/src/knowledge_graph/__init__.py): neo4j graph layer · EKG/SKG writers, queries, embeddings, and materialization · `packages/knowledge_graph/src/knowledge_graph/__init__.py`

### Serving

HTTP API and dashboard reads

- [API](https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/api/src/api/main.py): http serving · FastAPI boundary for dashboard, forecast, inventory, signal, and health reads · `packages/api/src/api/main.py`
- [Dashboard](https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/api/src/api/routers/dashboard_v3.py): read router · Dashboard and static investor UI endpoints · `packages/api/src/api/routers/dashboard_v3.py`
- [Forecasts](https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/api/src/api/routers/forecasts.py): read router · Forecast and model read endpoints · `packages/api/src/api/routers/forecasts.py`

### Orchestration

Scheduled jobs and workflow entry points

- [Jobs](https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/pipeline/src/pipeline/jobs/daily_inference.py): scheduled jobs · Daily, weekly, hourly, and monitoring job entry points · `packages/pipeline/src/pipeline/jobs/daily_inference.py`
- [Airflow DAGs](https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/pipeline/src/pipeline/airflow/dag_daily_inference.py): workflow definitions · Workflow scheduling layer, partly stale relative to v3 schemas · `packages/pipeline/src/pipeline/airflow/dag_daily_inference.py`

### Monitoring

Health, drift, metrics, and alerting

- [Monitoring](https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/monitoring/src/monitoring/__init__.py): metrics and drift · Data quality, drift, health checks, metrics, and alerting inputs · `packages/monitoring/src/monitoring/__init__.py`

### Legacy

Older subsystems kept outside the main v3 flow

- [ACO](https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/aco/src/aco/coordinator.py): legacy routing · Older fulfillment-routing subsystem kept as v2/legacy · `packages/aco/src/aco/coordinator.py`
- [Router](https://github.com/kamrulhasan12345/nodecommerce-backend/blob/main/packages/router/src/router/router.py): legacy routing · Legacy routing and HITL queue path outside the main v3 loop · `packages/router/src/router/router.py`

## Connections

- Seed data → Migrations: seed schema · Synthetic data and metadata align with evolved tables
- Migrations → Core: models · Schema and table shape follow canonical shared models
- Local agents → Knowledge graph: write signals · Local agents can persist graph-derived or graph-adjacent signals
- Central agents → Knowledge graph: write findings · Central agents materialize findings into Neo4j-backed graph data
- Knowledge graph → Feature store: graph features · Graph materialization feeds graph-derived feature computation
- Core → Feature store: db clients · Feature jobs use shared clients and models
- Core → Forecasting: db clients · Forecasting reads/writes through shared infrastructure clients
- Core → Local agents: clients · Local agent runtime uses common config and connectors
- Core → Central agents: clients · Central agent runtime uses common config and connectors
- Core → Knowledge graph: neo4j client · Graph layer depends on the shared Neo4j connector
- Recsys → Feature store: support features · Affinity and cold-start outputs feed feature computation
- Feature store → Forecasting: typed features · Downstream inference consumes stored engineered features
- Feature store → Monitoring: quality signals · Feature jobs expose drift and data-quality inputs
- Forecasting → API: predictions · API serves forecast reads from persisted predictions
- API → Dashboard: routes · Dashboard endpoints and static UI live behind the API boundary
- API → Forecasts: routes · Forecast read endpoints are mounted in the API
- Jobs → Feature store: daily features · Scheduled jobs invoke feature generation
- Jobs → Forecasting: daily inference · Scheduled jobs run inference and persist outputs
- Jobs → Central agents: central agent job · Scheduled jobs refresh centralized agent signals
- Jobs → Knowledge graph: graph refresh · Hourly jobs refresh graph-RAG/materialized graph data
- Jobs → Recsys: weekly recsys · Recommendation training and scoring are scheduled here
- Jobs → Monitoring: monitor jobs · Operational jobs collect health, drift, and quality signals
- Airflow DAGs → Jobs: orchestrate · Airflow definitions wrap the job entry points
- Monitoring → API: health data · Metrics and health checks surface through API read paths
- ACO → Router: legacy flow · Older routing components remain connected as a legacy path