kamrulhasan12345/nodecommerce-backend

1. What this repo is

This is a Python 3.14 monorepo for a Bangladesh demand-intelligence platform. The main production loop is: transactions and inventory data go into PostgreSQL, feature jobs build typed features, LightGBM produces forecasts, agent signals adjust them, and the API serves dashboard/forecast reads.

2. Shared platform boundary

packages/core/ is the common foundation: config, logging, DB clients, Redis client, Neo4j client, and the canonical layer-oriented models. Most other packages depend on it rather than talking to infrastructure directly.

3. Primary storage and infrastructure

The runtime infrastructure is PostgreSQL 16, Redis 7, and Neo4j Community, with Prometheus/Grafana for observability. Docker Compose and infra/ define local service setup, and Alembic migrations under packages/migrations/ own schema evolution.

4. Raw-data and seeding path

The raw layer includes nodes, products, transactions, customers, and inventory state. data_generation/ plus scripts/ produce synthetic/demo datasets and coordinate metadata so the system can be seeded without the full ML pipeline.

5. Feature engineering subsystem

packages/feature_store/ is the main feature computation layer. It builds calendar, lag, node/product, event, graph, cold-start, and agent-aggregate features, then writes them to the feature store tables that downstream inference consumes.

6. Forecasting subsystem

packages/forecasting/ owns model training, inference, imputation, trust update logic, and persistence of model artifacts and predictions. The README indicates LightGBM is the main model, and daily inference also emits SHAP-style explanation components for the dashboard.

7. Agentic signal subsystem

packages/agents/ splits into local and central agents. Local agents ingest or scrape signals, apply guardrails, and write back local signal data; central agents preprocess, detect spikes, and write centralized findings. This layer is partially migrated to the v3 schema and still has some transitional code.

8. Knowledge-graph subsystem

packages/knowledge_graph/ wraps Neo4j-backed graph data. It has separate EKG and SKG writer/query modules, embedding support, and materialization logic. feature_store can consume graph-derived signals, and some pipeline jobs are intended to refresh graph features hourly.

9. API and dashboard serving boundary

packages/api/ is the HTTP serving layer. It exposes dashboard, forecast, inventory, signal, health, ACO, and HITL routers, and serves the static Leaflet-based investor dashboard from packages/api/src/api/static/. The read path is mostly PostgreSQL-backed; no model training happens here.

10. Pipeline orchestration

packages/pipeline/ contains the scheduled job entry points and Airflow DAG definitions. The active path is job-centric: daily feature generation, daily inference, weekly training, weekly recsys, hourly graph-RAG refresh, and monitoring. The README notes some Airflow DAG files are stale or partially incompatible with v3 schemas.

11. Recommendations and supporting analytics

packages/recsys/ builds product affinity scores and cold-start support used by the forecasting/feature layers. packages/monitoring/ computes drift, data quality, health checks, and metrics that feed Prometheus/Grafana dashboards and alerting rules.

12. Legacy and not-currently-primary subsystems

packages/aco/ and packages/router/ represent the older fulfillment-routing direction. The repository treats them as v2/legacy and not part of the current v3 production flow, while the README’s “hierarchical fulfillment routing” is described as planned future work rather than an active subsystem.