aliahmed369000000-ai/neural-service-mesh

1. What this repo is

Neural Service Mesh is a local, modular AI system with a graph-based runtime, multiple cognitive engines, a knowledge ingestion layer, sensor inputs, and persistence for training/checkpoints. It is not just one app; it is a platform for running, training, and evolving a “mesh” of AI capabilities.

2. Primary entrypoints

The runtime appears to start from main.py, with additional execution surfaces in api/app.py (Flask API), Streamlit_app.py / streamlit_app.py (UI), wsgi.py (deployment entrypoint), and helper scripts like train_simulate.py, fix_and_run.py, and restore_weights.py.

3. Core execution model

The core/ package is the structural backbone: engine.py, graph.py, node.py, and registry.py suggest a node-and-graph orchestration model where capabilities are registered as nodes, connected in a graph, and executed by a central engine.

4. AI/cognition subsystem

The ai/ package is the main intelligence layer. It includes planning, reasoning, routing, optimization, memory, scoring, validation, and governance components. The Phase 7 additions called out in the README are especially important: evolution_pipeline.py, governance_p7.py, code_generator.py, objectives.py, phase6_validator.py, sandbox_lab.py, and self_awareness.py. These suggest the system can propose changes, validate them, and evolve itself under policy constraints.

5. Memory and learning

The repo has several overlapping learning/memory mechanisms: ai/memory_engine.py, episodic_memory.py, memory_consolidator.py, experience_store.py, experience_trainer.py, and brain_checkpoint.py. Persistent memory also lives outside code in memory/episodic.db and multiple JSON/NPY checkpoint files under checkpoints/.

6. Sensors and input capture

sensors/ defines the system’s inbound observation layer. api_sensor.py, filesystem_sensor.py, log_sensor.py, and webhook_sensor.py feed into sensor_hub.py, meaning the mesh can ingest events from APIs, files, logs, and webhooks rather than only interactive user input.

7. Services and processing pipeline

services/ provides the operational layer that handles incoming work: input_service.py, processor_service.py, output_service.py, and dynamic_node.py. Architecturally, this is likely the bridge between raw sensor events and the core graph/AI modules.

8. Knowledge subsystem

The knowledge/ package stores and reasons over domain data. It includes knowledge_store.py, cognitive_graph.py, qa_engine.py, relation_inferencer.py, and Quran-specific training/indexing artifacts such as quran_index.json, many quran_chunk_*.json files, and quran_continuous_trainer.py. This layer is clearly intended to build a structured knowledge graph and answer/query engine.

9. Knowledge ingestion and source management

knowledge_sources/ is the ingestion pipeline into the knowledge layer. It has a base source abstraction plus source-specific modules for Quran data, web fetching, concept extraction, domain sources, metadata, tracking, validation, and quality scoring. This is the boundary where external content is normalized into the internal knowledge model.

10. Storage and persistence

storage/db.py and storage/file_storage.py handle durable state, while data/ holds runtime JSON artifacts and SQLite state such as mesh.db. The repo also tracks model state in weights/, checkpoints/, and NumPy files like nsm_embedding.npz and dynamic_layer.npy, which are part of the system’s learned state rather than source code.

11. World/simulation boundary

world_model/environment_model.py and world_model/environment.json define a simulated or abstract environment used by the AI/training loop. This is a separate boundary from the knowledge store: knowledge represents facts and relations, while the world model represents stateful dynamics or scenarios.

12. Observability and audit trail

logs/mesh_logger.py and the many JSON logs in data/ indicate the system is designed to record activity, training results, source tracking, and graph metrics. This is important for understanding how decisions, ingestion, and evolution are audited.

13. UI and human interaction

The repo includes both a Streamlit-facing UI and an HTML dashboard (dashboard.html, NSM_Agent_v8.html, nsm_chat_ui.py, nsm_chat.py). These are likely thin presentation layers over the core engine and API, rather than the center of the architecture.

14. Overall data flow

The most likely flow is: sensors and external sources ingest data → connectors/services transform it → core graph/engine routes work to AI modules → knowledge and memory layers persist results → storage/checkpoints retain state → logs/metrics capture behavior → the evolution/training pipeline updates models and policies. The architecture is therefore a closed loop of observation, reasoning, storage, and self-improvement.