How reemasaif/ndi-ai-agent-capstone-project works
1. What this repo is
- NDI-Sentinel is a local, document-driven governance and compliance system for monitoring NDMO / NDI alignment.
- The repository combines policy retrieval, compliance analysis, scoring, report generation, and a dashboard for visibility.
2. Core architecture pattern
- The system is organized as a pipeline around three kinds of artifacts:
policy/knowledge documents, operational data, and generated compliance outputs.
- Most components appear to run as offline or batch jobs rather than a continuously hosted cloud service.
3. Knowledge and retrieval layer
- RAG/Vectors/db_setup.py and the persisted chroma_db/ folder indicate a Chroma-based vector store.
- Policy PDFs in src/knowledge_base/ and data/ are the main retrieval corpus.
- RAG/Retrieval/query_agent.py is the retrieval entry point that likely turns policy questions or compliance checks into vector search queries.
4. Compliance analysis layer
- src/compliance_agent.py is the central compliance reasoning component.
- It likely combines retrieved policy context with source data from files and/or the database to detect gaps, classify records, or produce evidence for compliance evaluation.
- The presence of data/security_document.txt, data/top_secret_document.txt, data/medical_report.txt, and data/public_announcement.txt suggests the agent evaluates sensitivity or policy adherence across mixed document types.
5. Scoring subsystem
- The scoring/ package is a separate evaluation pipeline for turning findings into numeric or categorical maturity/compliance scores.
- scoring/scoring_engine.py appears to hold the scoring rules.
- scoring/score_db.py and scoring/score_export.py connect scoring results to storage and export, while scoring/main_execution.py likely orchestrates the full scoring run.
6. Relational database boundary
- The database/ directory contains schema creation scripts, connection handling, and mock data generation.
- This is the structured-data side of the system, distinct from the vector store used for policy retrieval.
- database/test_db_connection.py indicates the database is a first-class dependency and likely used to validate environment setup before analysis.
7. Reporting layer
- The reporting/ package converts analysis results into human-readable artifacts, especially PDFs.
- reporting/pdf_builder.py, reporting/layouts.py, and reporting/font.py suggest custom document formatting, likely including Arabic-aware output.
- reporting/run_report.py is the orchestration entry point for generating final deliverables.
8. Strategic advisor outputs
- Strategic Advisor/advisor_agent.py appears to be a higher-level synthesis agent.
- Its outputs in Strategic Advisor/Outputs/ include governance_report.txt and final_report.txt, which likely summarize technical findings into executive guidance.
- This is a distinct layer from raw compliance scoring: it turns evidence into strategic recommendations.
9. Dashboard / presentation boundary
- dashboard/app.py is the user-facing app for viewing results.
- It likely reads from generated outputs, the database, or report artifacts rather than recomputing compliance from scratch.
- This keeps presentation separate from analysis and makes the dashboard a consumer of pipeline outputs.
10. Data flow across the system
- Source documents and records enter through the data/, src/knowledge_base/, and database/ layers.
- Policies are embedded into Chroma for retrieval.
- The compliance agent queries retrieved policy context, scoring converts findings into metrics, and reporting produces PDFs/JSON/text summaries.
- The dashboard and strategic advisor consume those derived artifacts.
11. Artifacts and persistence
- The repo includes generated outputs in output/, outputs/, and Strategic Advisor/Outputs/, showing the system persists analysis results as files.
- chroma_db/ is a durable local index for semantic retrieval.
- This makes the project reproducible without requiring a remote service dependency.
12. Important technology signals
- The stack is Python-based, with Chroma for vector search, a relational database layer, PDF generation tooling, and file-based reporting.
- Arabic document names and Arabic report outputs indicate multilingual compliance support is an architectural requirement, not just a content detail.
13. How to draw the architecture
- Model it as three major zones:
Data/Knowledge sources → Analysis and scoring services → Outputs and visualization.
- Place the vector database beside the policy corpus, the relational database beside operational records, and connect both into the compliance/scoring pipeline.
- Treat the dashboard and executive advisor as downstream consumers of generated findings.
Main entry points: compliance_agent.py Compliance agent · score_export.py Score export · scoring_engine.py Scoring engine · database Database layer
Components 18
Data & knowledge
Policy documents, operational records, and persisted semantic index.
- Policy corpus · PDF policy and guidance documents used as retrieval knowledge.
src/knowledge_base - Operational data · Mixed sensitive and public sample records used for compliance evaluation.
data - Chroma store · Persistent local embedding store for policy retrieval.
chroma_db - Database layer · Structured database setup, schema, and seeded records.
database - DB connection · Database connection and environment validation boundary.
database/connection.py
Retrieval & compliance
Vector retrieval and compliance reasoning over retrieved policy context.
- Query agent · Turns policy questions into vector search and retrieved context.
RAG/Retrieval/query_agent.py - Vector setup · Creates and initializes the Chroma vector database.
RAG/Vectors/db_setup.py - Compliance agent · Core compliance analysis component combining retrieved policy with source records.
src/compliance_agent.py
Scoring & reports
Scoring, exports, and document generation from analysis results.
- Scoring engine · Converts compliance findings into scores and maturity signals.
scoring/scoring_engine.py - Score store · Stores scoring outputs for reuse and export.
scoring/score_db.py - Score export · Formats score results into exportable artifacts.
scoring/score_export.py - Scoring run · Coordinates the full scoring execution flow.
scoring/main_execution.py - PDF builder · Builds formatted compliance and governance PDFs.
reporting/pdf_builder.py - Layouts · Document layout and rendering rules for report generation.
reporting/layouts.py - Report runner · Runs the reporting pipeline end to end.
reporting/run_report.py
Advisor & dashboard
Downstream consumers for executive synthesis and viewing.
- Advisor agent · Produces executive governance guidance from analysis outputs.
Strategic Advisor/advisor_agent.py - Dashboard · User-facing app for viewing results and outputs.
dashboard/app.py - Advisor outputs · Persisted strategic summaries and final governance reports.
Strategic Advisor/Outputs
Connections 22
- Policy corpus → Vector setup: embed · Policy PDFs are indexed into the local vector store.
- Vector setup → Chroma store: persist · Vector setup writes the durable Chroma index.
- Operational data → Database layer: load · Operational records are loaded into the relational layer.
- DB connection → Database layer: connect · Connection module mediates access to structured data.
- Chroma store → Query agent: search · Retriever queries the semantic index for relevant policy context.
- Query agent → Compliance agent: context · Retrieved policy evidence feeds compliance reasoning.
- Operational data → Compliance agent: records · Source documents and records are evaluated by the compliance agent.
- Database layer → Compliance agent: structured data · Compliance analysis can draw on relational records and mock data.
- Compliance agent → Scoring engine: findings · Compliance findings are translated into scores.
- Scoring engine → Score store: store · Computed scores are persisted for downstream use.
- Score store → Score export: export · Stored results are formatted for external consumption.
- Scoring engine → Scoring run: run · Main execution coordinates scoring rules and outputs.
- Scoring run → Score export: emit · Pipeline emits exportable scoring artifacts.
- Compliance agent → PDF builder: evidence · Analysis results provide the basis for generated reports.
- Layouts → PDF builder: format · Layout rules shape report presentation.
- PDF builder → Report runner: generate · Builder is invoked by the reporting orchestrator.
- Score export → Report runner: inputs · Reporting consumes exported score artifacts.
- Report runner → Advisor outputs: write · Reporting writes final deliverables and summaries.
- Compliance agent → Advisor agent: synthesize · Strategic advisor consumes compliance findings for executive guidance.
- Advisor agent → Advisor outputs: publish · Advisor writes governance and final reports.
- Score export → Dashboard: view · Dashboard reads prepared outputs for presentation.
- Advisor outputs → Dashboard: consume · Dashboard can surface strategic summaries and final reports.