How haaroonafroz/legal_compliance_agent works
1. Overall shape
- legal-agent is a Python 3.11+ compliance pipeline with two major stages: regulatory ingestion and compliance analysis.
- The system is organized around a local CLI, a Qdrant-backed document store, a Scrapy crawler, and a LlamaIndex event-driven workflow.
2. Primary runtime entrypoint
- src/legal_agent/cli.py is the main control surface.
- It exposes operational commands such as database initialization, policy ingestion, scraping, workflow execution, and status checks.
- config.py centralizes environment-backed settings, so runtime behavior is mostly driven by .env and external service credentials.
3. Persistent data boundary
- Qdrant is the key persistence layer and retrieval index.
- The system uses two collections:
- regulatory_updates for scraped regulation chunks plus processing state
- internal_policies for chunked internal policy text used during retrieval
- db/client.py and db/schemas.py define the Qdrant connection and collection/index schema.
4. Regulatory harvesting subsystem
- src/legal_agent/scraping/ contains a Scrapy-based crawler.
- spiders/regulatory_spider.py reads crawl targets from data/sources/targets.json.
- The crawl path is designed to collect both HTML and PDF sources, then normalize them into chunked, metadata-rich records for storage.
5. Document ingestion and chunking
- The scraping pipeline uses items and pipelines to move extracted content through chunking, enrichment, and upsert.
- PDFs are converted through Docling before chunking, which matters because the architecture treats PDFs and HTML as separate extraction problems but a common downstream storage format.
- utils/loader.py and utils/models.py support document parsing and structured representation.
6. Policy ingestion path
- Internal policies are ingested separately from regulatory scraping, typically from PDFs in data/policies/.
- This path produces embeddings and stores chunks in the internal_policies Qdrant collection.
- Keeping policy ingestion separate from regulatory ingestion creates a clean boundary between “system of record” policy content and external regulatory updates.
7. Compliance workflow subsystem
- src/legal_agent/workflow/ implements the analysis phase as an explicit LlamaIndex workflow rather than a free-form agent loop.
- workflow.py contains the main ComplianceWorkflow, while events.py defines typed step-to-step events.
- The workflow steps are:
- Horizon Scanner
- Librarian
- Relevance Check
- Analyst
- Redliner
- Auditor
- This stepwise design is the core architectural choice for traceability and evaluability.
8. Retrieval and reasoning flow
- The workflow begins by grouping staged regulatory chunks back into complete documents.
- It then retrieves relevant policy chunks from Qdrant using hybrid retrieval, combining semantic and lexical signals.
- Retrieved context feeds the comparison step, which generates a gap analysis, then a redline-style policy draft, and finally an audit pass that can reject weak or unsupported output.
9. Model routing and external AI services
- workflow/llm_provider.py and workflow/prompts.py encapsulate model selection and prompt design.
- The repo supports OpenAI by default, with optional Gemini for workflow steps and VoyageAI for reranking.
- This is architecturally important because different steps have different cost/quality requirements: cheaper models can be used for filtering, while stronger models are reserved for analysis, drafting, and auditing.
10. Observability and evaluation
- instrumentation/setup.py wires trace/monitoring integrations such as Langfuse, Phoenix, and LangWatch.
- The architecture is intentionally inspectable: intermediate artifacts are stored, workflow steps are explicit, and traces can be correlated with saved reports.
- This makes the repo suitable for compliance-style evaluation where you need to audit retrieval quality, reasoning quality, and final output quality separately.
11. Outputs and artifacts
- The workflow writes report artifacts to data/reports/ as Markdown and JSON.
- The final output is not just a generated answer; it is an auditable package containing the analysis, draft amendments, and auditor verdict.
- That output boundary is important for diagramming: the pipeline ends in persisted reports, not in a live service response.
12. Tests and validation scope
- tests/test_chunking.py and tests/test_events.py indicate lightweight validation around document segmentation and workflow event construction.
- The test suite supports sanity checks, but the architecture relies more heavily on runtime traces and generated artifacts for meaningful evaluation.
13. Key architectural flow to diagram
- Sources in data/sources/targets.json feed the Scrapy crawler.
- Crawled content is extracted, converted, chunked, and stored in Qdrant.
- The workflow pulls unprocessed regulatory documents, retrieves internal policy evidence, generates analysis and redlines, audits the result, and saves the report.
- The main boundary crossings are: filesystem/config → crawler → Qdrant → workflow/LLM services → report artifacts.
Main entry points: workflow.py Compliance workflow · cli.py CLI · client.py Qdrant client · regulatory_spider.py Regulatory spider
Components 18
Control & config
CLI entrypoint and runtime settings.
Data layer
Persistent collections and schema definitions.
- Qdrant client · Connection and collection access layer.
src/legal_agent/db/client.py - Schemas · Collection and indexing definitions.
src/legal_agent/db/schemas.py - Regulatory store · Scraped regulation chunks and processing state.
- Policy store · Chunked internal policy evidence for retrieval.
Ingestion
Regulatory crawling and policy import paths.
- Targets · Crawl targets for regulatory harvesting.
data/sources/targets.json - Regulatory spider · Harvests HTML and PDF regulatory sources.
src/legal_agent/scraping/spiders/regulatory_spider.py - Scrape pipeline · Normalizes, chunks, enriches, and upserts crawl output.
src/legal_agent/scraping/pipelines.py - Scrape items · Structured crawl item payloads.
src/legal_agent/scraping/items.py - Document loader · Helpers for parsing and extraction, including PDF handling.
src/legal_agent/utils/loader.py - Doc models · Structured document representations.
src/legal_agent/utils/models.py
Compliance workflow
Event-driven analysis and drafting pipeline.
- Compliance workflow · Main event-driven analysis pipeline.
src/legal_agent/workflow/workflow.py - Workflow events · Step-to-step event definitions.
src/legal_agent/workflow/events.py - Model router · Selects OpenAI, Gemini, and reranking services.
src/legal_agent/workflow/llm_provider.py - Prompts · Step-specific prompts for scanning, analysis, and auditing.
src/legal_agent/workflow/prompts.py
Connections 22
- CLI → Config: loads · CLI uses environment-backed settings.
- CLI → Qdrant client: initializes · CLI manages database and persistence operations.
- CLI → Regulatory spider: runs · CLI triggers regulatory scraping commands.
- CLI → Compliance workflow: executes · CLI launches the compliance analysis workflow.
- CLI → Tracing: enables · CLI/runtime wiring includes tracing setup.
- Qdrant client → Schemas: uses · Client relies on collection and schema definitions.
- Qdrant client → Regulatory store: stores · Regulatory updates are written to Qdrant.
- Qdrant client → Policy store: stores · Internal policy chunks are written to Qdrant.
- Targets → Regulatory spider: feeds · Spider reads crawl targets from the source list.
- Regulatory spider → Scrape items: emits · Crawler emits structured extraction items.
- Regulatory spider → Document loader: extracts · Loader supports HTML/PDF document extraction.
- Document loader → Doc models: normalizes · Parsed content is shaped into document models.
- Scrape items → Scrape pipeline: processed by · Pipeline consumes crawl items for downstream handling.
- Document loader → Scrape pipeline: chunks · Pipeline uses parsing helpers for chunking and conversion.
- Scrape pipeline → Regulatory store: upserts · Chunked regulatory records are stored in the regulatory collection.
- Compliance workflow → Workflow events: drives · Workflow steps communicate through typed events.
- Compliance workflow → Model router: routes models · Workflow chooses models per step and cost tier.
- Compliance workflow → Prompts: uses · Workflow step behavior is prompt-driven.
- Compliance workflow → Regulatory store: reads staged docs · Workflow pulls unprocessed regulatory documents for analysis.
- Compliance workflow → Policy store: retrieves evidence · Workflow performs hybrid retrieval over policy chunks.
- Compliance workflow → Reports: writes · Final analysis, redlines, and verdict are persisted as reports.
- Compliance workflow → Tracing: traces · Workflow execution is instrumented for inspection.