How jrats/karbon-ai-agent-challenge works
Project Type & Purpose
• This is an automation/orchestration tool (an “agent”) built around a LangGraph pipeline.
• Its goal: generate, execute, test, and iteratively fix custom Python parsers for bank statement PDFs by leveraging an LLM.
Pinpoint the Main Components
a. Input Data Layer
– “data/<bank>/” directory containing PDF (e.g., icici sample.pdf) and ground-truth CSV (result.csv).
b. Orchestrator (agent.py + LangGraph)
– Core Python script that defines and wires together four nodes.
c. Nodes in the Workflow
1. CodeGen
• Calls OpenAI API with prompts (prompts.py) to generate parser code.
2. Executor
• Writes the code to a .py file in working directory.
3. Tester
• Runs pytest (tests/test_icici_parser.py) against the newly generated parser, comparing output to CSV.
4. Fixer
• On test failures, sends logs back to the LLM for repair suggestions.
d. Cache & Storage
– “cache/<bank>/success.py” for successful parser code.
– “custom_parsers/<bank>_parser.py” as final output.
– Local filesystem for intermediate logs.
e. External Services & Libraries
– OpenAI API (LLM)
– pdfplumber (PDF → text extraction)
– pandas (table processing)
– pytest (automated testing)
Relationships & Data Flows
• Data flow is cyclical:
1. Input PDF & CSV → CodeGen → generated parser code.
2. Parser code → Executor → file on disk.
3. Parser file + PDF → Tester → test result (pass/fail + diagnostics).
4. On fail → Fixer → updated code suggestions → back to Executor.
• On success, branch out: cache code and dump final parser.
• All calls to the LLM go through an API boundary.
Architectural Patterns & Design Principles
• Pipeline/orchestration pattern (LangGraph nodes wired in sequence with a loop back on failures).
• Separation of concerns: code generation, execution, testing, fixing.
• Caching layer to avoid redundant LLM calls.
• Plugin-style extensibility: each bank’s parser is isolated under its own folder.