How codereverser/casparser works
1. What this repository is
- casparser is a Python package for parsing *issuer-generated* Consolidated Account Statement PDFs.
- It has two main product paths:
- Mutual fund CAS for CAMS and KFintech, which produces transaction-level portfolio data.
- Demat CAS for NSDL and CDSL, which produces holdings-level portfolio data.
2. Top-level runtime boundary
- The package is a local PDF-processing library, not a service.
- Its core runtime dependency is pypdfium2 for PDF text/page extraction.
- Parsed results are modeled with pydantic types and can be returned as Python objects, JSON, or CSV.
3. Primary entry points
- casparser.read_cas_pdf(...) is the main API used by applications and the CLI.
- casparser.cli wraps that API for command-line usage and adds summary/export/gains behaviors.
- The CLI is the user-facing orchestration layer; the parsing logic stays inside casparser.parsers.
4. Parser subsystem
- casparser/parsers/ contains the document recognition and issuer-specific parsing pipeline.
- detect.py identifies the statement type and issuer.
- extract.py, pageobj.py, and utils.py provide lower-level PDF extraction and page/text handling.
- Issuer-specific modules handle template differences:
- cams_detailed.py
- cams_summary.py
- kfintech support appears to be handled alongside CAMS-related parsing paths and tests
- nsdl.py
- cdsl.py
- Shared helpers such as _classify.py, _investor.py, and _isin.py normalize entities, classify transactions, extract investor details, and map schemes to identifiers.
5. Parsing boundaries and failure modes
- The parser is intentionally strict about input provenance: original issuer PDFs are accepted, while reprinted/reformatted PDFs are rejected.
- exceptions.py captures parse and validation failure conditions.
- The design preserves provenance through file_type, cas_type, and parse warnings so downstream consumers can trust or reject partial parses.
6. Data model boundary
- casparser/types.py is the central contract for structured output.
- The repository exposes two main data shapes:
- CASData for CAMS/KFintech mutual fund statements
- NSDLCASData for NSDL/CDSL demat statements
- enums.py defines normalized categorical values used throughout parsing and output.
- This model layer is the bridge between raw PDF extraction and downstream analysis/export.
7. Mutual-fund analysis subsystem
- casparser/analysis/ is a separate post-processing layer for mutual fund statements.
- gains.py computes capital gains reports, including the beta 112A export path mentioned in the README.
- utils.py likely contains calculation helpers and reconciliation logic.
- This subsystem consumes parsed CAS transaction data and produces derived reporting outputs; it is not used for NSDL/CDSL holdings-only statements.
8. Output and serialization
- The library supports multiple output forms from the same parsed core:
- native pydantic models
- JSON serialization
- CSV exports
- summary tables
- The CLI chooses the output path based on flags and file extension.
- Gains output can generate additional companion CSV files, and gift-transfer rows are treated as separate informational output rather than part of capital-gains calculations.
9. ISIN/AMFI enrichment
- The parser enriches scheme and holding records with ISIN and AMFI metadata via logic in _isin.py.
- The README notes this depends on the external casparser-isin database/tooling.
- Architecturally, this is a lookup/enrichment boundary: parsing can succeed without full identifiers, but enrichment improves downstream classification and reporting.
10. Schema and contract generation
- schema/ contains checked-in JSON Schema files for the main output models.
- scripts/generate_schema.py suggests the schemas are generated from the pydantic types and committed for consumers.
- This makes the data contract explicit for validation, client generation, and integration with other tools.
11. Testing strategy
- tests/ is broad and file-based, covering CAMS, KFintech, NSDL, CDSL, CLI behavior, gains, schema docs, helpers, and error cases.
- The presence of encrypted test fixtures (tests/files.enc) indicates the parser is validated against realistic protected PDFs.
- End-to-end gains tests show that report generation is considered part of the supported architecture, not just unit-level math.
12. Repository-level tooling
- The project uses standard Python packaging via pyproject.toml and a locked environment via uv.lock.
- CI workflows cover pytest, code scanning, and PyPI publishing.
- Pre-commit and Codecov configuration indicate the repo is maintained as a production library with release and quality gates.
13. How the data flows end to end
- Input PDF + password
- PDF extraction and issuer detection
- Issuer-specific parsing into normalized records
- Optional enrichment and classification
- Construction of pydantic output models
- Optional post-processing for summary/gains/CSV/JSON exports
14. Diagramming hint
- The cleanest architecture diagram would show three main layers:
- PDF ingestion/parsing
- Normalized statement models
- Analysis/export
- Then branch the parser layer into the four issuer families, and branch the output layer into mutual-fund CAS versus demat CAS, with gains analysis attached only to the mutual-fund path.
Main entry points: types.py CAS models · tests Tests · detect.py Detect · _classify.py Parse helpers
Components 19
PDF parsing runtime
Input PDF ingestion, issuer detection, and issuer-specific parsing.
- Public API · Top-level package entrypoint exposing read_cas_pdf.
casparser/__init__.py - CLI · User-facing orchestration for parse, summary, export, and gains flows.
casparser/cli.py - Detect · Identifies statement type, issuer, and parse path.
casparser/parsers/detect.py - PDF extract · Low-level PDF/page text extraction and page object handling.
casparser/parsers/extract.py - CAMS detailed · Mutual-fund transaction parsing for detailed CAMS statements.
casparser/parsers/cams_detailed.py - CAMS summary · Mutual-fund summary parsing path for CAMS statements.
casparser/parsers/cams_summary.py - NSDL parser · Demat holdings parsing for NSDL statements.
casparser/parsers/nsdl.py - CDSL parser · Demat holdings parsing for CDSL statements.
casparser/parsers/cdsl.py - Parse helpers · Shared classification, investor extraction, and ISIN/AMFI enrichment logic.
casparser/parsers/_classify.py - pypdfium2 · PDF text/page extraction runtime used by the parsing layer.
- casparser-isin · External enrichment source for ISIN and AMFI metadata.
Statement model
Normalized output types and shared contracts.
- CAS models · Structured output contracts for mutual-fund and demat statements.
casparser/types.py - Enums · Shared categorical values used across parsing and output.
casparser/enums.py - Parse errors · Failure modes and validation boundaries for strict issuer-PDF parsing.
casparser/exceptions.py
Analysis and export
Mutual-fund post-processing, serialization, and schema generation.
- Gains calc · Capital gains and 112A reporting for parsed mutual-fund CAS data.
casparser/analysis/gains.py - Analysis utils · Computation and reconciliation helpers for gains processing.
casparser/analysis/utils.py - JSON Schema · Checked-in schemas for main output models.
schema/CASData.schema.json - Schema gen · Generates schema artifacts from the pydantic models.
scripts/generate_schema.py
Tooling and tests
Repository maintenance and validation.
- Tests · File-based regression and end-to-end coverage for parsers, CLI, gains, and schema docs.
tests
Connections 31
- Public API → Detect: reads CAS PDF · Main API dispatches into parser detection and extraction.
- CLI → Public API: invokes · CLI wraps the public parsing API and post-processing options.
- Detect → PDF extract: uses · Detection relies on extracted page text and layout clues.
- PDF extract → pypdfium2: backend · PDF extraction is powered by pypdfium2.
- Detect → CAMS detailed: routes · Recognizes detailed CAMS mutual-fund statements.
- Detect → CAMS summary: routes · Recognizes summary CAMS mutual-fund statements.
- Detect → NSDL parser: routes · Recognizes NSDL demat statements.
- Detect → CDSL parser: routes · Recognizes CDSL demat statements.
- CAMS detailed → Parse helpers: normalizes · Issuer parser uses shared classification and enrichment helpers.
- CAMS summary → Parse helpers: normalizes · Issuer parser uses shared classification and enrichment helpers.
- NSDL parser → Parse helpers: normalizes · Demat parser uses shared investor and identifier helpers.
- CDSL parser → Parse helpers: normalizes · Demat parser uses shared investor and identifier helpers.
- Parse helpers → casparser-isin: enriches · ISIN/AMFI metadata lookup is an external enrichment boundary.
- CAMS detailed → CAS models: builds · Produces mutual-fund CAS data objects.
- CAMS summary → CAS models: builds · Produces mutual-fund CAS data objects.
- NSDL parser → CAS models: builds · Produces NSDL demat CAS data objects.
- CDSL parser → CAS models: builds · Produces CDSL demat CAS data objects.
- CAS models → Enums: uses · Structured models rely on normalized categorical values.
- CAS models → Parse errors: validates · Strict parsing boundaries surface through parse and validation exceptions.
- Gains calc → CAS models: consumes · Capital gains analysis consumes parsed mutual-fund transaction data.
- Gains calc → Analysis utils: uses · Gains logic delegates calculations and reconciliation helpers.
- CLI → Gains calc: optional path · CLI can generate capital-gains outputs for mutual-fund statements.
- CAS models → JSON Schema: publishes · Checked-in schemas document the output contract.
- Schema gen → CAS models: generates from · Schema generation is driven by the pydantic model layer.
- Tests → Public API: covers · Regression tests exercise the public API.
- Tests → CLI: covers · CLI behavior is covered end to end.
- Tests → CAMS detailed: covers · Parser fixtures validate CAMS behavior.
- Tests → NSDL parser: covers · Parser fixtures validate NSDL behavior.
- Tests → CDSL parser: covers · Parser fixtures validate CDSL behavior.
- Tests → Gains calc: covers · Gains calculation and export behavior are tested.
- Tests → JSON Schema: covers · Schema documentation and contract checks are tested.