rawatpranjal/prefgraph

1. PrefGraph is a dual-language analytics library for revealed-preference and choice-consistency scoring. The public user-facing API is Python (prefgraph), while the heavy computation lives in a Rust engine (rpt-core) exposed through a Python binding crate (rpt-python).

2. The main runtime boundary is Python orchestration vs. Rust execution. Python accepts DataFrames, arrays, parquet, and domain-specific logs; it normalizes them into internal session/panel objects and dispatches scoring. Rust performs graph algorithms, LP-based checks, cycle detection, and batch evaluation in parallel, with Rayon used for multi-user throughput.

3. The Python package under src/prefgraph is organized around three layers: core data structures (BehaviorLog, Session, Panel, result types), the analysis/engine entry points (engine.py, analyze.py, auditor.py, encoder.py), and algorithm wrappers (algorithms/). This is the primary application boundary that users interact with.

4. src/prefgraph/algorithms contains the per-metric implementations and domain-specific analyzers. It spans budget-choice methods such as GARP, CCEI/AEI, HARP, MPI, Houtman-Maks, VEI, welfare, production, quasilinear, intertemporal, and stochastic choice, plus discrete-choice and menu-related routines such as attention, ranking, risk, spatial, and integrability. contrib/ mirrors many of these as experimental or extended variants.

5. src/prefgraph/core holds the shared result and reporting model. The results/ subpackage defines structured outputs for different analysis families, while display.py, summary.py, and mixins.py support human-readable reporting and consistent result behavior across algorithms.

6. Data ingestion is split by domain. src/prefgraph/datasets provides loaders for demo and real-world datasets; src/prefgraph/io/parquet.py handles columnar data interchange; and src/prefgraph/datasets/_generators.py plus test generators support synthetic scenarios and stress cases. The repository also includes many dataset-specific adapters for e-commerce and recommender-style inputs.

7. The Rust workspace under rust/ is the computational backend. rpt-core contains the algorithms and low-level primitives: graph traversal/SCC/transitive closure, LP utilities, choice axioms, production and welfare routines, stochastic and intertemporal models, and type definitions. rpt-python is the bridge crate that converts Python inputs into Rust-native structures and supports batch/parquet-oriented execution.

8. The data flow for a typical batch analysis is: user data enters Python, gets transformed into a normalized session/panel representation, is partitioned for streaming or per-user scoring, then passed into Rust for metric evaluation, and finally returned as flat result records or rich diagnostic objects. The README’s “Engine” vs “Function API” split reflects this architecture: the engine is optimized for high-volume scoring, while the function API exposes detailed diagnostics, matrices, and graphs for single-case investigation.

9. Visualization and human inspection are first-class but downstream concerns. src/prefgraph/viz generates attention plots, generic plots, and HTML templates, while the docs/ and examples/ trees provide narrative notebooks/scripts for quickstarts, benchmarks, and theory tutorials. These do not sit in the critical path of scoring, but they document and validate expected usage patterns.

10. Benchmarking and validation are extensive and form a separate subsystem. benchmarks/ and case_studies/ contain performance experiments, scaling tests, and domain studies for e-commerce, menu choice, and LLM-style benchmarks. They measure throughput, memory behavior, and cross-language parity, including a dedicated Rust benchmark crate and output artifacts for published results.

11. The test suite is broad and architecture-revealing: unit tests cover algorithm correctness, backend parity, parquet I/O, engine behavior, numerical stability, and result summaries; simulation tests verify theoretical properties across synthetic scenarios; and stress/evals tests probe edge cases such as singular matrices, overflow, LP failures, and tolerance sensitivity. This indicates the project prioritizes reproducibility and backend equivalence across Python and Rust.

12. Tooling and release infrastructure are centered on Python packaging plus CI/CD. pyproject.toml and the GitHub Actions workflows cover testing, installation, auditing, documentation, and release automation; ReadTheDocs configuration supports hosted docs; and the repository includes changelog/security/contribution files, suggesting a mature library distribution model rather than an application server or service.