How noviscient/novi-tally works
Project type and purpose
• This is a Python-based, open-source library (module) called “novi_tally” whose sole purpose is to reconcile position (and eventually trade) data from various brokers and sources. It provides:
• A user-facing API (the Position class)
• A plugin-style framework of dataloaders and connection wrappers
• Schemas to standardize data into pandas DataFrames
• Reconciliation logic to compare two positions by instrument
File structure
Top-level
• README.md, SETUP.md, config-example.toml, pyproject.toml
• examples/ – sample scripts that show how users invoke the library
Core package: novi_tally/
• api.py – entrypoint functions, e.g. Position.from_config_file
• config.py – TOML parsing and config validation
• connections/ – wrappers for external data sources (S3, SFTP, OpenFIGI, Formidium file)
• dataloaders/ – classes implementing the DataLoader protocol for each broker
• protocols.py – definitions of the DataLoader protocol interfaces
• schemas.py – definitions of standardized DataFrame schemas (positions, trades)
• errors.py – custom exception types
README for architecture clues
• The library uses a layered approach:
1) Configuration →
2) Instantiate DataLoader →
3) DataLoader.extract() → pulls raw data via a Connection →
4) DataLoader.transform() → enforces a Schema →
5) .reconcile_with() → reconciliation logic
Elements
– Blue for core modules (api, config, protocols, schemas)
– Green for extension modules (dataloaders, connections)
– Orange for external systems (S3, APIs, file system)
Final advice
• Keep it readable: use consistent iconography for modules vs. external services, and annotate any complex flow with brief notes.
• The goal is to capture all essential layers—configuration, extraction, transformation, reconciliation—and show exactly where an engineer would add a new plugin.