How eprbell/dali-rp2 works
Main Components
• Configuration Manager – This component (see files like configuration.py, configuration_generator.py, and test_config.ini in the config directory) is responsible for reading and parsing the INI configuration file. It initializes all subsequent behaviors.
• Core Data Loader – Represented by the main executable (dali_main.py) in the src/dali directory, this module orchestrates the entire process by calling the relevant subcomponents.
• Plugin System – A key architectural element, the plugin system is divided into:
° Input Plugins – Under src/dali/plugin/input; these handle various native data sources in CSV, ODS, and REST formats.
° Pair Converter Plugins – Under src/dali/plugin/pair_converter; these manage the conversion of exchange symbols or currency pairs.
° Country Plugins – Under src/dali/plugin/country; these adapt behavior and formatting based on ISO country codes.
• Data Processing & Transaction Management – Modules such as abstract_transaction.py, in_transaction.py, intra_transaction.py, and out_transaction.py handle the modeling and processing of transactions.
• Auxiliary Utilities – Additional supporting components include caching (cache.py), pagination (ccxt_pagination.py), and other modules like historical_bar.py, mapped_graph.py, and ods_generator.py for generating the final ODS output file.
Determining Relationships and Data Flow
• The Core Data Loader (dali_main.py) then uses this configuration to set up the processing pipeline.
• The system delegates data ingestion to the proper Input plugins (CSV, ODS, REST) to read files from the input directory.
• Once raw data has been loaded, it is processed (merged and validated) and routed through Pair Converter plugins that normalize trading pairs.
• The Country plugins then format or adjust transaction details according to regional tax rules.
• Finally, the Output Generators (such as ods_generator.py and configuration_generator.py) compile the processed data into final RP2 input files.
• Optionally, caching and graph mapping modules support data flow control and performance optimization between these stages.
• The final outputs are written to the output directory, ready for RP2 to process.
Architectural Patterns and Design Principles
• Plugin Architecture – The system is loosely coupled with clearly defined abstract classes (see abstract_ccxt_input_plugin.py and abstract_pair_converter_plugin.py) ensuring extensibility.
• Separation of Concerns – Different folders handle configuration, data processing, plugin logic, and testing, which supports maintainability.
• Test Coverage – Extensive unit tests in the tests directory indicate a focus on validating core interactions and plugin behaviors.
Final Touches
– Show test modules separately (indicating that these exist to ensure correct behavior).
– Optionally include external dependencies such as RP2 or the Internet (when fetching spot prices).
– Highlight the dependency that installing DaLI pulls in RP2, signaling how the output is used by another system.