How nayakwadis/mftool works
1. What this repository is
- mftool is a Python library, not an application server.
- Its main job is to fetch and normalize public mutual fund data in India: live NAV quotes, bulk quotes, historical NAVs, scheme lookup, and validation of scheme codes.
- The architecture is centered on a single importable library API that talks to external public data sources and returns Python-native structures or DataFrames.
2. Primary runtime boundary
- The key boundary is between library consumers and the external AMFI/public mutual-fund data sources.
- Client code calls into mftool.py; that code fetches data over the network, parses it, and returns formatted results.
- There is no separate backend process, database, or message queue in this repo.
3. Core API module
- mftool.py is the main entry point and likely contains the public class/function surface used by callers.
- The README feature list maps directly to this layer: scheme-wise quote lookup, bulk quotes, historical NAVs, scheme search, and scheme-code validation.
- This module is the orchestration layer that ties together fetching, parsing, cache handling, and result shaping.
4. Data-fetching and transformation flow
- The expected flow is: input scheme code or search term → fetch remote AMFI data → parse/clean → return as dict / JSON / DataFrame.
- The repo explicitly supports multiple output shapes, so one architectural concern is presentation/serialization rather than just retrieval.
- utils.py likely hosts shared helpers for parsing, date handling, conversion, and response normalization.
5. Caching subsystem
- cache.py and CACHING_GUIDE.md indicate a dedicated local caching layer.
- This layer exists to reduce repeated network calls and improve performance for repeated quote/history requests.
- Architecturally, cache sits between the public API and the remote data source, with tests validating that behavior.
6. Static reference data
- data/Scheme_codes.txt, scheme_codes.json, and const.json are repository-managed data assets used by the library.
- These files likely support scheme lookup, validation, or mapping between scheme names and codes.
- This creates an important boundary between dynamic market data fetched remotely and static reference metadata shipped with the package.
7. Package and dependency structure
- setup.py, requirements.txt, and __init__.py define the library packaging and import surface.
- The project targets Python 3.10+ per the README badge.
- The presence of DataFrame output in the README suggests a dependency on pandas or a similar data library, though the exact usage is encapsulated inside the package.
8. Testing boundary
- tests/mftool_tests.py, tests/test_bulk_quotes.py, tests/test_caching.py, and tests/test_search.py show that the repo is organized around behavioral tests for API features.
- Tests likely validate response parsing, search correctness, bulk quote handling, and cache semantics.
- This helps confirm the library’s main seams: network integration, cache behavior, and lookup logic.
9. Documentation and developer experience
- docs/ with docs/conf.py, docs/index.rst, and .readthedocs.yaml shows that documentation is a first-class part of the project.
- CACHING_GUIDE.md is an additional architecture-oriented usage guide for a nontrivial subsystem.
- Documentation and examples are important because the library is consumed directly by other Python projects.
10. Automation and quality tooling
- .github/workflows/codeql-analysis.yml adds static security/code scanning.
- This repo also appears ready for public distribution via PyPI and ReadTheDocs.
- These are supporting layers, not runtime subsystems, but they matter for how the library is maintained and released.
11. External integration and usage model
- The library is designed for downstream use in analytics, automation, and data collection workflows.
- The external dependency is the live mutual fund data source; the repo itself provides the client-side adapter and local reference data.
- For an architecture diagram, the key picture is: consumer application → mftool API → cache/static data → external mutual fund data source → normalized output.
Main entry points: mftool.py mftool.py · cache.py cache.py · conf.py docs config · utils.py utils.py
Components 20
Core library
Importable API, caching, and shared utilities that power the package.
- mftool.py · Main library entry point and orchestration surface for callers.
mftool.py - utils.py · Shared parsing, conversion, and normalization helpers.
utils.py - cache.py · Local caching logic to reduce repeated remote fetches.
cache.py - __init__.py · Package import surface.
__init__.py
Reference data
Static files shipped with the package for scheme lookup and validation.
- Scheme codes · Packaged scheme code list used for lookup and validation.
data/Scheme_codes.txt - scheme_codes.json · Structured scheme metadata used by the library.
scheme_codes.json - const.json · Static configuration and mapping values.
const.json
Tests
Behavioral tests around the main library features.
- API tests · Tests for core library behavior and responses.
tests/mftool_tests.py - Bulk quotes · Tests for bulk quote retrieval and formatting.
tests/test_bulk_quotes.py - Cache tests · Tests covering cache behavior and reuse.
tests/test_caching.py - Search tests · Tests for scheme search and lookup behavior.
tests/test_search.py
Docs
User and developer documentation for the package.
- README.md · Primary project overview and API examples.
README.md - Caching guide · Documentation for the caching subsystem.
CACHING_GUIDE.md - docs config · Sphinx documentation configuration.
docs/conf.py - docs index · Documentation landing page.
docs/index.rst
Maintenance
Packaging and automation files for release, docs, and analysis.
- setup.py · Package metadata and install configuration.
setup.py - requirements · Runtime dependency specification.
requirements.txt - docs deps · Documentation build dependencies.
docs/requirements.txt - .readthedocs.yaml · Hosted documentation build configuration.
.readthedocs.yaml - codeql workflow · Static analysis workflow.
.github/workflows/codeql-analysis.yml
Connections 19
- mftool.py → utils.py: uses · Core API relies on shared parsing and normalization helpers.
- mftool.py → cache.py: uses · API consults cache before or after remote fetches.
- mftool.py → Scheme codes: reads · Uses packaged scheme code reference data.
- mftool.py → scheme_codes.json: reads · Uses structured scheme metadata for lookup and validation.
- mftool.py → const.json: reads · Consumes static constants and mappings.
- cache.py → utils.py: uses · Cache logic shares normalization and conversion helpers.
- __init__.py → mftool.py: exports · Package import surface exposes the main library entry point.
- API tests → mftool.py: verifies · Covers primary API behavior.
- Bulk quotes → mftool.py: verifies · Covers bulk quote flow.
- Cache tests → cache.py: verifies · Covers cache semantics.
- Search tests → mftool.py: verifies · Covers search and lookup behavior.
- README.md → mftool.py: documents · README explains the main public API.
- Caching guide → cache.py: documents · Guide explains cache usage and behavior.
- docs config → docs index: builds · Sphinx config drives the docs site.
- docs deps → docs config: supports · Documentation build dependencies.
- .readthedocs.yaml → docs config: builds · Hosted docs pipeline uses Sphinx configuration.
- setup.py → requirements: depends on · Packaging references runtime dependencies.
- codeql workflow → mftool.py: scans · Static analysis targets the library code.
- codeql workflow → utils.py: scans · Static analysis includes shared helper code.