ronnyling/income_research_os

1. What this repo is

- incomos is an investment research pipeline, not an execution/trading system.

- Its main output is a ranked, human-reviewed funnel of income stocks: universe → screen → KIV watchlist → candidates → finalists → decision queue.

2. Primary orchestration path

- The end-to-end run is driven by scripts/run_funnel.py.

- That script pulls together the modules under src/incomos/funnel, screening, scoring, llm, macro, data, and persistence.

- The architecture is stage-based and mostly batch-oriented, with expensive work deferred until later stages.

3. Universe and data ingestion boundary

- src/incomos/data is the external-data layer.

- It contains fetchers for EDGAR/SEC filings, market prices, FRED macro series, Bursa Malaysia data, NOBL holdings, and ticker universe construction.

- This is the main boundary between the system and external services such as EDGAR, yfinance, FRED, and Bursa sources.

4. Stage 0–1: fast deterministic screening

- src/incomos/screening/stage01.py performs the first hard filter over the stock universe.

- It uses structured financial facts to reject names that fail basic quality gates such as dividend history, FCF, payout logic, market cap, and ROIC-style checks.

- This stage is intentionally cheap and rule-based; it reduces the universe before any LLM work.

5. Stage 1–2: entry attractiveness ranking

- src/incomos/funnel/entry_attractiveness.py ranks screened stocks on a 0–100 attractiveness score.

- The score combines drawdown, yield expansion, dividend growth, valuation/FCF quality, and trend.

- This stage decides which stocks enter the KIV basket, so it is the bridge between screening and ongoing monitoring.

6. KIV basket as the active watch layer

- src/incomos/funnel/kiv.py and refresh.py manage the watchlist lifecycle.

- KIV is not passive storage: names can be promoted, demoted to dormant, or rejected based on TTL expiry, new bad filings, dividend risk, or deteriorating fundamentals.

- This layer is where daily/weekly refresh logic concentrates, so the rest of the pipeline does not recompute everything every run.

7. Stage 2–4: LLM-assisted analysis

- src/incomos/llm/mimo.py and schemas.py encapsulate MiMo 2.5 use.

- The code only invokes the model after cheaper rule-based tools have run, and its outputs are schema-validated before they affect scoring.

- In the architecture diagram, this should be shown as an expensive, gated branch that consumes filing text and structured context, then returns classification and narrative reasoning.

8. Scoring engine and decision model

- src/incomos/scoring contains the business, income, dip, oversold, and aggregate scoring logic.

- engine.py combines these sub-scores into the final opportunity score, while dip.py, business.py, income.py, and oversold.py handle component calculations.

- The scoring model is config-driven via src/incomos/core/config.py, so thresholds and weights are not hardcoded.

9. Macro context as a cross-cutting input

- src/incomos/macro/regime.py supplies a regime view used in later-stage interpretation.

- Macro inputs come from FRED-style series and are treated as a decision context layer, not as a primary driver of the whole universe screen.

- In the diagram, this is a side input into candidate/finalist evaluation, not a standalone pipeline branch.

10. Persistence and checkpointing

- src/incomos/persistence holds the PostgreSQL boundary: session management, DB setup, and query/upsert logic.

- The funnel also writes a checkpoint and human-readable MiMo results so later runs can regenerate reports without redoing all network and LLM work.

- This is a useful artifact boundary: raw data and intermediate state are persisted separately from final report output.

11. Backtest and validation layer

- src/incomos/backtest/validation.py and mcp_servers/backtest_mcp.py support forward-return validation and calibration reporting.

- This layer evaluates whether score buckets and classifications have predictive value.

- It sits alongside the main research pipeline rather than inside the live funnel path.

12. MCP server decomposition

- The mcp_servers/ package exposes domain tools as separate servers: market data, SEC filings, macro regime, research store, backtest, and Bursa Malaysia.

- Architecturally, these are tool-facing wrappers around the internal modules, giving an agent or client a narrow domain API instead of direct access to internal code.

- If you are drawing the system, treat them as service façades over the same core domain logic.

13. Execution model and concurrency

- The repo is Python-based and uses ThreadPoolExecutor heavily for parallel fetching and batch MiMo work.

- Parallelism matters architecturally because fetching prices, downloading filings, and classifying batches are all concurrency-aware.

- This reduces wall-clock time, but the pipeline still preserves stage ordering and validation gates.

14. User-facing surfaces

- src/incomos/api/main.py provides a FastAPI surface, while docs/index.html is the generated report output.

- The primary human interaction is still CLI-driven and report-based, not a transactional web workflow.

- So the cleanest diagram is: CLI/API entrypoints → funnel pipeline → optional DB/report artifacts.

15. Key architectural constraints

- src/incomos/core centralizes typed config, exceptions, and shared types, which enforces the repo’s “no silent fallback” and “no hardcoded threshold” design.

- Those constraints are important to show because they define how data failure propagates: missing inputs become typed errors, not default scores.

- That makes the pipeline deterministic, auditable, and suitable for human-in-the-loop research rather than automated trading.