How ceta-research/backtests works
This repository is a strategy research workspace, not a single application. Its core architecture is a shared backtesting framework at the root, plus many self-contained strategy folders that follow the same execution pattern and write comparable result artifacts.
1. Shared foundation at the repo root
The reusable engine lives in root-level modules: cr_client.py for Ceta Research API access, cloud_runner.py for cloud project execution, data_utils.py for loading and aligning market/fundamental data, metrics.py for portfolio statistics, costs.py for transaction cost models, and cli_utils.py for common CLI parsing and exchange preset logic.
2. Primary execution flow
Most strategy folders implement the same pipeline: fetch data from the Ceta Research SQL API, load/cache it in DuckDB, run screening or backtesting logic, compute performance metrics, and persist outputs to results/. This is the key data flow to show in an architecture diagram.
3. Strategy directories are isolated research units
Each strategy folder is its own boundary with its own README.md, screen.py, and usually backtest.py. Some also include generate_charts.py, charts/, and extra helpers such as generate_share_links.py, build_comparison.py, or merge_results.py. The repo contains many factor families: value, quality, momentum, dividend, cash flow, debt, event-driven, ETF analysis, and pairs trading.
4. Two execution modes: local and cloud
The README describes three modes: local execution, cloud Code Execution, and cloud Projects. Local runs are the default and use DuckDB plus requests. Cloud mode is abstracted through cr_client.py; cloud_runner.py wraps project upload/run behavior. The architectural boundary here is “run strategy code on local machine vs managed cloud compute.”
5. Screening vs backtesting
screen.py typically represents the current-point screen for a strategy, while backtest.py runs the historical simulation. In many folders, the screen is a lighter-weight filter and the backtest is the full time-series simulation. This separation is important because some folders are screening-only or event-analysis-only.
6. Result artifacts are the main output interface
Results are stored as JSON, CSV, and occasionally logs. Common patterns include exchange_comparison.json, returns_<EXCHANGE>.json, event CSVs, and summary metrics files. These outputs are the handoff between compute and reporting and are what downstream chart scripts consume.
7. Charting/report generation is a separate post-processing step
Where present, generate_charts.py reads from results/ and produces plots in charts/. This keeps visualization out of the backtest path and makes chart regeneration independent from recomputation.
8. Exchange coverage is a major dimension of the model
Many strategies run across multiple exchanges and region presets: US major markets, Europe, Asia, and country-specific sets. Some folders include run_all_exchanges.py to batch-compute per-exchange outputs, and the repo-wide CLI preset system standardizes that targeting. A diagram should show exchange selection as a parameter feeding the same strategy logic.
9. Strategy families with distinct output shapes
- Cross-sectional factor screens/backtests: QARP, Piotroski, low P/E, value-momentum, dividend-growth, etc.
- Event-driven strategies: earnings surprise, post/pre-earnings dip, stock split, analyst revision, beat streaks, upgrade cluster; these often emit event CSVs in addition to JSON summaries.
- Pairs/statistical analysis: pairs-*, sector-correlation, etf-overlap, sp500-survivorship; these emphasize relationship analysis rather than single-name ranking.
- NSE intraday system bundle: nse_arena/ is a separate mini-framework with strategy implementations like ORB, VWAP reversion, and gap-up scalp, plus shared charges/framework code.
10. Methodology and governance live alongside code
METHODOLOGY.md, DATA_QUALITY_ISSUES.md, and CLAUDE.md document assumptions, data caveats, and validation approach. These are not runtime components, but they explain why outputs are reproducible and how the strategies should be interpreted.
11. Precomputed results are part of the repository state
The many files under results/ act as cached research outputs and regression references. They let charts, comparison tables, and documentation be generated without re-running every strategy.
12. Overall boundary map
The clean architecture to diagram is:
external data/API + cloud execution services → shared utility layer → strategy-specific screening/backtest modules → persisted results/artifacts → chart/report generation.
The repo is optimized for repeatable research experiments across many strategies, with a common toolchain but isolated per-strategy implementations.
Main entry points: METHODOLOGY.md Method docs · cr_client.py Ceta client · cloud_runner.py Cloud runner · data_utils.py Data utils
Components 17
Shared foundation
Repository-wide utilities, cloud access, and governance at the root.
- Ceta client · Access to Ceta Research services and cloud execution primitives.
cr_client.py - Cloud runner · Wraps upload/run behavior for managed cloud project execution.
cloud_runner.py - Data utils · Loads, aligns, and caches market and fundamental inputs.
data_utils.py - Metrics · Portfolio and return-statistic calculations shared by strategies.
metrics.py - Costs · Transaction-cost and slippage modeling shared by backtests.
costs.py - CLI utils · Common command-line parsing and exchange preset logic.
cli_utils.py - Method docs · Research assumptions, data caveats, and validation guidance.
METHODOLOGY.md
Strategy families
Mostly cross-sectional factor and thematic research folders that follow the shared pattern.
- Screeners · Current-point filters that select candidates for a strategy.
- Backtests · Historical simulations that produce return series and summary statistics.
- Exchange presets · Region and exchange targeting that feeds the same strategy logic.
- Results store · JSON, CSV, and log outputs persisted under each strategy's results folder.
- Charting · Reads results and generates plots independently of recomputation.
- Factor families · Value, quality, momentum, dividend, and balance-sheet style strategies.
- Fundamental growth · DCF, cash flow, asset growth, ROIC, and related fundamentals-driven work.
Event studies
Strategies centered on discrete events and event-window outputs.
- Event studies · Earnings, revisions, splits, spinoffs, and similar event-window strategies.
Relative-value research
Pairs, correlation, ETF structure, and similar relationship-focused analysis.
- Relative value · Pairs, ETF overlap/crowding, correlation, and survivorship-style analysis.
NSE arena
A separate intraday strategy mini-framework.
- NSE arena · Separate intraday framework for ORB, VWAP reversion, and gap-up scalp systems.
nse_arena
Connections 19
- Ceta client → Backtests: cloud data · Cloud API supplies remote data access and managed execution context.
- Cloud runner → Backtests: managed run · Runs strategy code in cloud projects when not executing locally.
- Data utils → Backtests: inputs · Shared loading and alignment utilities feed the strategy pipeline.
- Metrics → Backtests: scores · Backtests compute performance using common metrics.
- Costs → Backtests: frictions · Backtests incorporate transaction costs and slippage.
- CLI utils → Exchange presets: targeting · CLI presets standardize exchange and region selection.
- Exchange presets → Screeners: parameterize · Exchange selection configures the screening entrypoint.
- Exchange presets → Backtests: parameterize · Exchange selection configures the historical simulation entrypoint.
- Screeners → Results store: writes · Screeners persist shortlist and summary artifacts.
- Backtests → Results store: writes · Backtests persist returns, metrics, and exchange comparison outputs.
- Results store → Charting: consumes · Chart generation reads persisted outputs without rerunning strategies.
- Factor families → Screeners: implements · Many factor folders are screening-first or screen-plus-backtest units.
- Fundamental growth → Backtests: implements · Fundamental workflows typically culminate in a backtest or evaluation run.
- Event studies → Results store: event outputs · Event-driven strategies often emit both summary JSON and event CSVs.
- Relative value → Backtests: implements · Relative-value research runs pair or basket simulations and comparisons.
- NSE arena → Results store: outputs · The intraday bundle writes comparison and sweep artifacts under its own results tree.
- Method docs → Backtests: explains · Documentation frames assumptions, validation, and interpretation of outputs.
- Method docs → Results store: governs · Governance docs describe caveats for cached and precomputed artifacts.
- Method docs → Exchange presets: defines · Documentation explains the exchange and region coverage model.