# pratikpandey994-dotcom/fx-terminal architecture

> Architecture diagram of the GitHub repository pratikpandey994-dotcom/fx-terminal, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pratikpandey994-dotcom/fx-terminal
- Repository: https://github.com/pratikpandey994-dotcom/fx-terminal

## Overview

1. **Overall shape**
   This repo is a multi-runtime intelligence platform, not a single web app. The architecture is split across a **Next.js 14 frontend/API layer**, a **Node.js worker** for ingestion and computation, and a **Python FastAPI pattern service** for technical analog matching. Persistent state lives in **PostgreSQL with pgvector** and **Redis**.

2. **User-facing application**
   The `app/` and `components/` trees implement the terminal-style UI and dashboard surfaces. The main experience is the **research terminal** under `app/(dashboard)/terminal`, with supporting dashboard panels and reusable terminal widgets in `components/terminal` and `components/dashboard`. This layer is primarily presentation, orchestration, and live data fetching.

3. **API boundary**
   The `app/api/` tree is the primary HTTP contract. It groups endpoints by domain: **market**, **intelligence**, **bias**, **pattern**, **feed**, **knowledge**, **public**, **v1**, plus **health/ready** and **terminal bootstrap**. These routes act as the boundary between the UI and the backend engines, normalizing responses and shielding the frontend from direct provider complexity.

4. **Fundamental intelligence engine**
   The Node worker is the core computation plane for the “Engine 1” side of the product. Its jobs ingest and synthesize news, macro releases, central-bank communications, COT, positioning, sentiment, event risk, and narrative structure. The worker’s engine modules cover things like **market structure**, **regime detection**, **funding stress**, **intermarket chains**, **positioning**, **risk sentiment**, **daily brief generation**, and **forecast/recommendation combining**.

5. **Technical pattern engine**
   “Engine 2” is isolated in `services/pattern/`, a separate Python service built around candle encoding, motif discovery, similarity search, and probability estimation. It stores and queries historical analogs using **pgvector-backed similarity**. The worker also contains pattern-related jobs for encoding, resolving, and backfilling candle episodes, which suggests a pipeline from raw price history to analog-based predictions.

6. **Data ingestion layer**
   The worker is responsible for pulling from many external sources via provider adapters and scrapers: **Alpha Vantage, BLS, CFTC, EIA, FRED, Finnhub, OANDA, Twelve Data, World Bank, ForexFactory, GDELT, SEC, RSS, Google News proxy**, and others. There are dedicated jobs for macro, news, earnings, politician trades, source discovery, staleness checks, and feed promotion. This is the main ingress path into the platform.

7. **Persistence and indexing**
   `prisma/schema.prisma` and the large migration history show that PostgreSQL is the system of record for users, layouts, market snapshots, intelligence artifacts, knowledge documents, alerts, and pattern data. `pgvector` is used for semantic or analog retrieval. Redis is used as a shared cache/tiered-speed layer for worker and API paths, with additional edge/cache helpers in the app code.

8. **Knowledge and retrieval subsystem**
   The platform has a separate knowledge-corpus pipeline: embeddings, chunking, hybrid retrieval, taxonomy classification, entity linking, and search. The `services/search/` microservice handles discovery and retrieval across sources using DuckDuckGo/SearxNG-style engines plus LLM-assisted ranking. This subsystem feeds the intelligence layer rather than serving as a user-facing search product.

9. **Alerts and external delivery**
   The system is not a trading bot, but it does produce outbound intelligence via **Telegram** and automation hooks. `lib/telegram` and `worker/src/telegram-notify.ts` indicate alert generation, while `services/telegram/` and `app/api/n8n/ingest` connect to external ingestion/automation flows. A Google Drive cold-storage path is also part of the operational data lifecycle.

10. **AI and model integration**
    Both app and worker have AI client wrappers for **Gemini** and **Groq**. These are used for enrichment, summarization, narrative synthesis, and likely retrieval-augmented reasoning. The AI layer sits above raw data ingestion and below the UI, and is used to turn market facts into explainable intelligence outputs.

11. **Operational architecture**
    Deployment is centered on **Railway** for the app and worker, with a separate **Oracle Cloud n8n** automation environment. Dockerfiles, GitHub Actions, Sentry configs, health/ready routes, and `docker-compose.yml` show that the repo is designed for containerized deployment with explicit observability and environment separation. There is also a strong runbook/docs culture around operations.

12. **Primary data flow**
    The typical flow is: **external providers and scrapers → worker jobs → Postgres/Redis → API routes → terminal/dashboard UI**. For pattern analysis, the flow is: **historical candles → Python pattern service and/or worker encoding jobs → vector similarity/probability outputs → API/UI panels**. The platform’s architecture is therefore a two-engine intelligence pipeline with shared storage and a thin presentation layer on top.

Main entry points: [`worker/src/jobs/index.ts`](https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/worker/src/jobs/index.ts) (Job Suite), [`prisma/schema.prisma`](https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/prisma/schema.prisma) (PostgreSQL), [`app/(dashboard)/terminal/page.tsx`](https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/app/(dashboard)/terminal/page.tsx) (Research Terminal), [`app/api/intelligence/daily-brief/route.ts`](https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/app/api/intelligence/daily-brief/route.ts) (Intel API)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_frontend["Frontend + API"]
  node_ui_terminal["Research Terminal<br/>Next.js UI<br/>[page.tsx]"]
  node_ui_panels["Dashboard Panels<br/>React panels"]
  node_api_market["Market API<br/>route boundary<br/>[route.ts]"]
  node_api_intel["Intel API<br/>route boundary<br/>[route.ts]"]
  node_api_pattern["Pattern API<br/>route boundary<br/>[route.ts]"]
  node_api_knowledge["Knowledge API<br/>route boundary<br/>[route.ts]"]
  node_api_bootstrap["Bootstrap API<br/>startup route<br/>[route.ts]"]
end

subgraph group_worker["Node Worker"]
  node_worker_entry["Worker Core<br/>job runtime<br/>[index.ts]"]
  node_worker_jobs["Job Suite<br/>job orchestrator<br/>[index.ts]"]
  node_worker_providers["Providers<br/>data adapters<br/>[alphaVantage.ts]"]
  node_worker_ingest["Ingestion Flow<br/>data pipeline<br/>[newsIngestion.ts]"]
  node_worker_market["Market Engines<br/>signal engines"]
  node_worker_narrative["Narrative Synthesis<br/>synthesis engine"]
  node_worker_knowledge["Knowledge Pipeline<br/>corpus pipeline"]
  node_worker_pattern["Pattern Jobs<br/>analog pipeline"]
end

subgraph group_pattern["Python Services"]
  node_pattern_service["Pattern Service<br/>Python FastAPI<br/>[main.py]"]
  node_search_service["Search Service<br/>Python retrieval<br/>[main.py]"]
end

subgraph group_data["Storage"]
  node_postgres[("PostgreSQL<br/>system of record<br/>[schema.prisma]")]
  node_redis[("Redis<br/>cache layer<br/>[redis.ts]")]
end

subgraph group_ops["Delivery + Ops"]
  node_telegram["Telegram Alerts<br/>delivery channel<br/>[alerts.ts]"]
  node_n8n["Automation Ingest<br/>external automation<br/>[route.ts]"]
end

node_ui_terminal -->|"bootstraps"| node_api_bootstrap
node_ui_terminal -->|"fetches market"| node_api_market
node_ui_terminal -->|"renders intel"| node_api_intel
node_ui_terminal -->|"shows analogs"| node_api_pattern
node_ui_terminal -->|"searches corpus"| node_api_knowledge
node_ui_panels -->|"drives views"| node_api_market
node_ui_panels -->|"drives views"| node_api_intel
node_api_market -->|"caches"| node_redis
node_api_market -->|"reads state"| node_postgres
node_api_intel -->|"serves signals"| node_worker_market
node_api_intel -->|"serves briefs"| node_worker_narrative
node_api_intel -->|"uses corpus"| node_worker_knowledge
node_api_pattern -->|"analogs"| node_pattern_service
node_api_knowledge -->|"retrieves"| node_search_service
node_api_bootstrap -->|"hydrates"| node_postgres
node_worker_entry -->|"starts"| node_worker_jobs
node_worker_jobs -->|"orchestrates"| node_worker_providers
node_worker_providers -->|"feeds"| node_worker_ingest
node_worker_ingest -->|"writes"| node_postgres
node_worker_ingest -->|"hot cache"| node_redis
node_worker_jobs -->|"computes"| node_worker_market
node_worker_jobs -->|"synthesizes"| node_worker_narrative
node_worker_jobs -->|"builds corpus"| node_worker_knowledge
node_worker_jobs -->|"encodes patterns"| node_worker_pattern
node_worker_pattern -->|"syncs analogs"| node_pattern_service
node_worker_knowledge -->|"indexes"| node_search_service
node_pattern_service -->|"vector store"| node_postgres
node_search_service -->|"reads corpus"| node_postgres
node_worker_narrative -->|"notifies"| node_telegram
node_telegram -->|"records alerts"| node_postgres
node_n8n -->|"ingests"| node_worker_ingest
node_n8n -->|"writes external"| node_postgres

click node_ui_terminal "https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/app/(dashboard)/terminal/page.tsx"
click node_ui_panels "https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/components/terminal/TerminalMainView.tsx"
click node_api_market "https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/app/api/market/candles/route.ts"
click node_api_intel "https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/app/api/intelligence/daily-brief/route.ts"
click node_api_pattern "https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/app/api/pattern/analyze/route.ts"
click node_api_knowledge "https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/app/api/knowledge/search/route.ts"
click node_api_bootstrap "https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/app/api/terminal/bootstrap/route.ts"
click node_worker_entry "https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/worker/src/index.ts"
click node_worker_jobs "https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/worker/src/jobs/index.ts"
click node_worker_providers "https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/worker/src/providers/alphaVantage.ts"
click node_worker_ingest "https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/worker/src/jobs/newsIngestion.ts"
click node_worker_market "https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/worker/src/engines/market-structure.ts"
click node_worker_narrative "https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/worker/src/engines/narrative-engine.ts"
click node_worker_knowledge "https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/worker/src/engines/knowledge-ingest.ts"
click node_worker_pattern "https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/worker/src/jobs/pattern-encode-worker.ts"
click node_pattern_service "https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/services/pattern/main.py"
click node_search_service "https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/services/search/main.py"
click node_postgres "https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/prisma/schema.prisma"
click node_redis "https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/lib/redis.ts"
click node_telegram "https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/lib/telegram/alerts.ts"
click node_n8n "https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/app/api/n8n/ingest/route.ts"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ui_terminal,node_ui_panels,node_api_market,node_api_intel,node_api_pattern,node_api_knowledge,node_api_bootstrap toneBlue
class node_worker_entry,node_worker_jobs,node_worker_providers,node_worker_ingest,node_worker_market,node_worker_narrative,node_worker_knowledge,node_worker_pattern toneAmber
class node_pattern_service,node_search_service toneMint
class node_postgres,node_redis toneRose
class node_telegram,node_n8n toneIndigo
```

## Components

### Frontend + API

Next.js terminal UI and HTTP boundary

- [Research Terminal](https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/app/(dashboard)/terminal/page.tsx): Next.js UI · Main terminal workspace and dashboard experience · `app/(dashboard)/terminal/page.tsx`
- [Dashboard Panels](https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/components/terminal/TerminalMainView.tsx): React panels · Reusable terminal and dashboard widgets · `components/terminal/TerminalMainView.tsx`
- [Market API](https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/app/api/market/candles/route.ts): route boundary · Market data, quotes, candles, and session endpoints · `app/api/market/candles/route.ts`
- [Intel API](https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/app/api/intelligence/daily-brief/route.ts): route boundary · Core intelligence endpoints for briefs, regimes, narratives, and signals · `app/api/intelligence/daily-brief/route.ts`
- [Pattern API](https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/app/api/pattern/analyze/route.ts): route boundary · Analog analysis and event-study endpoints · `app/api/pattern/analyze/route.ts`
- [Knowledge API](https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/app/api/knowledge/search/route.ts): route boundary · Corpus search and retrieval boundary · `app/api/knowledge/search/route.ts`
- [Bootstrap API](https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/app/api/terminal/bootstrap/route.ts): startup route · Terminal bootstrap and readiness checks · `app/api/terminal/bootstrap/route.ts`

### Node Worker

Primary ingestion and intelligence compute plane

- [Worker Core](https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/worker/src/index.ts): job runtime · Main Node worker entrypoint and job scheduler · `worker/src/index.ts`
- [Job Suite](https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/worker/src/jobs/index.ts): job orchestrator · Coordinates ingestion, analysis, and backfill jobs · `worker/src/jobs/index.ts`
- [Providers](https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/worker/src/providers/alphaVantage.ts): data adapters · External market, macro, and news connectors · `worker/src/providers/alphaVantage.ts`
- [Ingestion Flow](https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/worker/src/jobs/newsIngestion.ts): data pipeline · Pulls and normalizes external feeds into the platform · `worker/src/jobs/newsIngestion.ts`
- [Market Engines](https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/worker/src/engines/market-structure.ts): signal engines · Market structure, regime, positioning, funding stress, and risk sentiment · `worker/src/engines/market-structure.ts`
- [Narrative Synthesis](https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/worker/src/engines/narrative-engine.ts): synthesis engine · Daily briefs, narratives, convergence, and recommendations · `worker/src/engines/narrative-engine.ts`
- [Knowledge Pipeline](https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/worker/src/engines/knowledge-ingest.ts): corpus pipeline · Embedding, ingest, taxonomy, and retrieval preparation · `worker/src/engines/knowledge-ingest.ts`
- [Pattern Jobs](https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/worker/src/jobs/pattern-encode-worker.ts): analog pipeline · Candle encoding, pattern resolution, and backfill jobs · `worker/src/jobs/pattern-encode-worker.ts`

### Python Services

Standalone pattern and search runtimes

- [Pattern Service](https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/services/pattern/main.py): Python FastAPI · Standalone technical analog matching and probability service · `services/pattern/main.py`
- [Search Service](https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/services/search/main.py): Python retrieval · Multi-engine discovery and ranked retrieval service · `services/search/main.py`

### Storage

Persistent state and shared caches

- [PostgreSQL](https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/prisma/schema.prisma): system of record · Primary database for users, layouts, intelligence, knowledge, and patterns · `prisma/schema.prisma`
- [Redis](https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/lib/redis.ts): cache layer · Shared cache and speed tier for API and worker flows · `lib/redis.ts`

### Delivery + Ops

Outbound channels and automation

- [Telegram Alerts](https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/lib/telegram/alerts.ts): delivery channel · Outbound alerting and intelligence delivery · `lib/telegram/alerts.ts`
- [Automation Ingest](https://github.com/pratikpandey994-dotcom/fx-terminal/blob/main/app/api/n8n/ingest/route.ts): external automation · Automation and external workflow ingestion entrypoint · `app/api/n8n/ingest/route.ts`

## Connections

- Research Terminal → Bootstrap API: bootstraps · Terminal loads initial state and health before rendering
- Research Terminal → Market API: fetches market · UI requests quotes, candles, sessions, and market summaries
- Research Terminal → Intel API: renders intel · UI consumes intelligence panels, briefs, and regime views
- Research Terminal → Pattern API: shows analogs · UI requests pattern analysis and analog outputs
- Research Terminal → Knowledge API: searches corpus · UI queries knowledge retrieval
- Dashboard Panels → Market API: drives views · Dashboard widgets consume market endpoints
- Dashboard Panels → Intel API: drives views · Dashboard widgets consume intelligence endpoints
- Market API → Redis: caches · Market responses are accelerated through shared cache
- Market API → PostgreSQL: reads state · Market views read persisted snapshots and metadata
- Intel API → Market Engines: serves signals · Intel routes surface engine outputs from the worker plane
- Intel API → Narrative Synthesis: serves briefs · Daily brief, narratives, and synthesis are exposed via API
- Intel API → Knowledge Pipeline: uses corpus · Intel routes can incorporate knowledge and embeddings
- Pattern API → Pattern Service: analogs · Pattern endpoints call the Python analog matching service
- Knowledge API → Search Service: retrieves · Knowledge search delegates to the retrieval service
- Bootstrap API → PostgreSQL: hydrates · Bootstrap pulls persisted workspace and user state
- Worker Core → Job Suite: starts · Worker runtime launches the job suite
- Job Suite → Providers: orchestrates · Job runner coordinates provider-backed ingestion tasks
- Providers → Ingestion Flow: feeds · Adapters supply raw market and news data to ingestion jobs
- Ingestion Flow → PostgreSQL: writes · Ingested artifacts and snapshots are persisted
- Ingestion Flow → Redis: hot cache · Recent data is staged in cache for fast reads
- Job Suite → Market Engines: computes · Worker executes core market intelligence engines
- Job Suite → Narrative Synthesis: synthesizes · Worker generates briefs, narratives, and combined recommendations
- Job Suite → Knowledge Pipeline: builds corpus · Worker maintains knowledge embeddings and retrieval data
- Job Suite → Pattern Jobs: encodes patterns · Worker runs candle and episode processing for analog workflows
- Pattern Jobs → Pattern Service: syncs analogs · Worker pattern jobs complement the standalone pattern runtime
- Knowledge Pipeline → Search Service: indexes · Knowledge assets are prepared for cross-source search
- Pattern Service → PostgreSQL: vector store · Pattern service stores and queries analog history in Postgres pgvector
- Search Service → PostgreSQL: reads corpus · Search runtime queries stored documents and metadata
- Narrative Synthesis → Telegram Alerts: notifies · Narrative and risk outputs can be pushed as alerts
- Telegram Alerts → PostgreSQL: records alerts · Alert lifecycle and delivery state are persisted
- Automation Ingest → Ingestion Flow: ingests · Automation flows can inject external data into ingestion paths
- Automation Ingest → PostgreSQL: writes external · Automation can persist imported records and artifacts