# perennial-asian-boiz/perennialbackend architecture

> Architecture diagram of the GitHub repository perennial-asian-boiz/perennialbackend, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/perennial-asian-boiz/perennialbackend
- Repository: https://github.com/perennial-asian-boiz/perennialbackend

## Overview

PerennialBackend’s landed product workflow is a scheduled consensus-watchlist pipeline: collect congressional trades and ARK holdings, enrich candidate tickers with insider and short-interest signals, then aggregate and rank them. The backend also contains separate news-feed and Bing web-search ingestion scripts. The README marks those ingestion capabilities as not yet landed; their sampled source shows standalone functionality, not integration into the watchlist pipeline. Edges are limited to calls and data relationships evidenced by excerpts or documentation; unsampled wiring is omitted.

Main entry points: [`development/backend/src/services/consensus_watchlist/scheduler/cron.py`](https://github.com/perennial-asian-boiz/perennialbackend/blob/main/development/backend/src/services/consensus_watchlist/scheduler/cron.py) (Pipeline scheduler), [`development/database/local_data`](https://github.com/perennial-asian-boiz/perennialbackend/tree/main/development/database/local_data) (Local JSON data), [`development/backend/src/services/consensus_watchlist/fetchers/insider.py`](https://github.com/perennial-asian-boiz/perennialbackend/blob/main/development/backend/src/services/consensus_watchlist/fetchers/insider.py) (Insider activity), [`development/backend/src/services/consensus_watchlist/fetchers/short_interest.py`](https://github.com/perennial-asian-boiz/perennialbackend/blob/main/development/backend/src/services/consensus_watchlist/fetchers/short_interest.py) (Short interest)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_pipeline["Watchlist pipeline"]
  node_scheduler["Pipeline scheduler<br/>[cron.py]"]
  node_consensus["Signal aggregation<br/>[consensus.py]"]
  node_localdata[("Local JSON data")]
end

subgraph group_signals["Signal collection"]
  node_fmp["Congress trades<br/>[fmp.py]"]
  node_ark["ARK holdings<br/>[ark.py]"]
  node_insider["Insider activity<br/>[insider.py]"]
  node_shortinterest["Short interest<br/>[short_interest.py]"]
end

subgraph group_ingestion["Content ingestion"]
  node_news["News ingestion<br/>[news_fetcher.py]"]
  node_search["Web search ingestion<br/>[search_engine.py]"]
end

subgraph group_shared["Shared runtime"]
  node_paths["Path resolution<br/>[paths.py]"]
  node_logger["Logging utility<br/>[logger.py]"]
end

node_operator(("Operator"))
node_financialmodelingprep["Financial Modeling Prep"]
node_arkservice["ARK holdings service"]
node_securitiesdb["SecuritiesDB"]
node_nasdaq["Nasdaq API"]
node_newsfeeds["News RSS feeds"]
node_bing["Bing search"]
node_websites["Article websites"]

node_operator -->|"starts"| node_scheduler
node_scheduler -->|"runs"| node_fmp
node_scheduler -->|"runs"| node_ark
node_scheduler -->|"runs"| node_insider
node_scheduler -->|"runs"| node_shortinterest
node_scheduler -->|"runs"| node_consensus
node_fmp -->|"fetches trades"| node_financialmodelingprep
node_ark -->|"fetches holdings"| node_arkservice
node_insider -->|"requests activity"| node_securitiesdb
node_shortinterest -->|"fetches data"| node_nasdaq
node_fmp -->|"writes trades"| node_localdata
node_ark -->|"writes holdings"| node_localdata
node_insider -->|"writes results"| node_localdata
node_shortinterest -->|"writes results"| node_localdata
node_consensus -->|"reads signals"| node_localdata
node_insider -->|"reads candidate tickers"| node_localdata
node_shortinterest -->|"reads candidates"| node_localdata
node_fmp -->|"uses paths"| node_paths
node_ark -->|"uses paths"| node_paths
node_insider -->|"uses paths"| node_paths
node_shortinterest -->|"uses paths"| node_paths
node_consensus -->|"uses paths"| node_paths
node_newsfeeds -->|"provides RSS"| node_news
node_news -->|"fetches articles"| node_websites
node_bing -->|"returns results"| node_search
node_search -->|"fetches pages"| node_websites
node_news -->|"uses logger"| node_logger
node_search -->|"uses logger"| node_logger

click node_scheduler "https://github.com/perennial-asian-boiz/perennialbackend/blob/main/development/backend/src/services/consensus_watchlist/scheduler/cron.py"
click node_consensus "https://github.com/perennial-asian-boiz/perennialbackend/blob/main/development/backend/src/services/consensus_watchlist/consensus.py"
click node_localdata "https://github.com/perennial-asian-boiz/perennialbackend/tree/main/development/database/local_data"
click node_fmp "https://github.com/perennial-asian-boiz/perennialbackend/blob/main/development/backend/src/services/consensus_watchlist/fetchers/fmp.py"
click node_ark "https://github.com/perennial-asian-boiz/perennialbackend/blob/main/development/backend/src/services/consensus_watchlist/fetchers/ark.py"
click node_insider "https://github.com/perennial-asian-boiz/perennialbackend/blob/main/development/backend/src/services/consensus_watchlist/fetchers/insider.py"
click node_shortinterest "https://github.com/perennial-asian-boiz/perennialbackend/blob/main/development/backend/src/services/consensus_watchlist/fetchers/short_interest.py"
click node_paths "https://github.com/perennial-asian-boiz/perennialbackend/blob/main/development/backend/src/services/consensus_watchlist/paths.py"
click node_news "https://github.com/perennial-asian-boiz/perennialbackend/blob/main/development/backend/src/services/news/news_fetcher.py"
click node_search "https://github.com/perennial-asian-boiz/perennialbackend/blob/main/development/backend/src/services/search_engine/search_engine.py"
click node_logger "https://github.com/perennial-asian-boiz/perennialbackend/blob/main/development/backend/src/utils/logger.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_scheduler,node_consensus,node_localdata toneBlue
class node_fmp,node_ark,node_insider,node_shortinterest toneAmber
class node_news,node_search,node_nasdaq toneMint
class node_paths,node_logger toneRose
class node_operator,node_financialmodelingprep,node_arkservice,node_securitiesdb,node_newsfeeds,node_bing,node_websites toneIndigo
```

## Components

### Watchlist pipeline

- [Pipeline scheduler](https://github.com/perennial-asian-boiz/perennialbackend/blob/main/development/backend/src/services/consensus_watchlist/scheduler/cron.py): `development/backend/src/services/consensus_watchlist/scheduler/cron.py`
- [Signal aggregation](https://github.com/perennial-asian-boiz/perennialbackend/blob/main/development/backend/src/services/consensus_watchlist/consensus.py): `development/backend/src/services/consensus_watchlist/consensus.py`
- [Local JSON data](https://github.com/perennial-asian-boiz/perennialbackend/tree/main/development/database/local_data): `development/database/local_data`

### Signal collection

- [Congress trades](https://github.com/perennial-asian-boiz/perennialbackend/blob/main/development/backend/src/services/consensus_watchlist/fetchers/fmp.py): `development/backend/src/services/consensus_watchlist/fetchers/fmp.py`
- [ARK holdings](https://github.com/perennial-asian-boiz/perennialbackend/blob/main/development/backend/src/services/consensus_watchlist/fetchers/ark.py): `development/backend/src/services/consensus_watchlist/fetchers/ark.py`
- [Insider activity](https://github.com/perennial-asian-boiz/perennialbackend/blob/main/development/backend/src/services/consensus_watchlist/fetchers/insider.py): `development/backend/src/services/consensus_watchlist/fetchers/insider.py`
- [Short interest](https://github.com/perennial-asian-boiz/perennialbackend/blob/main/development/backend/src/services/consensus_watchlist/fetchers/short_interest.py): `development/backend/src/services/consensus_watchlist/fetchers/short_interest.py`

### Content ingestion

- [News ingestion](https://github.com/perennial-asian-boiz/perennialbackend/blob/main/development/backend/src/services/news/news_fetcher.py): `development/backend/src/services/news/news_fetcher.py`
- [Web search ingestion](https://github.com/perennial-asian-boiz/perennialbackend/blob/main/development/backend/src/services/search_engine/search_engine.py): `development/backend/src/services/search_engine/search_engine.py`

### Shared runtime

- [Path resolution](https://github.com/perennial-asian-boiz/perennialbackend/blob/main/development/backend/src/services/consensus_watchlist/paths.py): `development/backend/src/services/consensus_watchlist/paths.py`
- [Logging utility](https://github.com/perennial-asian-boiz/perennialbackend/blob/main/development/backend/src/utils/logger.py): `development/backend/src/utils/logger.py`

### Other components

- **Operator**
- **Financial Modeling Prep**
- **ARK holdings service**
- **SecuritiesDB**
- **Nasdaq API**
- **News RSS feeds**
- **Bing search**
- **Article websites**

## Connections

- Operator → Pipeline scheduler: starts (evidence: `README.md`)
- Pipeline scheduler → Congress trades: runs (evidence: `development/backend/src/services/consensus_watchlist/scheduler/cron.py`)
- Pipeline scheduler → ARK holdings: runs (evidence: `development/backend/src/services/consensus_watchlist/scheduler/cron.py`)
- Pipeline scheduler → Insider activity: runs (evidence: `development/backend/src/services/consensus_watchlist/scheduler/cron.py`)
- Pipeline scheduler → Short interest: runs (evidence: `development/backend/src/services/consensus_watchlist/scheduler/cron.py`)
- Pipeline scheduler → Signal aggregation: runs (evidence: `development/backend/src/services/consensus_watchlist/scheduler/cron.py`)
- Congress trades → Financial Modeling Prep: fetches trades (evidence: `development/backend/src/services/consensus_watchlist/fetchers/fmp.py`)
- ARK holdings → ARK holdings service: fetches holdings (evidence: `development/backend/src/services/consensus_watchlist/fetchers/ark.py`)
- Insider activity → SecuritiesDB: requests activity (evidence: `development/backend/src/services/consensus_watchlist/fetchers/insider.py`)
- Short interest → Nasdaq API: fetches data (evidence: `development/backend/src/services/consensus_watchlist/fetchers/short_interest.py`)
- Congress trades → Local JSON data: writes trades
- ARK holdings → Local JSON data: writes holdings (evidence: `development/backend/src/services/consensus_watchlist/fetchers/ark.py`)
- Insider activity → Local JSON data: writes results
- Short interest → Local JSON data: writes results
- Signal aggregation → Local JSON data: reads signals (evidence: `development/backend/src/services/consensus_watchlist/consensus.py`)
- Insider activity → Local JSON data: reads candidate tickers (evidence: `development/backend/src/services/consensus_watchlist/fetchers/insider.py`)
- Short interest → Local JSON data: reads candidates (evidence: `development/backend/src/services/consensus_watchlist/fetchers/short_interest.py`)
- Congress trades → Path resolution: uses paths (evidence: `development/backend/src/services/consensus_watchlist/fetchers/fmp.py`)
- ARK holdings → Path resolution: uses paths (evidence: `development/backend/src/services/consensus_watchlist/fetchers/ark.py`)
- Insider activity → Path resolution: uses paths (evidence: `development/backend/src/services/consensus_watchlist/fetchers/insider.py`)
- Short interest → Path resolution: uses paths (evidence: `development/backend/src/services/consensus_watchlist/fetchers/short_interest.py`)
- Signal aggregation → Path resolution: uses paths (evidence: `development/backend/src/services/consensus_watchlist/consensus.py`)
- News RSS feeds → News ingestion: provides RSS (evidence: `development/backend/src/services/news/news_fetcher.py`)
- News ingestion → Article websites: fetches articles (evidence: `development/backend/src/services/news/news_fetcher.py`)
- Bing search → Web search ingestion: returns results (evidence: `development/backend/src/services/search_engine/search_engine.py`)
- Web search ingestion → Article websites: fetches pages (evidence: `development/backend/src/services/search_engine/search_engine.py`)
- News ingestion → Logging utility: uses logger (evidence: `development/backend/src/services/news/news_fetcher.py`)
- Web search ingestion → Logging utility: uses logger (evidence: `development/backend/src/services/search_engine/search_engine.py`)
