# vindyang/agrigator architecture

> Architecture diagram of the GitHub repository vindyang/agrigator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vindyang/agrigator
- Repository: https://github.com/vindyang/agrigator

## Overview

#### Purpose and System Shape
TinyfishHack is an agricultural monitoring and advisory system. It combines Indonesian commodity prices, crop-pest alerts, agricultural news, and weather inputs into stored signals and farmer-facing advisories. The repository is split into a Python backend and a Next.js frontend.

#### Backend Entry and Runtime
The backend application entry point is `backend/main.py`, with runtime settings in `backend/config.py`. Its Python dependencies are declared in `backend/requirements.txt`. A scheduler coordinates recurring collection and processing work through `backend/scheduler.py`; this makes the system a continuously refreshed data pipeline rather than a request-only API.

#### Data Persistence and Domain Model
`backend/database.py` establishes the persistence boundary, while `backend/cache.py` provides a cache layer for frequently used or transient data. Core domain records are represented by:
- `backend/models/price.py` for commodity price observations
- `backend/models/alert.py` for agricultural/pest alerts
- `backend/models/advisory.py` for generated recommendations

Database setup and operational data scripts live in `scripts/create_tables.py`, `scripts/list_tables.py`, and `scripts/seed_demo_data.py`.

#### External Collection Layer
Scrapers isolate external-source integration from agent logic:
- `backend/scrapers/kemendag_prices.py` retrieves Ministry of Trade price data.
- `backend/scrapers/bbpopt_alerts.py` retrieves BBPOPT pest or crop-protection alerts.
- `backend/scrapers/pertanian_news.py` collects agricultural news.
- `backend/scrapers/weather.py` supplies weather inputs.

These collectors feed normalized/stored domain data, creating a boundary between unreliable external formats and internal models.

#### Signal and Advisory Pipeline
The agent subsystem turns collected data into actionable output:
- `backend/agents/orchestrator.py` coordinates the analysis workflow.
- `backend/agents/signal_detector.py` identifies relevant changes, risks, or notable conditions from prices, alerts, news, and weather.
- `backend/agents/advisory_generator.py` produces advisory content from detected signals.
- `backend/agents/tools.py` centralizes agent-accessible data operations or integrations.

The principal flow is: scheduled scraping → persistence/cache → orchestration → signal detection → advisory generation → API/UI consumption.

#### Frontend Delivery
The frontend is a Next.js application, rooted at `frontend/app/layout.tsx` and `frontend/app/page.tsx`; the primary product view is `frontend/app/dashboard/page.tsx`. It communicates with the backend through `frontend/lib/api.ts`. Key presentation units include `frontend/components/AdvisoryCard.tsx`, `frontend/components/ChatFeed.tsx`, and `frontend/components/StatusBar.tsx`, which surface generated advice, conversational/history-like content, and system/data state.

#### Infrastructure Boundaries
The major runtime boundary is Python backend services versus the TypeScript/Next.js frontend. External dependencies include the Kemendag, BBPOPT, news, and weather sources represented by scrapers. Persistence and caching are backend-managed, with table lifecycle handled by scripts rather than frontend code. Debug scripts such as `debug_kemendag.py` and `debug_bbpopt.py` support source-integration diagnosis outside the production pipeline.

Main entry points: [`backend/scrapers/kemendag_prices.py`](https://github.com/vindyang/agrigator/blob/main/backend/scrapers/kemendag_prices.py) (Source Scrapers), [`backend/database.py`](https://github.com/vindyang/agrigator/blob/main/backend/database.py) (Database), [`backend/agents/signal_detector.py`](https://github.com/vindyang/agrigator/blob/main/backend/agents/signal_detector.py) (Signal Detector), [`frontend/app/dashboard/page.tsx`](https://github.com/vindyang/agrigator/blob/main/frontend/app/dashboard/page.tsx) (Farmer Dashboard)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_collection["External Collection"]
  node_kemendag["Kemendag Prices<br/>external source"]
  node_bbpopt["BBPOPT Alerts<br/>external source"]
  node_news_source["Agricultural News<br/>external source"]
  node_weather_source["Weather Data<br/>external source"]
  node_scrapers["Source Scrapers<br/>collection adapters<br/>[kemendag_prices.py]"]
end

subgraph group_backend["Python Backend"]
  node_scheduler(("Scheduler<br/>recurring jobs<br/>[scheduler.py]"))
  node_database[("Database<br/>persistence boundary<br/>[database.py]")]
  node_cache["Cache<br/>[cache.py]"]
  node_domain_models["Domain Models<br/>price, alert, advisory<br/>[advisory.py]"]
  node_orchestrator["Analysis Orchestrator<br/>agent workflow<br/>[orchestrator.py]"]
  node_signal_detector["Signal Detector<br/>agent analysis<br/>[signal_detector.py]"]
  node_advisory_generator["Advisory Generator<br/>agent generation"]
  node_backend_api["Backend API<br/>Python application<br/>[main.py]"]
end

subgraph group_frontend["Next.js Frontend"]
  node_frontend_api["API Client<br/>frontend data client<br/>[api.ts]"]
  node_dashboard["Farmer Dashboard<br/>primary product view<br/>[page.tsx]"]
  node_frontend_shell["App Shell<br/>Next.js entry<br/>[layout.tsx]"]
  node_dashboard_components["Dashboard Components<br/>advisory and status UI<br/>[AdvisoryCard.tsx]"]
end

node_scheduler -->|"triggers"| node_scrapers
node_kemendag -->|"price data"| node_scrapers
node_bbpopt -->|"pest alerts"| node_scrapers
node_news_source -->|"news"| node_scrapers
node_weather_source -->|"weather inputs"| node_scrapers
node_scrapers -->|"normalizes"| node_domain_models
node_domain_models -->|"persists"| node_database
node_scrapers -->|"refreshes"| node_cache
node_scheduler -->|"runs analysis"| node_orchestrator
node_orchestrator -->|"coordinates"| node_signal_detector
node_database -->|"observations and alerts"| node_signal_detector
node_cache -.->|"transient inputs"| node_signal_detector
node_signal_detector -->|"detected signals"| node_advisory_generator
node_advisory_generator -->|"stores advisories"| node_database
node_database -->|"serves records"| node_backend_api
node_frontend_api -->|"requests data"| node_backend_api
node_frontend_shell -->|"routes to"| node_dashboard
node_dashboard -->|"loads data"| node_frontend_api
node_dashboard_components -->|"renders in"| node_dashboard

click node_scheduler "https://github.com/vindyang/agrigator/blob/main/backend/scheduler.py"
click node_scrapers "https://github.com/vindyang/agrigator/blob/main/backend/scrapers/kemendag_prices.py"
click node_database "https://github.com/vindyang/agrigator/blob/main/backend/database.py"
click node_cache "https://github.com/vindyang/agrigator/blob/main/backend/cache.py"
click node_domain_models "https://github.com/vindyang/agrigator/blob/main/backend/models/advisory.py"
click node_orchestrator "https://github.com/vindyang/agrigator/blob/main/backend/agents/orchestrator.py"
click node_signal_detector "https://github.com/vindyang/agrigator/blob/main/backend/agents/signal_detector.py"
click node_advisory_generator "https://github.com/vindyang/agrigator/blob/main/backend/agents/advisory_generator.py"
click node_backend_api "https://github.com/vindyang/agrigator/blob/main/backend/main.py"
click node_frontend_api "https://github.com/vindyang/agrigator/blob/main/frontend/lib/api.ts"
click node_dashboard "https://github.com/vindyang/agrigator/blob/main/frontend/app/dashboard/page.tsx"
click node_frontend_shell "https://github.com/vindyang/agrigator/blob/main/frontend/app/layout.tsx"
click node_dashboard_components "https://github.com/vindyang/agrigator/blob/main/frontend/components/AdvisoryCard.tsx"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_kemendag,node_bbpopt,node_news_source,node_weather_source,node_scrapers toneBlue
class node_scheduler,node_database,node_cache,node_domain_models,node_orchestrator,node_signal_detector,node_advisory_generator,node_backend_api toneAmber
class node_frontend_api,node_dashboard,node_frontend_shell,node_dashboard_components toneMint
```

## Components

### External Collection

Scheduled ingestion from agricultural data sources.

- **Kemendag Prices**: external source
- **BBPOPT Alerts**: external source
- **Agricultural News**: external source
- **Weather Data**: external source
- [Source Scrapers](https://github.com/vindyang/agrigator/blob/main/backend/scrapers/kemendag_prices.py): collection adapters · Normalizes unreliable external formats before storage. · `backend/scrapers/kemendag_prices.py`

### Python Backend

Persistent data, analysis, and API runtime.

- [Scheduler](https://github.com/vindyang/agrigator/blob/main/backend/scheduler.py): recurring jobs · Coordinates continuous collection and processing. · `backend/scheduler.py`
- [Database](https://github.com/vindyang/agrigator/blob/main/backend/database.py): persistence boundary · Stores normalized observations, alerts, and advisories. · `backend/database.py`
- [Cache](https://github.com/vindyang/agrigator/blob/main/backend/cache.py): cache layer · Holds frequently used or transient data. · `backend/cache.py`
- [Domain Models](https://github.com/vindyang/agrigator/blob/main/backend/models/advisory.py): price, alert, advisory · `backend/models/advisory.py`
- [Analysis Orchestrator](https://github.com/vindyang/agrigator/blob/main/backend/agents/orchestrator.py): agent workflow · Coordinates advisory analysis. · `backend/agents/orchestrator.py`
- [Signal Detector](https://github.com/vindyang/agrigator/blob/main/backend/agents/signal_detector.py): agent analysis · Finds changes, risks, and notable conditions. · `backend/agents/signal_detector.py`
- [Advisory Generator](https://github.com/vindyang/agrigator/blob/main/backend/agents/advisory_generator.py): agent generation · Turns detected signals into farmer recommendations. · `backend/agents/advisory_generator.py`
- [Backend API](https://github.com/vindyang/agrigator/blob/main/backend/main.py): Python application · Serves stored signals and generated advisories. · `backend/main.py`

### Next.js Frontend

Farmer-facing dashboard application.

- [API Client](https://github.com/vindyang/agrigator/blob/main/frontend/lib/api.ts): frontend data client · `frontend/lib/api.ts`
- [Farmer Dashboard](https://github.com/vindyang/agrigator/blob/main/frontend/app/dashboard/page.tsx): primary product view · Displays advice, conversation-like history, and data status. · `frontend/app/dashboard/page.tsx`
- [App Shell](https://github.com/vindyang/agrigator/blob/main/frontend/app/layout.tsx): Next.js entry · `frontend/app/layout.tsx`
- [Dashboard Components](https://github.com/vindyang/agrigator/blob/main/frontend/components/AdvisoryCard.tsx): advisory and status UI · `frontend/components/AdvisoryCard.tsx`

## Connections

- Scheduler → Source Scrapers: triggers
- Kemendag Prices → Source Scrapers: price data
- BBPOPT Alerts → Source Scrapers: pest alerts
- Agricultural News → Source Scrapers: news
- Weather Data → Source Scrapers: weather inputs
- Source Scrapers → Domain Models: normalizes
- Domain Models → Database: persists
- Source Scrapers → Cache: refreshes
- Scheduler → Analysis Orchestrator: runs analysis
- Analysis Orchestrator → Signal Detector: coordinates
- Database → Signal Detector: observations and alerts
- Cache → Signal Detector: transient inputs
- Signal Detector → Advisory Generator: detected signals
- Advisory Generator → Database: stores advisories
- Database → Backend API: serves records
- API Client → Backend API: requests data
- App Shell → Farmer Dashboard: routes to
- Farmer Dashboard → API Client: loads data
- Dashboard Components → Farmer Dashboard: renders in