vindyang/agrigator

Purpose and System Shape

TinyfishHack is an agricultural monitoring and advisory system. It combines Indonesian commodity prices, crop-pest alerts, agricultural news, and weather inputs into stored signals and farmer-facing advisories. The repository is split into a Python backend and a Next.js frontend.

Backend Entry and Runtime

The backend application entry point is backend/main.py, with runtime settings in backend/config.py. Its Python dependencies are declared in backend/requirements.txt. A scheduler coordinates recurring collection and processing work through backend/scheduler.py; this makes the system a continuously refreshed data pipeline rather than a request-only API.

Data Persistence and Domain Model

backend/database.py establishes the persistence boundary, while backend/cache.py provides a cache layer for frequently used or transient data. Core domain records are represented by:

• backend/models/price.py for commodity price observations

• backend/models/alert.py for agricultural/pest alerts

• backend/models/advisory.py for generated recommendations

Database setup and operational data scripts live in scripts/create_tables.py, scripts/list_tables.py, and scripts/seed_demo_data.py.

External Collection Layer

Scrapers isolate external-source integration from agent logic:

• backend/scrapers/kemendag_prices.py retrieves Ministry of Trade price data.

• backend/scrapers/bbpopt_alerts.py retrieves BBPOPT pest or crop-protection alerts.

• backend/scrapers/pertanian_news.py collects agricultural news.

• backend/scrapers/weather.py supplies weather inputs.

These collectors feed normalized/stored domain data, creating a boundary between unreliable external formats and internal models.

Signal and Advisory Pipeline

The agent subsystem turns collected data into actionable output:

• backend/agents/orchestrator.py coordinates the analysis workflow.

• backend/agents/signal_detector.py identifies relevant changes, risks, or notable conditions from prices, alerts, news, and weather.

• backend/agents/advisory_generator.py produces advisory content from detected signals.

• backend/agents/tools.py centralizes agent-accessible data operations or integrations.

The principal flow is: scheduled scraping → persistence/cache → orchestration → signal detection → advisory generation → API/UI consumption.

Frontend Delivery

The frontend is a Next.js application, rooted at frontend/app/layout.tsx and frontend/app/page.tsx; the primary product view is frontend/app/dashboard/page.tsx. It communicates with the backend through frontend/lib/api.ts. Key presentation units include frontend/components/AdvisoryCard.tsx, frontend/components/ChatFeed.tsx, and frontend/components/StatusBar.tsx, which surface generated advice, conversational/history-like content, and system/data state.

Infrastructure Boundaries

The major runtime boundary is Python backend services versus the TypeScript/Next.js frontend. External dependencies include the Kemendag, BBPOPT, news, and weather sources represented by scrapers. Persistence and caching are backend-managed, with table lifecycle handled by scripts rather than frontend code. Debug scripts such as debug_kemendag.py and debug_bbpopt.py support source-integration diagnosis outside the production pipeline.