# anudeepkotaru/corteva-coding-challenge architecture

> Architecture diagram of the GitHub repository anudeepkotaru/corteva-coding-challenge, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/anudeepkotaru/corteva-coding-challenge
- Repository: https://github.com/anudeepkotaru/corteva-coding-challenge

## Overview

##### Project Type and Purpose
- This is a Django-based RESTful API that ingests raw weather text files, computes yearly station statistics, and exposes both raw and aggregated data via HTTP endpoints.
- Core use cases:
– Ingest raw station data (`wx_data/*.txt`) into a PostgreSQL database.
– Compute per-station, per-year statistics.
– Serve records and stats over RESTful endpoints (with filtering & pagination).
– Provide interactive docs via Swagger.

##### File Structure
- Top-level:
– `manage.py`, `pyproject.toml`, `requirements.txt` (project entry & dependencies).
– `weather_project/` (Django settings, URL router, WSGI).
– `weather/` (app folder: models, serializers, views, filters, migrations, tests).
– `weather/management/commands/` (two CLI commands for ingestion and stats).
– `wx_data/` and `yld_data/` (raw input files).
- Architectural hints:
– Two distinct pipelines (ingest & compute).
– A single monolithic Django app exposing REST endpoints.
– PostgreSQL as the persistent store.

##### Extract Architecture Details from the README
- Django REST Framework powers the API.
- Management commands are scheduled or invoked manually.
- Supports Swagger (`drf-yasg`) and `django-filter`.
- Extra-credit deployment on AWS (RDS, Docker, Elastic Beanstalk/ECS, Lambda).

##### Main Components & Their Relationships
a. Raw Data Storage
– A folder (`wx_data/`, `yld_data/`) holding daily text files.
b. Ingestion Service
– `ingest_weather_data` command: reads raw files, parses lines into `WeatherRecord` models.
– Logs progress in `weather_ingestion.log`.
c. Compute Service
– `calculate_weather_stats` command: aggregates `WeatherRecord` into `WeatherStat` per station/year.
– Logs to `compute_stats.log`.
d. Primary Data Store
– PostgreSQL database (two tables: `weather_weatherrecord`, `weather_weatherstat`).
e. Django REST API
– DRF ViewSets (`weather/views.py`) for records and stats.
– Serializers (`weather/serializers.py`) define JSON schema.
– Filters (`weather/filters.py`) handle query parameters.
– URL routes (`weather/urls.py`, included in `weather_project/urls.py`).
f. Clients & External Interfaces
– Web/API clients hitting `/api/weather/` and `/api/weather/stats/`.
– Swagger UI at `/swagger/`.
– (Optional) AWS services for scheduling ingestion (Lambda + EventBridge).

##### Architectural Patterns & Design Principles
- Batch Pipeline Pattern: two separate management-command pipelines for ETL and analytics.
- Layered MVC-style design within Django: models → serializers → views → URL routing.
- Clear separation of concerns: ingestion, compute, API.
- Use of filters and pagination per REST best-practices.
- Logging at each pipeline stage.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Raw Data Storage
    subgraph "Raw Data Storage" 
        direction TB
        WX[/"wx_data/\n(raw weather files)"/]:::datastore
        YLD[/"yld_data/\n(raw yield files)"/]:::datastore
    end

    %% Batch Pipeline
    subgraph "Batch Pipeline" 
        direction TB
        INGEST["ingest_weather_data\n(management command)"]:::service
        COMPUTE["calculate_weather_stats\n(management command)"]:::service
    end

    %% Database
    DB[(PostgreSQL\nweather_weatherrecord\n& weather_weatherstat)]:::datastore

    %% Django Application
    subgraph "Django Application"
        direction TB
        SETTINGS["settings.py\n(Django settings)"]:::config
        WSGI["wsgi.py\n(WSGI entrypoint)"]:::config
        MODELS["models.py\n(WeatherRecord & WeatherStat)"]:::api
        SERIAL["serializers.py\n(JSON schema)"]:::api
        FILT["filters.py\n(query handlers)"]:::api
        VIEWS["views.py\n(ViewSets)"]:::api
        URLS_APP["weather/urls.py\n(app routes)"]:::api
        URLS_PROJ["weather_project/urls.py\n(project router)"]:::config
    end

    %% Management Bootstrap
    MANAGE["manage.py\n(command bootstrap)"]:::config

    %% Clients & Swagger
    subgraph "Clients & External"
        direction TB
        CLIENTS["API Clients\n(HTTP GET/POST)"]:::external
        SWAGGER["Swagger UI\n(/swagger/)"]:::external
        AWS["AWS Scheduler\n(EventBridge → Lambda)"]:::external
        TESTS["test_api.py\n(API Test Suite)"]:::external
    end

    %% Dependencies
    DEPS["requirements.txt\n& pyproject.toml"]:::config

    %% Data Flows
    WX -->|parse & write records| INGEST
    YLD -->|parse & write records| INGEST
    INGEST -->|writes Raw Records| DB
    DB -->|read records| COMPUTE
    COMPUTE -->|writes WeatherStat| DB
    DB -->|reads data| VIEWS
    VIEWS -->|serializes| SERIAL
    SERIAL -->|applies filters| FILT
    FILT -->|returns JSON| CLIENTS
    URLS_APP -->|routes requests| VIEWS
    URLS_PROJ -->|includes app URLs & Swagger| URLS_APP
    SETTINGS -->|configures DB & DRF| VIEWS
    WSGI -->|serves Django app| URLS_PROJ
    MANAGE -->|runs commands| INGEST
    MANAGE -->|runs commands| COMPUTE
    CLIENTS -->|calls| URLS_PROJ
    CLIENTS -->|views docs| SWAGGER
    AWS -->|triggers| INGEST
    TESTS -->|tests| URLS_PROJ
    DEPS -->|provides| SETTINGS

    %% Click Events
    click WX "https://github.com/anudeepkotaru/corteva-coding-challenge/tree/main/wx_data/"
    click YLD "https://github.com/anudeepkotaru/corteva-coding-challenge/tree/main/yld_data/"
    click INGEST "https://github.com/anudeepkotaru/corteva-coding-challenge/blob/main/weather/management/commands/ingest_weather_data.py"
    click COMPUTE "https://github.com/anudeepkotaru/corteva-coding-challenge/blob/main/weather/management/commands/calculate_weather_stats.py"
    click MODELS "https://github.com/anudeepkotaru/corteva-coding-challenge/blob/main/weather/models.py"
    click SERIAL "https://github.com/anudeepkotaru/corteva-coding-challenge/blob/main/weather/serializers.py"
    click FILT "https://github.com/anudeepkotaru/corteva-coding-challenge/blob/main/weather/filters.py"
    click VIEWS "https://github.com/anudeepkotaru/corteva-coding-challenge/blob/main/weather/views.py"
    click URLS_APP "https://github.com/anudeepkotaru/corteva-coding-challenge/blob/main/weather/urls.py"
    click URLS_PROJ "https://github.com/anudeepkotaru/corteva-coding-challenge/blob/main/weather_project/urls.py"
    click SETTINGS "https://github.com/anudeepkotaru/corteva-coding-challenge/blob/main/weather_project/settings.py"
    click WSGI "https://github.com/anudeepkotaru/corteva-coding-challenge/blob/main/weather_project/wsgi.py"
    click MANAGE "https://github.com/anudeepkotaru/corteva-coding-challenge/blob/main/manage.py"
    click TESTS "https://github.com/anudeepkotaru/corteva-coding-challenge/blob/main/weather/tests/test_api.py"
    click DEPS "https://github.com/anudeepkotaru/corteva-coding-challenge/blob/main/requirements.txt"

    %% Styles
    classDef datastore fill:#a8d0e6,stroke:#1e3d59,color:#1e3d59
    classDef service fill:#a3d9a5,stroke:#1e3d59,color:#1e3d59
    classDef api fill:#f8b195,stroke:#f67280,color:#1e3d59
    classDef external fill:#cbaacb,stroke:#1e3d59,color:#1e3d59
    classDef config fill:#e0e0e0,stroke:#1e3d59,color:#1e3d59
```
