How anudeepkotaru/corteva-coding-challenge works
Project Type and Purpose
• This is a Django-based RESTful API that ingests raw weather text files, computes yearly station statistics, and exposes both raw and aggregated data via HTTP endpoints.
• Core use cases:
– Ingest raw station data (wx_data/*.txt) into a PostgreSQL database.
– Compute per-station, per-year statistics.
– Serve records and stats over RESTful endpoints (with filtering & pagination).
– Provide interactive docs via Swagger.
File Structure
• Top-level:
– manage.py, pyproject.toml, requirements.txt (project entry & dependencies).
– weather_project/ (Django settings, URL router, WSGI).
– weather/ (app folder: models, serializers, views, filters, migrations, tests).
– weather/management/commands/ (two CLI commands for ingestion and stats).
– wx_data/ and yld_data/ (raw input files).
• Architectural hints:
– Two distinct pipelines (ingest & compute).
– A single monolithic Django app exposing REST endpoints.
– PostgreSQL as the persistent store.
Extract Architecture Details from the README
• Django REST Framework powers the API.
• Management commands are scheduled or invoked manually.
• Supports Swagger (drf-yasg) and django-filter.
• Extra-credit deployment on AWS (RDS, Docker, Elastic Beanstalk/ECS, Lambda).
Main Components & Their Relationships
a. Raw Data Storage
– A folder (wx_data/, yld_data/) holding daily text files.
b. Ingestion Service
– ingest_weather_data command: reads raw files, parses lines into WeatherRecord models.
– Logs progress in weather_ingestion.log.
c. Compute Service
– calculate_weather_stats command: aggregates WeatherRecord into WeatherStat per station/year.
– Logs to compute_stats.log.
d. Primary Data Store
– PostgreSQL database (two tables: weather_weatherrecord, weather_weatherstat).
e. Django REST API
– DRF ViewSets (weather/views.py) for records and stats.
– Serializers (weather/serializers.py) define JSON schema.
– Filters (weather/filters.py) handle query parameters.
– URL routes (weather/urls.py, included in weather_project/urls.py).
f. Clients & External Interfaces
– Web/API clients hitting /api/weather/ and /api/weather/stats/.
– Swagger UI at /swagger/.
– (Optional) AWS services for scheduling ingestion (Lambda + EventBridge).
Architectural Patterns & Design Principles
• Batch Pipeline Pattern: two separate management-command pipelines for ETL and analytics.
• Layered MVC-style design within Django: models → serializers → views → URL routing.
• Clear separation of concerns: ingestion, compute, API.
• Use of filters and pagination per REST best-practices.
• Logging at each pipeline stage.