How 50shadescode/data-gathering-with-api- works
Project Type and Purpose
• This is a data-gathering Python application (a single-node ETL pipeline) implemented in a Jupyter Notebook.
• Purpose:
• Fetch NASA’s Astronomy Picture of the Day (APOD) and display it.
• Retrieve Near-Earth Object (NEO) asteroid data, process it into tabular form, and save it as CSV.
Main Components
a. Notebook / Python Runtime
b. HTTP Client Layer (“requests” library)
c. NASA External APIs
– APOD API endpoint
– Asteroids-NeoWs API endpoint
d. JSON Parser (built-in in Python via response.json())
e. Data Processing Module (pandas DataFrame builder & transformer)
f. Subsystem (IPython.display.Image for APOD)
g. Storage Subsystem (CSV file output via pandas.to_csv())
Relationships & Data Flow
1. User triggers the notebook → Python runtime
2. Python runtime uses requests → sends GET to APOD API → receives JSON
3. JSON flows into Display Subsystem → renders image + prints title/explanation
4. Python runtime uses requests → sends GET to NeoWs API → receives JSON
5. JSON flows into Data Processing Module → loops through near_earth_objects → constructs list of dicts → pandas DataFrame
6. DataFrame → Storage Subsystem → writes asteroid_data.csv
Architectural Patterns & Principles
• ETL-style pipeline: Extract (API calls), Transform (JSON → DataFrame), Load (display + CSV).
• Single-process, linear data flow – no microservices or separate backend.
• Separation of concerns:
• External API access (requests)
• Data transformation (pandas)
• Presentation (IPython.display)
• Persistence (CSV)
Technology & Framework Callouts
• Python 3.x
• requests (HTTP client)
• pandas (tabular data)
• IPython.display (notebook UI)
• NASA public APIs (APOD + NeoWs)