# 50shadescode/data-gathering-with-api- architecture

> Architecture diagram of the GitHub repository 50shadescode/data-gathering-with-api-, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/50shadescode/data-gathering-with-api-
- Repository: https://github.com/50shadescode/data-gathering-with-api-

## Overview

##### Project Type and Purpose
- This is a data-gathering Python application (a single-node ETL pipeline) implemented in a Jupyter Notebook.
- Purpose:
- Fetch NASA’s Astronomy Picture of the Day (APOD) and display it.
- Retrieve Near-Earth Object (NEO) asteroid data, process it into tabular form, and save it as CSV.

##### Main Components
a. Notebook / Python Runtime
b. HTTP Client Layer (“requests” library)
c. NASA External APIs
– APOD API endpoint
– Asteroids-NeoWs API endpoint
d. JSON Parser (built-in in Python via response.json())
e. Data Processing Module (pandas DataFrame builder & transformer)
f. Subsystem (IPython.display.Image for APOD)
g. Storage Subsystem (CSV file output via pandas.to_csv())

##### Relationships & Data Flow
1. User triggers the notebook → Python runtime
2. Python runtime uses requests → sends GET to APOD API → receives JSON
3. JSON flows into Display Subsystem → renders image + prints title/explanation
4. Python runtime uses requests → sends GET to NeoWs API → receives JSON
5. JSON flows into Data Processing Module → loops through near_earth_objects → constructs list of dicts → pandas DataFrame
6. DataFrame → Storage Subsystem → writes asteroid_data.csv

##### Architectural Patterns & Principles
- ETL-style pipeline: Extract (API calls), Transform (JSON → DataFrame), Load (display + CSV).
- Single-process, linear data flow – no microservices or separate backend.
- Separation of concerns:
- External API access (requests)
- Data transformation (pandas)
- Presentation (IPython.display)
- Persistence (CSV)

##### Technology & Framework Callouts
- Python 3.x
- requests (HTTP client)
- pandas (tabular data)
- IPython.display (notebook UI)
- NASA public APIs (APOD + NeoWs)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Components
    Notebook(("Notebook / Python Runtime")):::runtime
    Requests["requests\n(HTTP Client)"]:::library
    APOD["APOD API Endpoint"]:::external
    NEO["Asteroids-NeoWs API Endpoint"]:::external
    JSONP["JSON Parser\n(response.json())"]:::library
    PAND["Data Processing\n(pandas DataFrame)"]:::library

    subgraph "Load/Output"
        DISPLY["Display Subsystem\n(IPython.display.Image)"]:::library
        CSV[("asteroid_data.csv")]:::storage
    end

    %% Data Flow
    Notebook -->|"invoke HTTP Client"| Requests
    Requests -->|"HTTP GET"| APOD
    APOD -->|"JSON response"| JSONP
    JSONP -->|"parsed JSON"| DISPLY

    Requests -->|"HTTP GET"| NEO
    NEO -->|"JSON response"| JSONP
    JSONP -->|"parsed JSON"| PAND
    PAND -->|"DataFrame write"| CSV

    %% Click Events
    click Notebook "https://github.com/50shadescode/data-gathering-with-api-/blob/main/Data Gathering with API .ipynb"
    click CSV "https://github.com/50shadescode/data-gathering-with-api-/blob/main/asteroid_data.csv"

    %% Styles
    classDef runtime fill:#DDDDDD,stroke:#000000,color:#000000;
    classDef external fill:#ADD8E6,stroke:#0000FF,color:#000000;
    classDef library fill:#C6EFCE,stroke:#006400,color:#000000;
    classDef storage fill:#FFF2CC,stroke:#996600,color:#000000;
```
