# matdotviguier/histdata-fx-1-minute-data architecture

> Architecture diagram of the GitHub repository matdotviguier/histdata-fx-1-minute-data, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/matdotviguier/histdata-fx-1-minute-data
- Repository: https://github.com/matdotviguier/histdata-fx-1-minute-data

## Overview

This repository provides a HistData historical-market-data downloader, a bulk FX-pair download workflow, and a standalone timestamp-shifting utility. The main workflow selects a pair, period, platform, and timeframe; requests HistData’s download page and archive; then saves the ZIP locally. README-documented data formats and instrument coverage are represented as capabilities, but their dataset generation is not otherwise sampled. Edges reflect the sampled API and bulk script; the conversion utility is independent.

Main entry points: [`histdata/api.py`](https://github.com/matdotviguier/histdata-fx-1-minute-data/blob/master/histdata/api.py) (Download API), [`download_all_fx_data.py`](https://github.com/matdotviguier/histdata-fx-1-minute-data/blob/master/download_all_fx_data.py) (Bulk downloader), [`convert_est_to_target_time.py`](https://github.com/matdotviguier/histdata-fx-1-minute-data/blob/master/convert_est_to_target_time.py) (Timestamp converter), [`pairs.csv`](https://github.com/matdotviguier/histdata-fx-1-minute-data/blob/master/pairs.csv) (Pair catalog)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_download["Data download"]
  node_bulk["Bulk downloader"]
  node_pairlist["Pair catalog<br/>[pairs.csv]"]
  node_api["Download API<br/>[api.py]"]
  node_routing["Format routing<br/>[api.py]"]
  node_page["Download page<br/>[api.py]"]
  node_archive["Archive retrieval<br/>[api.py]"]
end

subgraph group_dataset["Local datasets"]
  node_localarchives[("Downloaded archives")]
  node_marketdata["Market data"]
end

subgraph group_utility["Data utility"]
  node_converter["Timestamp converter"]
  node_csvinput["Input CSV"]
  node_csvoutput["Shifted CSV"]
end

node_user(("Data user"))
node_histdata(("HistData service"))

node_user -->|"requests data"| node_api
node_user -->|"starts bulk download"| node_bulk
node_bulk -->|"reads pairs"| node_pairlist
node_bulk -->|"requests archives"| node_api
node_api -->|"selects endpoint"| node_routing
node_api -->|"fetches page"| node_page
node_page -->|"requests page"| node_histdata
node_page -->|"provides token"| node_archive
node_api -->|"posts download request"| node_archive
node_archive -->|"requests archive"| node_histdata
node_archive -->|"writes ZIP"| node_localarchives
node_user -->|"runs conversion"| node_converter
node_converter -->|"reads CSV"| node_csvinput
node_converter -->|"writes shifted CSV"| node_csvoutput
node_localarchives -.->|"contains downloaded data"| node_marketdata

click node_bulk "https://github.com/matdotviguier/histdata-fx-1-minute-data/blob/master/download_all_fx_data.py"
click node_pairlist "https://github.com/matdotviguier/histdata-fx-1-minute-data/blob/master/pairs.csv"
click node_api "https://github.com/matdotviguier/histdata-fx-1-minute-data/blob/master/histdata/api.py"
click node_routing "https://github.com/matdotviguier/histdata-fx-1-minute-data/blob/master/histdata/api.py"
click node_page "https://github.com/matdotviguier/histdata-fx-1-minute-data/blob/master/histdata/api.py"
click node_archive "https://github.com/matdotviguier/histdata-fx-1-minute-data/blob/master/histdata/api.py"
click node_converter "https://github.com/matdotviguier/histdata-fx-1-minute-data/blob/master/convert_est_to_target_time.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_bulk,node_pairlist,node_api,node_routing,node_page,node_archive,node_user toneBlue
class node_localarchives,node_marketdata toneAmber
class node_converter,node_csvinput,node_csvoutput toneMint
class node_histdata toneIndigo
```

## Components

### Data download

- [Bulk downloader](https://github.com/matdotviguier/histdata-fx-1-minute-data/blob/master/download_all_fx_data.py): `download_all_fx_data.py`
- [Pair catalog](https://github.com/matdotviguier/histdata-fx-1-minute-data/blob/master/pairs.csv): `pairs.csv`
- [Download API](https://github.com/matdotviguier/histdata-fx-1-minute-data/blob/master/histdata/api.py): `histdata/api.py`
- [Format routing](https://github.com/matdotviguier/histdata-fx-1-minute-data/blob/master/histdata/api.py): `histdata/api.py`
- [Download page](https://github.com/matdotviguier/histdata-fx-1-minute-data/blob/master/histdata/api.py): `histdata/api.py`
- [Archive retrieval](https://github.com/matdotviguier/histdata-fx-1-minute-data/blob/master/histdata/api.py): `histdata/api.py`

### Local datasets

- **Downloaded archives**
- **Market data**

### Data utility

- [Timestamp converter](https://github.com/matdotviguier/histdata-fx-1-minute-data/blob/master/convert_est_to_target_time.py): `convert_est_to_target_time.py`
- **Input CSV**
- **Shifted CSV**

### Other components

- **Data user**
- **HistData service**

## Connections

- Data user → Download API: requests data
- Data user → Bulk downloader: starts bulk download
- Bulk downloader → Pair catalog: reads pairs
- Bulk downloader → Download API: requests archives
- Download API → Format routing: selects endpoint
- Download API → Download page: fetches page
- Download page → HistData service: requests page
- Download page → Archive retrieval: provides token
- Download API → Archive retrieval: posts download request
- Archive retrieval → HistData service: requests archive
- Archive retrieval → Downloaded archives: writes ZIP
- Data user → Timestamp converter: runs conversion
- Timestamp converter → Input CSV: reads CSV
- Timestamp converter → Shifted CSV: writes shifted CSV
- Downloaded archives → Market data: contains downloaded data