# grascii/datasets architecture

> Architecture diagram of the GitHub repository grascii/datasets, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/grascii/datasets
- Repository: https://github.com/grascii/datasets

## Overview

This repository packages four Gregg shorthand image datasets and scripts for preparing their labels and publishing them. The user-facing workflow is to provide dictionary sources and an image dataset, normalize dictionary entries, generate image metadata, then load the image folder and push it to Hugging Face. The README identifies the four historical word and phrase collections. The script excerpts establish the preparation and upload behavior; the specific dataset-directory wiring is not shown, so shared script edges represent their parameterized inputs rather than a fixed run configuration.

Main entry points: [`scripts/metadata.py`](https://github.com/grascii/datasets/blob/main/scripts/metadata.py) (Metadata generator), [`scripts/push.py`](https://github.com/grascii/datasets/blob/main/scripts/push.py) (Dataset uploader), [`images/gregg-preanniversary-words`](https://github.com/grascii/datasets/tree/main/images/gregg-preanniversary-words) (Preanniversary words), [`images/gregg-preanniversary-phrases`](https://github.com/grascii/datasets/tree/main/images/gregg-preanniversary-phrases) (Preanniversary phrases)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_assets["Dataset assets"]
  node_pre_words[("Preanniversary words")]
  node_pre_phrases[("Preanniversary phrases")]
  node_ann_words[("Anniversary words")]
  node_ann_phrases[("Anniversary phrases")]
  node_dict_sources["Dictionary sources"]
  node_normalized[("Normalized dictionary")]
end

subgraph group_prep["Dataset preparation"]
  node_normalizer["Dictionary normalizer<br/>[dictionary.py]"]
  node_metadata["Metadata generator<br/>[metadata.py]"]
end

subgraph group_publish["Publishing"]
  node_uploader["Dataset uploader<br/>[push.py]"]
end

node_huggingface{{"Hugging Face"}}

node_normalizer -->|"reads sources"| node_dict_sources
node_normalizer -->|"builds dictionary"| node_normalized
node_metadata -->|"loads entries"| node_normalized
node_metadata -->|"scans images"| node_pre_words
node_metadata -->|"scans images"| node_pre_phrases
node_metadata -->|"scans images"| node_ann_words
node_metadata -->|"scans images"| node_ann_phrases
node_metadata -->|"writes metadata"| node_pre_words
node_metadata -->|"writes metadata"| node_pre_phrases
node_metadata -->|"writes metadata"| node_ann_words
node_metadata -->|"writes metadata"| node_ann_phrases
node_uploader -->|"loads image folder"| node_pre_words
node_uploader -->|"loads image folder"| node_pre_phrases
node_uploader -->|"loads image folder"| node_ann_words
node_uploader -->|"loads image folder"| node_ann_phrases
node_uploader -->|"pushes dataset"| node_huggingface

click node_pre_words "https://github.com/grascii/datasets/tree/main/images/gregg-preanniversary-words"
click node_pre_phrases "https://github.com/grascii/datasets/tree/main/images/gregg-preanniversary-phrases"
click node_ann_words "https://github.com/grascii/datasets/tree/main/images/gregg-anniversary-words"
click node_ann_phrases "https://github.com/grascii/datasets/tree/main/images/gregg-anniversary-phrases"
click node_dict_sources "https://github.com/grascii/datasets/tree/main/dictionaries"
click node_normalizer "https://github.com/grascii/datasets/blob/main/scripts/dictionary.py"
click node_normalized "https://github.com/grascii/datasets/tree/main/dictionaries"
click node_metadata "https://github.com/grascii/datasets/blob/main/scripts/metadata.py"
click node_uploader "https://github.com/grascii/datasets/blob/main/scripts/push.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_pre_words,node_pre_phrases,node_ann_words,node_ann_phrases,node_dict_sources,node_normalized toneBlue
class node_normalizer,node_metadata toneAmber
class node_uploader toneMint
class node_huggingface toneIndigo
```

## Components

### Dataset assets

- [Preanniversary words](https://github.com/grascii/datasets/tree/main/images/gregg-preanniversary-words): `images/gregg-preanniversary-words`
- [Preanniversary phrases](https://github.com/grascii/datasets/tree/main/images/gregg-preanniversary-phrases): `images/gregg-preanniversary-phrases`
- [Anniversary words](https://github.com/grascii/datasets/tree/main/images/gregg-anniversary-words): `images/gregg-anniversary-words`
- [Anniversary phrases](https://github.com/grascii/datasets/tree/main/images/gregg-anniversary-phrases): `images/gregg-anniversary-phrases`
- [Dictionary sources](https://github.com/grascii/datasets/tree/main/dictionaries): `dictionaries`
- [Normalized dictionary](https://github.com/grascii/datasets/tree/main/dictionaries): `dictionaries`

### Dataset preparation

- [Dictionary normalizer](https://github.com/grascii/datasets/blob/main/scripts/dictionary.py): `scripts/dictionary.py`
- [Metadata generator](https://github.com/grascii/datasets/blob/main/scripts/metadata.py): `scripts/metadata.py`

### Publishing

- [Dataset uploader](https://github.com/grascii/datasets/blob/main/scripts/push.py): `scripts/push.py`

### Other components

- **Hugging Face**

## Connections

- Dictionary normalizer → Dictionary sources: reads sources (evidence: `scripts/dictionary.py`)
- Dictionary normalizer → Normalized dictionary: builds dictionary (evidence: `scripts/dictionary.py`)
- Metadata generator → Normalized dictionary: loads entries (evidence: `scripts/metadata.py`)
- Metadata generator → Preanniversary words: scans images (evidence: `scripts/metadata.py`)
- Metadata generator → Preanniversary phrases: scans images (evidence: `scripts/metadata.py`)
- Metadata generator → Anniversary words: scans images (evidence: `scripts/metadata.py`)
- Metadata generator → Anniversary phrases: scans images (evidence: `scripts/metadata.py`)
- Metadata generator → Preanniversary words: writes metadata (evidence: `scripts/metadata.py`)
- Metadata generator → Preanniversary phrases: writes metadata (evidence: `scripts/metadata.py`)
- Metadata generator → Anniversary words: writes metadata (evidence: `scripts/metadata.py`)
- Metadata generator → Anniversary phrases: writes metadata (evidence: `scripts/metadata.py`)
- Dataset uploader → Preanniversary words: loads image folder (evidence: `scripts/push.py`)
- Dataset uploader → Preanniversary phrases: loads image folder (evidence: `scripts/push.py`)
- Dataset uploader → Anniversary words: loads image folder (evidence: `scripts/push.py`)
- Dataset uploader → Anniversary phrases: loads image folder (evidence: `scripts/push.py`)
- Dataset uploader → Hugging Face: pushes dataset (evidence: `scripts/push.py`)
