# andrewcshi/ai-index-2025 architecture

> Architecture diagram of the GitHub repository andrewcshi/ai-index-2025, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/andrewcshi/ai-index-2025
- Repository: https://github.com/andrewcshi/ai-index-2025

## Overview

This repository gathers AI-related papers from six conferences, classifies them into topic categories, and stores paper records as CSVs for later analysis. A separate image-analysis branch compares Harry Potter-themed Midjourney generations using pixel and CNN similarity. The graph also includes country attribution and dataset statistics. Conference scraper internals are only partly sampled, so links to conference sites are documented at the README level; relationships involving unsampled processing and image outputs are omitted rather than inferred.

Main entry points: [`data`](https://github.com/andrewcshi/ai-index-2025/tree/main/data) (Paper CSV archive), [`main/aaai.py`](https://github.com/andrewcshi/ai-index-2025/blob/main/main/aaai.py) (AAAI scraper), [`main/aies.py`](https://github.com/andrewcshi/ai-index-2025/blob/main/main/aies.py) (AIES scraper), [`main/facct.py`](https://github.com/andrewcshi/ai-index-2025/blob/main/main/facct.py) (FAccT scraper)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_collection["Paper collection"]
  node_aaai["AAAI scraper<br/>[aaai.py]"]
  node_aies["AIES scraper<br/>[aies.py]"]
  node_facct["FAccT scraper<br/>[facct.py]"]
  node_iclr["ICLR scraper<br/>[iclr.py]"]
  node_icml["ICML scraper<br/>[icml.py]"]
  node_neurips["NeurIPS scraper<br/>[neurips.py]"]
end

subgraph group_analysis["Dataset analysis"]
  node_paper_data[("Paper CSV archive")]
  node_country_attribution["Country attribution<br/>[countries.py]"]
  node_city_data[("City reference data<br/>[cities.csv]")]
  node_institution_data[("Institution reference data<br/>[institutions.csv]")]
  node_revisions["Keyword revisions<br/>[revisions.py]"]
  node_statistics["Paper statistics<br/>[stats.py]"]
end

subgraph group_images["Image comparison"]
  node_image_scripts["Similarity scripts"]
  node_fewshot["Few-shot similarity<br/>[few_shot_sim.py]"]
  node_oneshot["One-shot similarity<br/>[one_shot_sim.py]"]
  node_generated_images["Midjourney image sets"]
end

node_user(("Repository user"))
node_conference_sites["Conference websites"]

node_user -->|"runs"| node_aaai
node_user -->|"runs"| node_aies
node_user -->|"runs"| node_facct
node_user -->|"runs"| node_iclr
node_user -->|"runs"| node_icml
node_user -->|"runs"| node_neurips
node_aaai -->|"scrapes"| node_conference_sites
node_aies -->|"scrapes"| node_conference_sites
node_facct -->|"scrapes"| node_conference_sites
node_iclr -->|"scrapes"| node_conference_sites
node_icml -->|"scrapes"| node_conference_sites
node_neurips -->|"scrapes"| node_conference_sites
node_aaai -->|"writes CSV"| node_paper_data
node_aies -->|"writes CSV"| node_paper_data
node_facct -->|"writes CSV"| node_paper_data
node_iclr -->|"writes CSV"| node_paper_data
node_icml -.->|"writes CSV"| node_paper_data
node_neurips -->|"writes CSV"| node_paper_data
node_country_attribution -->|"reads"| node_city_data
node_country_attribution -->|"reads"| node_institution_data
node_revisions -->|"reads and augments"| node_paper_data
node_statistics -->|"reads"| node_paper_data
node_user -.->|"runs"| node_fewshot
node_user -.->|"runs"| node_oneshot
node_fewshot -->|"compares images"| node_generated_images
node_oneshot -.->|"compares images"| node_generated_images

click node_aaai "https://github.com/andrewcshi/ai-index-2025/blob/main/main/aaai.py"
click node_aies "https://github.com/andrewcshi/ai-index-2025/blob/main/main/aies.py"
click node_facct "https://github.com/andrewcshi/ai-index-2025/blob/main/main/facct.py"
click node_iclr "https://github.com/andrewcshi/ai-index-2025/blob/main/main/iclr.py"
click node_icml "https://github.com/andrewcshi/ai-index-2025/blob/main/main/icml.py"
click node_neurips "https://github.com/andrewcshi/ai-index-2025/blob/main/main/neurips.py"
click node_paper_data "https://github.com/andrewcshi/ai-index-2025/tree/main/data"
click node_country_attribution "https://github.com/andrewcshi/ai-index-2025/blob/main/main/countries.py"
click node_city_data "https://github.com/andrewcshi/ai-index-2025/blob/main/datasets/cities.csv"
click node_institution_data "https://github.com/andrewcshi/ai-index-2025/blob/main/datasets/institutions.csv"
click node_revisions "https://github.com/andrewcshi/ai-index-2025/blob/main/data/revisions.py"
click node_statistics "https://github.com/andrewcshi/ai-index-2025/blob/main/data/stats.py"
click node_image_scripts "https://github.com/andrewcshi/ai-index-2025/tree/main/midjourney/harry-potter/scripts"
click node_fewshot "https://github.com/andrewcshi/ai-index-2025/blob/main/midjourney/harry-potter/scripts/few_shot_sim.py"
click node_oneshot "https://github.com/andrewcshi/ai-index-2025/blob/main/midjourney/harry-potter/scripts/one_shot_sim.py"
click node_generated_images "https://github.com/andrewcshi/ai-index-2025/tree/main/midjourney/harry-potter"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_aaai,node_aies,node_facct,node_iclr,node_icml,node_neurips,node_user toneBlue
class node_paper_data,node_country_attribution,node_city_data,node_institution_data,node_revisions,node_statistics toneAmber
class node_image_scripts,node_fewshot,node_oneshot,node_generated_images toneMint
class node_conference_sites toneIndigo
```

## Components

### Paper collection

- [AAAI scraper](https://github.com/andrewcshi/ai-index-2025/blob/main/main/aaai.py): `main/aaai.py`
- [AIES scraper](https://github.com/andrewcshi/ai-index-2025/blob/main/main/aies.py): `main/aies.py`
- [FAccT scraper](https://github.com/andrewcshi/ai-index-2025/blob/main/main/facct.py): `main/facct.py`
- [ICLR scraper](https://github.com/andrewcshi/ai-index-2025/blob/main/main/iclr.py): `main/iclr.py`
- [ICML scraper](https://github.com/andrewcshi/ai-index-2025/blob/main/main/icml.py): `main/icml.py`
- [NeurIPS scraper](https://github.com/andrewcshi/ai-index-2025/blob/main/main/neurips.py): `main/neurips.py`

### Dataset analysis

- [Paper CSV archive](https://github.com/andrewcshi/ai-index-2025/tree/main/data): `data`
- [Country attribution](https://github.com/andrewcshi/ai-index-2025/blob/main/main/countries.py): `main/countries.py`
- [City reference data](https://github.com/andrewcshi/ai-index-2025/blob/main/datasets/cities.csv): `datasets/cities.csv`
- [Institution reference data](https://github.com/andrewcshi/ai-index-2025/blob/main/datasets/institutions.csv): `datasets/institutions.csv`
- [Keyword revisions](https://github.com/andrewcshi/ai-index-2025/blob/main/data/revisions.py): `data/revisions.py`
- [Paper statistics](https://github.com/andrewcshi/ai-index-2025/blob/main/data/stats.py): `data/stats.py`

### Image comparison

- [Similarity scripts](https://github.com/andrewcshi/ai-index-2025/tree/main/midjourney/harry-potter/scripts): `midjourney/harry-potter/scripts`
- [Few-shot similarity](https://github.com/andrewcshi/ai-index-2025/blob/main/midjourney/harry-potter/scripts/few_shot_sim.py): `midjourney/harry-potter/scripts/few_shot_sim.py`
- [One-shot similarity](https://github.com/andrewcshi/ai-index-2025/blob/main/midjourney/harry-potter/scripts/one_shot_sim.py): `midjourney/harry-potter/scripts/one_shot_sim.py`
- [Midjourney image sets](https://github.com/andrewcshi/ai-index-2025/tree/main/midjourney/harry-potter): `midjourney/harry-potter`

### Other components

- **Repository user**
- **Conference websites**

## Connections

- Repository user → AAAI scraper: runs (evidence: `README.md`)
- Repository user → AIES scraper: runs (evidence: `README.md`)
- Repository user → FAccT scraper: runs (evidence: `README.md`)
- Repository user → ICLR scraper: runs (evidence: `README.md`)
- Repository user → ICML scraper: runs (evidence: `README.md`)
- Repository user → NeurIPS scraper: runs (evidence: `README.md`)
- AAAI scraper → Conference websites: scrapes (evidence: `README.md`)
- AIES scraper → Conference websites: scrapes (evidence: `README.md`)
- FAccT scraper → Conference websites: scrapes (evidence: `README.md`)
- ICLR scraper → Conference websites: scrapes (evidence: `README.md`)
- ICML scraper → Conference websites: scrapes (evidence: `README.md`)
- NeurIPS scraper → Conference websites: scrapes (evidence: `README.md`)
- AAAI scraper → Paper CSV archive: writes CSV (evidence: `main/aaai.py`)
- AIES scraper → Paper CSV archive: writes CSV (evidence: `main/aies.py`)
- FAccT scraper → Paper CSV archive: writes CSV (evidence: `main/facct.py`)
- ICLR scraper → Paper CSV archive: writes CSV (evidence: `main/iclr.py`)
- ICML scraper → Paper CSV archive: writes CSV
- NeurIPS scraper → Paper CSV archive: writes CSV (evidence: `main/neurips.py`)
- Country attribution → City reference data: reads (evidence: `main/countries.py`)
- Country attribution → Institution reference data: reads (evidence: `main/countries.py`)
- Keyword revisions → Paper CSV archive: reads and augments (evidence: `data/revisions.py`)
- Paper statistics → Paper CSV archive: reads (evidence: `data/stats.py`)
- Repository user → Few-shot similarity: runs
- Repository user → One-shot similarity: runs
- Few-shot similarity → Midjourney image sets: compares images (evidence: `midjourney/harry-potter/scripts/few_shot_sim.py`)
- One-shot similarity → Midjourney image sets: compares images