# rodrigomaulen/attention_based_pca architecture

> Architecture diagram of the GitHub repository rodrigomaulen/attention_based_pca, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rodrigomaulen/attention_based_pca
- Repository: https://github.com/rodrigomaulen/attention_based_pca

## Overview

This repository is a collection of standalone JAX experiments for studying attention-based PCA. The user-facing workflow is to run an experiment script and obtain a paper figure. The scripts cover linear risk, softmax attention, in-context learning (ICL), and comparisons between finite-prompt SGD and infinite-prompt gradient descent; several vary dimension or sequence length. The excerpts show experiment and plotting logic, but not every script’s output details, so the figure relationship is documented at the collection level rather than as a detailed runtime pipeline.

Main entry points: [`linear_finite.py`](https://github.com/rodrigomaulen/attention_based_pca/blob/main/linear_finite.py) (Finite Linear), [`linear_finite_change_L.py`](https://github.com/rodrigomaulen/attention_based_pca/blob/main/linear_finite_change_L.py) (Linear vs Length), [`linear_analytic.py`](https://github.com/rodrigomaulen/attention_based_pca/blob/main/linear_analytic.py) (Analytic Linear), [`linear_analytic_change_dim.py`](https://github.com/rodrigomaulen/attention_based_pca/blob/main/linear_analytic_change_dim.py) (Linear vs Dimension)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_linear["Linear Risk"]
  node_linear_finite["Finite Linear<br/>[linear_finite.py]"]
  node_linear_finite_l["Linear vs Length"]
  node_linear_analytic["Analytic Linear<br/>[linear_analytic.py]"]
  node_linear_analytic_dim["Linear vs Dimension"]
end

subgraph group_softmax["Softmax Attention"]
  node_softmax_finite["Finite Softmax<br/>[softmax_finite.py]"]
  node_softmax_finite_l["Softmax vs Length"]
  node_softmax_infinite["Infinite Softmax"]
  node_softmax_infinite_dim["Softmax vs Dimension"]
end

subgraph group_icl["In-Context Learning"]
  node_icl_finite["Finite ICL<br/>[icl_finite.py]"]
  node_icl_finite_l["ICL vs Length"]
  node_icl_infinite["Infinite ICL<br/>[icl_infinite.py]"]
  node_icl_infinite_dim["ICL vs Dimension"]
end

subgraph group_comparisons["Prompt Comparisons"]
  node_compare_softmax["Softmax Comparison"]
  node_compare_icl["ICL Comparison"]
end

node_figures["Paper Figures"]

node_linear_finite -->|"creates figure"| node_figures
node_linear_finite_l -->|"creates figure"| node_figures
node_linear_analytic -->|"creates figure"| node_figures
node_linear_analytic_dim -->|"creates figure"| node_figures
node_softmax_finite -->|"creates figure"| node_figures
node_softmax_finite_l -->|"creates figure"| node_figures
node_softmax_infinite -->|"creates figure"| node_figures
node_softmax_infinite_dim -->|"creates figure"| node_figures
node_icl_finite -->|"creates figure"| node_figures
node_icl_finite_l -->|"creates figure"| node_figures
node_icl_infinite -->|"creates figure"| node_figures
node_icl_infinite_dim -->|"creates figure"| node_figures
node_compare_softmax -->|"creates figure"| node_figures
node_compare_icl -->|"creates figure"| node_figures

click node_linear_finite "https://github.com/rodrigomaulen/attention_based_pca/blob/main/linear_finite.py"
click node_linear_finite_l "https://github.com/rodrigomaulen/attention_based_pca/blob/main/linear_finite_change_L.py"
click node_linear_analytic "https://github.com/rodrigomaulen/attention_based_pca/blob/main/linear_analytic.py"
click node_linear_analytic_dim "https://github.com/rodrigomaulen/attention_based_pca/blob/main/linear_analytic_change_dim.py"
click node_softmax_finite "https://github.com/rodrigomaulen/attention_based_pca/blob/main/softmax_finite.py"
click node_softmax_finite_l "https://github.com/rodrigomaulen/attention_based_pca/blob/main/softmax_finite_alignment_vs_L.py"
click node_softmax_infinite "https://github.com/rodrigomaulen/attention_based_pca/blob/main/softmax_infinite.py"
click node_softmax_infinite_dim "https://github.com/rodrigomaulen/attention_based_pca/blob/main/softmax_infinite_change_dim.py"
click node_icl_finite "https://github.com/rodrigomaulen/attention_based_pca/blob/main/icl_finite.py"
click node_icl_finite_l "https://github.com/rodrigomaulen/attention_based_pca/blob/main/icl_finite_alignment_vs_L.py"
click node_icl_infinite "https://github.com/rodrigomaulen/attention_based_pca/blob/main/icl_infinite.py"
click node_icl_infinite_dim "https://github.com/rodrigomaulen/attention_based_pca/blob/main/icl_infinite_change_dim.py"
click node_compare_softmax "https://github.com/rodrigomaulen/attention_based_pca/blob/main/combined_softmax_infinite_finite.py"
click node_compare_icl "https://github.com/rodrigomaulen/attention_based_pca/blob/main/combined_icl_infinite_finite.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_linear_finite,node_linear_finite_l,node_linear_analytic,node_linear_analytic_dim toneBlue
class node_softmax_finite,node_softmax_finite_l,node_softmax_infinite,node_softmax_infinite_dim toneAmber
class node_icl_finite,node_icl_finite_l,node_icl_infinite,node_icl_infinite_dim toneMint
class node_compare_softmax,node_compare_icl toneRose
class node_figures toneIndigo
```

## Components

### Linear Risk

- [Finite Linear](https://github.com/rodrigomaulen/attention_based_pca/blob/main/linear_finite.py): `linear_finite.py`
- [Linear vs Length](https://github.com/rodrigomaulen/attention_based_pca/blob/main/linear_finite_change_L.py): `linear_finite_change_L.py`
- [Analytic Linear](https://github.com/rodrigomaulen/attention_based_pca/blob/main/linear_analytic.py): `linear_analytic.py`
- [Linear vs Dimension](https://github.com/rodrigomaulen/attention_based_pca/blob/main/linear_analytic_change_dim.py): `linear_analytic_change_dim.py`

### Softmax Attention

- [Finite Softmax](https://github.com/rodrigomaulen/attention_based_pca/blob/main/softmax_finite.py): `softmax_finite.py`
- [Softmax vs Length](https://github.com/rodrigomaulen/attention_based_pca/blob/main/softmax_finite_alignment_vs_L.py): `softmax_finite_alignment_vs_L.py`
- [Infinite Softmax](https://github.com/rodrigomaulen/attention_based_pca/blob/main/softmax_infinite.py): `softmax_infinite.py`
- [Softmax vs Dimension](https://github.com/rodrigomaulen/attention_based_pca/blob/main/softmax_infinite_change_dim.py): `softmax_infinite_change_dim.py`

### In-Context Learning

- [Finite ICL](https://github.com/rodrigomaulen/attention_based_pca/blob/main/icl_finite.py): `icl_finite.py`
- [ICL vs Length](https://github.com/rodrigomaulen/attention_based_pca/blob/main/icl_finite_alignment_vs_L.py): `icl_finite_alignment_vs_L.py`
- [Infinite ICL](https://github.com/rodrigomaulen/attention_based_pca/blob/main/icl_infinite.py): `icl_infinite.py`
- [ICL vs Dimension](https://github.com/rodrigomaulen/attention_based_pca/blob/main/icl_infinite_change_dim.py): `icl_infinite_change_dim.py`

### Prompt Comparisons

- [Softmax Comparison](https://github.com/rodrigomaulen/attention_based_pca/blob/main/combined_softmax_infinite_finite.py): `combined_softmax_infinite_finite.py`
- [ICL Comparison](https://github.com/rodrigomaulen/attention_based_pca/blob/main/combined_icl_infinite_finite.py): `combined_icl_infinite_finite.py`

### Other components

- **Paper Figures**

## Connections

- Finite Linear → Paper Figures: creates figure (evidence: `README.md`)
- Linear vs Length → Paper Figures: creates figure (evidence: `README.md`)
- Analytic Linear → Paper Figures: creates figure (evidence: `README.md`)
- Linear vs Dimension → Paper Figures: creates figure (evidence: `README.md`)
- Finite Softmax → Paper Figures: creates figure (evidence: `README.md`)
- Softmax vs Length → Paper Figures: creates figure (evidence: `README.md`)
- Infinite Softmax → Paper Figures: creates figure (evidence: `README.md`)
- Softmax vs Dimension → Paper Figures: creates figure (evidence: `README.md`)
- Finite ICL → Paper Figures: creates figure (evidence: `README.md`)
- ICL vs Length → Paper Figures: creates figure (evidence: `README.md`)
- Infinite ICL → Paper Figures: creates figure (evidence: `README.md`)
- ICL vs Dimension → Paper Figures: creates figure (evidence: `README.md`)
- Softmax Comparison → Paper Figures: creates figure (evidence: `README.md`)
- ICL Comparison → Paper Figures: creates figure (evidence: `README.md`)
