# cyberagentailab/profiling-agent-open-ecommerce architecture

> Architecture diagram of the GitHub repository cyberagentailab/profiling-agent-open-ecommerce, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cyberagentailab/profiling-agent-open-ecommerce
- Repository: https://github.com/cyberagentailab/profiling-agent-open-ecommerce

## Overview

This repository reproduces an LLM-based user-profiling pipeline over Open E-Commerce purchase histories, plus standalone mining and evaluation analyses. The documented main workflow prepares product titles, gates and enriches them with search, recovers canonical product names, infers user attributes, clusters and labels those attributes, then evaluates them against survey answers. Stages exchange files rather than importing one another. The README documents the stage order; sampled entry points and source references support selected internal connections. Several stage internals were not sampled, so their implementation-level links are omitted.

Main entry points: [`src/base_agent/config.py`](https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/base_agent/config.py) (Configuration), [`src/tag_cluster/model.py`](https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/tag_cluster/model.py) (Cluster tagging), [`src/search/model.py`](https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/search/model.py) (Web enrichment), [`src/base_agent/model.py`](https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/base_agent/model.py) (Model runtime)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_product_enrichment["Product enrichment"]
  node_prepare_titles["Prepare titles<br/>[dataset.py]"]
  node_scan["Search gate<br/>[model.py]"]
  node_search["Web enrichment<br/>[model.py]"]
  node_predict_transaction["Canonical recovery<br/>[model.py]"]
end

subgraph group_profiling["Attribute profiling"]
  node_predict_user["User attributes<br/>[model.py]"]
  node_cluster_attribute["Attribute clustering<br/>[model.py]"]
  node_tag_cluster["Cluster tagging<br/>[model.py]"]
end

subgraph group_analysis["Mining and evaluation"]
  node_frequent_patterns["Frequent patterns<br/>[model.py]"]
  node_judge_user["Attribute validity<br/>[model.py]"]
  node_judge_demographic["Demographic baseline<br/>[model.py]"]
  node_confidence_analysis["Confidence analysis<br/>[model.py]"]
end

subgraph group_shared_runtime["Shared runtime"]
  node_base_agent["Model runtime<br/>[model.py]"]
  node_config["Configuration<br/>[config.py]"]
  node_io["JSON I/O<br/>[io.py]"]
end

node_researcher(("Researcher"))
node_purchases["Purchase histories"]
node_stage_io["Stage files"]
node_web_search(("Web search services"))

node_researcher -->|"runs pipeline"| node_prepare_titles
node_purchases -->|"reads histories"| node_prepare_titles
node_prepare_titles -->|"passes files"| node_scan
node_scan -->|"passes files"| node_search
node_search -->|"searches web"| node_web_search
node_search -->|"passes results"| node_predict_transaction
node_predict_transaction -->|"passes products"| node_predict_user
node_frequent_patterns -->|"supplies combinations"| node_predict_user
node_predict_user -->|"passes attributes"| node_cluster_attribute
node_cluster_attribute -->|"passes clusters"| node_tag_cluster
node_tag_cluster -->|"provides analysis input"| node_judge_user
node_tag_cluster -->|"provides analysis input"| node_judge_demographic
node_tag_cluster -->|"provides analysis input"| node_confidence_analysis
node_prepare_titles -.->|"writes stage files"| node_stage_io
node_scan -.->|"writes stage files"| node_stage_io
node_search -.->|"writes stage files"| node_stage_io
node_predict_transaction -.->|"writes stage files"| node_stage_io
node_predict_user -.->|"writes stage files"| node_stage_io
node_cluster_attribute -.->|"writes stage files"| node_stage_io
node_tag_cluster -.->|"writes stage files"| node_stage_io
node_frequent_patterns -.->|"writes patterns"| node_stage_io
node_stage_io -.->|"reads inputs"| node_judge_user
node_stage_io -.->|"reads inputs"| node_judge_demographic
node_stage_io -.->|"reads inputs"| node_confidence_analysis
node_scan -->|"inherits runtime"| node_base_agent
node_search -->|"uses JSON I/O"| node_io
node_predict_transaction -->|"inherits runtime"| node_base_agent
node_predict_user -->|"inherits runtime"| node_base_agent
node_cluster_attribute -->|"inherits runtime"| node_base_agent
node_tag_cluster -->|"inherits runtime"| node_base_agent
node_judge_user -->|"uses runtime"| node_base_agent
node_judge_demographic -->|"uses runtime"| node_base_agent
node_prepare_titles -->|"loads settings"| node_config
node_scan -->|"loads settings"| node_config
node_search -->|"loads settings"| node_config
node_predict_transaction -->|"loads settings"| node_config
node_predict_user -->|"loads settings"| node_config
node_cluster_attribute -->|"loads settings"| node_config
node_tag_cluster -->|"loads settings"| node_config
node_frequent_patterns -->|"loads settings"| node_config
node_judge_user -->|"loads settings"| node_config
node_judge_demographic -->|"loads settings"| node_config
node_confidence_analysis -->|"loads settings"| node_config

click node_prepare_titles "https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/prepare_titles/dataset.py"
click node_scan "https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/scan/model.py"
click node_search "https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/search/model.py"
click node_predict_transaction "https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/predict_transaction/model.py"
click node_frequent_patterns "https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/frequent_pattern_mining/model.py"
click node_predict_user "https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/predict_user/model.py"
click node_cluster_attribute "https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/cluster_attribute/model.py"
click node_tag_cluster "https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/tag_cluster/model.py"
click node_judge_user "https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/judge_user_attribute/model.py"
click node_judge_demographic "https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/judge_demographic_attribute/model.py"
click node_confidence_analysis "https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/investigate_confidence_feasibility/model.py"
click node_base_agent "https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/base_agent/model.py"
click node_config "https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/base_agent/config.py"
click node_io "https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/common/io.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_prepare_titles,node_scan,node_search,node_predict_transaction toneBlue
class node_predict_user,node_cluster_attribute,node_tag_cluster toneAmber
class node_frequent_patterns,node_judge_user,node_judge_demographic,node_confidence_analysis toneMint
class node_base_agent,node_config,node_io toneRose
class node_researcher,node_purchases,node_stage_io,node_web_search toneIndigo
```

## Components

### Product enrichment

- [Prepare titles](https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/prepare_titles/dataset.py): `src/prepare_titles/dataset.py`
- [Search gate](https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/scan/model.py): `src/scan/model.py`
- [Web enrichment](https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/search/model.py): `src/search/model.py`
- [Canonical recovery](https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/predict_transaction/model.py): `src/predict_transaction/model.py`

### Attribute profiling

- [User attributes](https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/predict_user/model.py): `src/predict_user/model.py`
- [Attribute clustering](https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/cluster_attribute/model.py): `src/cluster_attribute/model.py`
- [Cluster tagging](https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/tag_cluster/model.py): `src/tag_cluster/model.py`

### Mining and evaluation

- [Frequent patterns](https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/frequent_pattern_mining/model.py): `src/frequent_pattern_mining/model.py`
- [Attribute validity](https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/judge_user_attribute/model.py): `src/judge_user_attribute/model.py`
- [Demographic baseline](https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/judge_demographic_attribute/model.py): `src/judge_demographic_attribute/model.py`
- [Confidence analysis](https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/investigate_confidence_feasibility/model.py): `src/investigate_confidence_feasibility/model.py`

### Shared runtime

- [Model runtime](https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/base_agent/model.py): `src/base_agent/model.py`
- [Configuration](https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/base_agent/config.py): `src/base_agent/config.py`
- [JSON I/O](https://github.com/cyberagentailab/profiling-agent-open-ecommerce/blob/main/src/common/io.py): `src/common/io.py`

### Other components

- **Researcher**
- **Purchase histories**
- **Stage files**
- **Web search services**

## Connections

- Researcher → Prepare titles: runs pipeline
- Purchase histories → Prepare titles: reads histories
- Prepare titles → Search gate: passes files
- Search gate → Web enrichment: passes files
- Web enrichment → Web search services: searches web
- Web enrichment → Canonical recovery: passes results
- Canonical recovery → User attributes: passes products
- Frequent patterns → User attributes: supplies combinations
- User attributes → Attribute clustering: passes attributes
- Attribute clustering → Cluster tagging: passes clusters
- Cluster tagging → Attribute validity: provides analysis input
- Cluster tagging → Demographic baseline: provides analysis input
- Cluster tagging → Confidence analysis: provides analysis input
- Prepare titles → Stage files: writes stage files
- Search gate → Stage files: writes stage files
- Web enrichment → Stage files: writes stage files
- Canonical recovery → Stage files: writes stage files
- User attributes → Stage files: writes stage files
- Attribute clustering → Stage files: writes stage files
- Cluster tagging → Stage files: writes stage files
- Frequent patterns → Stage files: writes patterns
- Stage files → Attribute validity: reads inputs
- Stage files → Demographic baseline: reads inputs
- Stage files → Confidence analysis: reads inputs
- Search gate → Model runtime: inherits runtime (evidence: `src/scan/model.py`)
- Web enrichment → JSON I/O: uses JSON I/O (evidence: `src/search/model.py`)
- Canonical recovery → Model runtime: inherits runtime (evidence: `src/predict_transaction/model.py`)
- User attributes → Model runtime: inherits runtime (evidence: `src/predict_user/model.py`)
- Attribute clustering → Model runtime: inherits runtime (evidence: `src/cluster_attribute/model.py`)
- Cluster tagging → Model runtime: inherits runtime (evidence: `src/tag_cluster/model.py`)
- Attribute validity → Model runtime: uses runtime (evidence: `src/judge_user_attribute/model.py`)
- Demographic baseline → Model runtime: uses runtime (evidence: `src/judge_demographic_attribute/model.py`)
- Prepare titles → Configuration: loads settings (evidence: `src/prepare_titles/__main__.py`)
- Search gate → Configuration: loads settings (evidence: `src/scan/__main__.py`)
- Web enrichment → Configuration: loads settings (evidence: `src/search/__main__.py`)
- Canonical recovery → Configuration: loads settings (evidence: `src/predict_transaction/__main__.py`)
- User attributes → Configuration: loads settings (evidence: `src/predict_user/__main__.py`)
- Attribute clustering → Configuration: loads settings (evidence: `src/cluster_attribute/__main__.py`)
- Cluster tagging → Configuration: loads settings (evidence: `src/tag_cluster/__main__.py`)
- Frequent patterns → Configuration: loads settings (evidence: `src/frequent_pattern_mining/__main__.py`)
- Attribute validity → Configuration: loads settings (evidence: `src/judge_user_attribute/__main__.py`)
- Demographic baseline → Configuration: loads settings (evidence: `src/judge_demographic_attribute/__main__.py`)
- Confidence analysis → Configuration: loads settings (evidence: `src/investigate_confidence_feasibility/__main__.py`)