# optas/artemis architecture

> Architecture diagram of the GitHub repository optas/artemis, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/optas/artemis
- Repository: https://github.com/optas/artemis

## Overview

##### Project Type & Purpose
- ArtEmis is a research codebase focused on affective language for visual art. It is designed to enable data preprocessing, analysis, emotion-classifier training, and neural speaker generation for painting captioning and emotion analysis.
- The project encompasses data processing (preprocessing and cleaning), exploratory analysis (via notebooks), and training pipelines for deep learning models that associate emotions to images and text.

##### Analyzing the File Structure
- Top-level directories and files:
- The main package is “artemis” which contains submodules including “analysis”, “captioning”, “evaluation”, “in_out”, “language”, “neural_models”, “utils”, and “data”. These denote clear modular separation.
- “notebooks” are provided for exploratory analysis, training experiments, and evaluation.
- The “scripts” directory encapsulates command-line interfaces for preprocessing, training, sampling, and evaluation.
- Each submodule focuses on a distinct aspect:
- “analysis”: Contains statistical and linguistic analysis scripts (e.g., emotion-centric and feature extraction functions).
- “captioning” and “neural_models”: Focus on building captioning models and neural network components for attention, encoder/decoder, and classification.
- “evaluation”: Includes metrics implementations (BLEU, METEOR, CIDEr, etc.) and evaluation scripts to critique the generated captions.
- “in_out”: Manages data input/output operations such as reading, cleaning, and manipulating datasets.
- “language”: Provides functions for linguistic processing (e.g., part-of-speech tagging, adjective-noun pair extraction, spell checking).
- “utils”: Houses common utilities (basic functions, vocabulary management, visualization).

##### Key Architectural Components
a. Data & Preprocessing Layer:
- Input Data: Raw ArtEmis dataset (CSV, images, pre-trained vocabulary files, etc.) located under “artemis/data”.
- Preprocessing Scripts: Found in “artemis/scripts/preprocess_artemis_data.py” and “artemis/in_out”. These handle cleaning, tokenization, spell-checking, data splits, and creating input files for modeling.

b. Analysis & Exploration Layer:
- Analysis modules in “artemis/analysis” and Jupyter notebooks under “artemis/notebooks/analysis” for linguistic and statistical analysis (e.g., emotion histogram extraction and entropy analysis).

c. Modeling & Training Layer:
- Neural Models: Located in “artemis/neural_models” (attentive decoders, image and text classifiers, etc.) and “artemis/language” (LSTM encoders, transformer-based modules).
- Training Scripts/Notebooks:
- Emotion-centric training notebooks (image-to-emotion, utterance-to-emotion) and scripts (e.g., “train_speaker.py” in “artemis/scripts”).
- Pretrained models are provided externally, with instructions in the README on how to integrate them.

d. Caption Generation (Speaker) & Evaluation:
- Captioning Module: Under “artemis/captioning” includes sample captions and sentiment analysis with sentiment-aware algorithms.
- Evaluation: “artemis/evaluation” includes various metrics (BLEU, METEOR, CIDEr, etc.) and notebooks for evaluating generated captions with aspects like emotional alignment.

e. External Dependencies & Configuration:
- The project relies on third-party libraries (PyTorch, CUDA, etc.) as specified in setup.py.
- The README describes environment setup and execution commands, which are integrated in the pipeline.

##### Relationships and Data Flow
- The Analysis layer (using notebooks) for exploratory analysis.
- The Modeling layer where training routines (via scripts and notebooks) consume preprocessed data to train emotion classifiers and neural speakers.
- After model training, predictions/captions can be generated (using sampling scripts) and then evaluated using the “Evaluation” module.
- External pretrained models (image-to-emotion, text emotion classifiers, SAT-speaker) act as inputs or dependencies for some of these pipelines.

##### Architectural Patterns & Principles
- Modularity: The codebase is clearly divided into self-contained modules addressing specific functionalities (preprocessing, analysis, modeling, evaluation).
- Separation of Concerns: Different directories handle distinct operational aspects such as neural models, language operations, and in/out operations.
- Reusability: Functions from “utils” and “language” can be reused across different parts of the system.

- Critical modules are represented as separate components with clear labels.
- Loop-backs or feedback cycles (like retraining based on evaluation outcomes).

## Diagram (Mermaid)

```mermaid
graph TD
    subgraph "Data Layer"
        A["Data Storage"]
    end

    subgraph "Preprocessing Layer"
        B1["Preprocessing Script"]
        B2["Data I/O"]
    end

    subgraph "Analysis & Exploration"
        C1["Analysis Module"]
        C2["Analysis Notebooks"]
    end

    subgraph "Modeling & Training"
        D1["Neural Models"]
        D2["Language Module"]
        E1["Training Script"]
        E2["Deep Nets (Emotions) Notebooks"]
    end

    subgraph "Caption Generation & Evaluation"
        F1["Captioning Module"]
        F2["Sample Speaker Script"]
        G1["Evaluation Module"]
        G2["Deep Nets (Speakers) Notebooks"]
    end

    %% Data Flow
    A --> B1
    A --> B2
    B1 --> B2
    B2 --> C1
    B2 --> D1
    B2 --> D2
    C1 --> C2
    C2 --> E1
    D1 --> E1
    D2 --> E1
    E1 --> E2
    E1 --> F1
    E2 --> F2
    F1 --> G1
    F2 --> G2
    G1 --> E1

    %% Click Events
    click A "https://github.com/optas/artemis/tree/master/artemis/data"
    click B1 "https://github.com/optas/artemis/blob/master/artemis/scripts/preprocess_artemis_data.py"
    click B2 "https://github.com/optas/artemis/tree/master/artemis/in_out"
    click C1 "https://github.com/optas/artemis/tree/master/artemis/analysis"
    click C2 "https://github.com/optas/artemis/tree/master/artemis/notebooks/analysis"
    click D1 "https://github.com/optas/artemis/tree/master/artemis/neural_models"
    click D2 "https://github.com/optas/artemis/tree/master/artemis/language"
    click E1 "https://github.com/optas/artemis/blob/master/artemis/scripts/train_speaker.py"
    click E2 "https://github.com/optas/artemis/tree/master/artemis/notebooks/deep_nets/emotions"
    click F1 "https://github.com/optas/artemis/tree/master/artemis/captioning"
    click F2 "https://github.com/optas/artemis/blob/master/artemis/scripts/sample_speaker.py"
    click G1 "https://github.com/optas/artemis/tree/master/artemis/evaluation"
    click G2 "https://github.com/optas/artemis/tree/master/artemis/notebooks/deep_nets/speakers"

    %% Class Assignments
    class A data;
    class B1,B2 preprocessing;
    class C1,C2 analysis;
    class D1,D2 modeling;
    class E1,E2 training;
    class F1,F2 caption;
    class G1,G2 evaluation;

    %% Styles
    classDef data fill:#f9f,stroke:#333,stroke-width:2px;
    classDef preprocessing fill:#bbf,stroke:#333,stroke-width:2px;
    classDef analysis fill:#cff,stroke:#333,stroke-width:2px;
    classDef modeling fill:#fcf,stroke:#333,stroke-width:2px;
    classDef training fill:#cfc,stroke:#333,stroke-width:2px;
    classDef caption fill:#ff9,stroke:#333,stroke-width:2px;
    classDef evaluation fill:#f99,stroke:#333,stroke-width:2px;
```
