# pedrobarbosa/dress architecture

> Architecture diagram of the GitHub repository pedrobarbosa/dress, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pedrobarbosa/dress
- Repository: https://github.com/pedrobarbosa/dress

## Overview

##### Project Type and Purpose
- Type: A Python-based, CLI-driven bioinformatics toolkit (open-source research software).
- Purpose: Generate and filter synthetic RNA exon-triplet datasets via genetic programming constrained by domain-specific grammars, guided by a deep-learning “oracle,” then provide evaluation utilities for motif analysis.

##### High-Level Components
- CLI Frontend (“dress” entry point)
– Subcommands: generate, filter
– Parses user arguments + YAML config files in dress/configs/*.yaml
- DatasetGeneration Module (dress/datasetgeneration)
– Preprocessing: gtf_cache + genome utilities
– Grammar loader (dress/datasetgeneration/grammars)
– Genetic programming controller (evolution.py, custom_callbacks, etc.)
– Underlying engine: GeneticEngine Python library
– Output: synthetic sequence dataset (FASTA/TSV)
- DatasetFiltering Module (dress/datasetfiltering)
– Input: synthetic or raw sequences
– Filters by splice-site probability, PSI, dPSI
– Validation, argument checking
– Output: filtered subset
- DatasetEvaluation Module (dress/datasetevaluation)
– Representation layer: APIs for sequences and motifs
– Motif DB loaders (ATtRACT, cisBP_RNA, oRNAment, RBNS_Encode)
– Enrichment and search utilities
– Output: evaluation reports, motif hit tables
- Data Stores & External Resources
– Genome reference (hg38 FASTA) + exon cache (GENCODE v44)
– Local motif databases under dress/datasetevaluation/representation/motifs/db
– Example inputs (data/examples)
- CI/Tests
– pytest suite under tests/
– GitHub Actions workflow (.github/workflows/run_tests.yml)

##### Relationships & Data Flow
→ Reads raw_input (data/examples or user-provided)
→ Loads YAML config
→ Calls DatasetGeneration
- Preprocesses via gtf_cache → Fetches exon sequences from genome + cache
- Loads grammar → Applies genetic operators → GeneticEngine evolves sequences
- At each generation, queries external oracle (SpliceAI/Pangolin) → returns scores
- Applies custom fitness, stopping, callbacks → produces synthetic dataset
→ Writes raw synthetic output (FASTA/TSV) to disk
b. User invokes “dress filter”
→ Reads synthetic or raw dataset
→ Applies probability/PSI thresholds → outputs filtered dataset
c. (Optional) User runs evaluation scripts in dress/datasetevaluation
→ Loads sequences + motif DBs
→ Runs enrichment/search → outputs visualizations or CSV summaries

##### Architectural Patterns & Design Principles
- Modular CLI → dispatch pattern (single entry point, subcommands)
- Clear separation of concerns: generation vs. filtering vs. evaluation
- Plugin-style grammars: grammars in their own package folder, easy to extend
- GeneticAlgorithm orchestration via GeneticEngine library
- YAML-based configuration → reproducibility
- Data-centric pipelines: each stage reads/writes files
- Test-driven development: extensive pytest suite

##### Elements
- Config files (dress/configs/*.yaml) feeding into both generate & filter
- GitHub Actions & pytest test suite as a side-component for CI portrayal

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% CLI Layer
    CLI["CLI: dress"]:::cli
    Config["YAML Configs"]:::cli
    CLI -->|"invokes subcommands"| Config

    %% Generate Command
    GenRun["dress generate"]:::cli
    CLI --> GenRun
    GenRun -->|reads| Config

    subgraph "DatasetGeneration Module"
        direction TB
        GTF["GTF Cache"]:::core
        UtilsPre["Preprocessing Utils"]:::core
        GrammarLoader["Grammar Loader"]:::core
        PWMGrammar["pwm_perturbation_grammar"]:::core
        RandomGrammar["random_perturbation_grammar"]:::core
        GrammarUtils["Grammar Utils"]:::core
        Evolution["Evolution Controller"]:::core
        ConfigEvo["Evolution Config"]:::core
        Callbacks["Custom Callbacks"]:::core
        Fitness["Custom Fitness"]:::core
        PopEval["Population Evaluators"]:::core
        Steps["Custom Steps"]:::core
        Stopping["Custom Stopping"]:::core
        BBModel["Deep-learning Oracle Adapter"]:::core
        Singleton["Singleton Model"]:::core
        JSONSchema["JSON Schema"]:::core
        Logger["Logger"]:::core
        ValidateGen["Validate Args"]:::core

        GenRun --> GTF
        GenRun --> UtilsPre
        GenRun --> GrammarLoader
        GrammarLoader --> PWMGrammar
        GrammarLoader --> RandomGrammar
        GrammarLoader --> GrammarUtils
        GenRun --> Evolution
        Evolution --> ConfigEvo
        Evolution --> Callbacks
        Evolution --> Fitness
        Evolution --> PopEval
        Evolution --> Steps
        Evolution --> Stopping
        Evolution <-->|queries| BBModel
        BBModel --> Singleton
        GenRun --> JSONSchema
        GenRun --> Logger
        GenRun --> ValidateGen
        ValidateGen --> Config
    end

    %% Filter Command
    FilterRun["dress filter"]:::cli
    CLI --> FilterRun
    FilterRun -->|reads| Config

    subgraph "DatasetFiltering Module"
        direction TB
        Filtering["Filtering Logic"]:::core
        ValidateFilter["Validate Args"]:::core

        FilterRun --> Filtering
        FilterRun --> ValidateFilter
        ValidateFilter --> Config
    end

    %% Evaluation Module
    EvalRun["Evaluation Scripts"]:::cli
    CLI --> EvalRun

    subgraph "DatasetEvaluation Module"
        direction TB
        API["API & Utilities"]:::core
        MotifRepr["Motif Representation"]:::core
        Search["Motif Search"]:::core
        SingleSeq["Single Sequence Motifs"]:::core
        MotifUtils["Motif Utils"]:::core
        Enrichment["Enrichment"]:::core
        MotifDBs["Motif DBs"]:::external

        EvalRun --> API
        API --> MotifRepr
        MotifRepr --> Search
        MotifRepr --> SingleSeq
        MotifRepr --> MotifUtils
        API --> Enrichment
        API --> MotifDBs
    end

    %% Data Stores & Examples
    subgraph "Data Stores & Examples"
        direction TB
        Examples["Example Inputs"]:::external
        TestData["Test Data Fixtures"]:::external
        GenomeRef["Genome FASTA"]:::external
        ExonCache["Exon Cache"]:::external
        MotifDBFiles["Raw Motif DB Files"]:::external

        Examples --> GenRun
        TestData --> GTF
        GenomeRef --> GTF
        ExonCache --> GTF
        MotifDBFiles --> MotifDBs
    end

    %% CI and Tests
    subgraph "CI & Tests"
        direction TB
        GHAction["GitHub Actions"]:::ci
        Pytests["pytest Suite"]:::ci

        GHAction --> Pytests
        Pytests --> API
        Pytests --> Filtering
        Pytests --> Evolution
    end

    %% Click Events
    click CLI "https://github.com/pedrobarbosa/dress/blob/main/dress/cli.py"
    click Config "https://github.com/pedrobarbosa/dress/blob/main/dress/configs/*.yaml"
    click GenRun "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetgeneration/run.py"
    click GTF "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetgeneration/preprocessing/gtf_cache.py"
    click UtilsPre "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetgeneration/preprocessing/utils.py"
    click GrammarLoader "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetgeneration/grammars/__init__.py"
    click PWMGrammar "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetgeneration/grammars/pwm_perturbation_grammar.py"
    click RandomGrammar "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetgeneration/grammars/random_perturbation_grammar.py"
    click GrammarUtils "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetgeneration/grammars/utils.py"
    click Evolution "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetgeneration/evolution.py"
    click ConfigEvo "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetgeneration/config_evolution.py"
    click Callbacks "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetgeneration/custom_callbacks.py"
    click Fitness "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetgeneration/custom_fitness.py"
    click PopEval "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetgeneration/custom_population_evaluators.py"
    click Steps "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetgeneration/custom_steps.py"
    click Stopping "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetgeneration/custom_stopping.py"
    click BBModel "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetgeneration/black_box/model.py"
    click Singleton "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetgeneration/black_box/singleton_model.py"
    click JSONSchema "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetgeneration/json_schema.py"
    click Logger "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetgeneration/logger.py"
    click ValidateGen "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetgeneration/validate_args.py"
    click FilterRun "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetfiltering/run.py"
    click Filtering "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetfiltering/filtering.py"
    click ValidateFilter "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetfiltering/validate_args.py"
    click API "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetevaluation/representation/api.py"
    click MotifRepr "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetevaluation/representation/motifs/repr.py"
    click Search "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetevaluation/representation/motifs/search.py"
    click SingleSeq "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetevaluation/representation/motifs/single_sequence_motifs.py"
    click MotifUtils "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetevaluation/representation/motifs/utils.py"
    click Enrichment "https://github.com/pedrobarbosa/dress/blob/main/dress/datasetevaluation/representation/motifs/enrichment.py"
    click GHAction "https://github.com/pedrobarbosa/dress/blob/main/.github/workflows/run_tests.yml"
    click Pytests "https://github.com/pedrobarbosa/dress/tree/main/tests/"
    click Examples "https://github.com/pedrobarbosa/dress/tree/main/data/examples/"
    click TestData "https://github.com/pedrobarbosa/dress/tree/main/tests/data/"
    click GenomeRef "https://github.com/pedrobarbosa/dress/tree/main/tests/data/"
    click ExonCache "https://github.com/pedrobarbosa/dress/tree/main/tests/data/"
    click MotifDBFiles "https://github.com/pedrobarbosa/dress/tree/main/dress/datasetevaluation/representation/motifs/db/ATtRACT/"
    click MotifDBFiles "https://github.com/pedrobarbosa/dress/tree/main/dress/datasetevaluation/representation/motifs/db/cisBP_RNA/"
    click MotifDBFiles "https://github.com/pedrobarbosa/dress/tree/main/dress/datasetevaluation/representation/motifs/db/RBNS_Encode/"
    click MotifDBFiles "https://github.com/pedrobarbosa/dress/tree/main/dress/datasetevaluation/representation/motifs/db/oRNAment/"

    %% Styles
    classDef core fill:#D0E8FF,stroke:#0366d6,stroke-width:1px
    classDef external fill:#DFFFE0,stroke:#228B22,stroke-width:1px
    classDef cli fill:#FFE4B5,stroke:#DAA520,stroke-width:1px
    classDef ci fill:#F0F0F0,stroke:#888888,stroke-width:1px
```
