# yogionbioinformatics/aals-epigenomics-signal-reproducibility architecture

> Architecture diagram of the GitHub repository yogionbioinformatics/aals-epigenomics-signal-reproducibility, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yogionbioinformatics/aals-epigenomics-signal-reproducibility
- Repository: https://github.com/yogionbioinformatics/aals-epigenomics-signal-reproducibility

## Overview

##### Project type and purpose
- This is a data‐analysis/pipeline project focused on assessing epigenomic signal reproducibility for Answer ALS using ATAC‐Seq data.
- It is not a traditional client‐server application but a scientific workflow composed of shell scripts, a Cromwell pipeline, QC outputs, and a series of Jupyter/R notebooks culminating in plots and a presentation.

##### File structure
- Top‐level directories:
- data_engineering/ – shell scripts to prepare raw FASTQ files
- cromwell/ – inputs & workflow identifiers for the Cromwell engine running the ATAC‐Seq pipeline
- qc/ – QC metrics generated by the ATAC‐Seq pipeline
- notebooks/ – subfolders for each downstream analysis stage (DiffBind, clustermaps, PCA)
- presentation/ – PDF of the final results
- Key files:
- mv_rename_fastqs.sh – renames raw data for the pipeline
- workflows.txt & input.json – specify the ATAC‐Seq workflow in Cromwell
- QC summary files – outputs from pipeline runs
- R/Python notebooks – generate count matrices, hierarchal clustering, PCA, and plots

##### README for additional insights
- Chronological order of analysis steps (data prep → pipeline → QC → sheet prep → count matrix → clustering → PCA → presentation)
- Technologies used: Bash, Cromwell (WDL), R (DiffBind, clustermap.R), Python (Seaborn), Jupyter notebooks

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Preparation Stage
    subgraph "Data Preparation" 
        direction TB
        RawFASTQ["Raw FASTQ files"]:::dataPrep
        DataEng["Data Engineering: mv_rename_fastqs.sh"]:::dataPrep
        RenamedFASTQ["Renamed FASTQ files"]:::dataPrep
    end

    %% Pipeline Execution Stage
    subgraph "Pipeline Execution"
        direction TB
        CromwellInputs1["Input JSON"]:::pipeline
        CromwellInputs2["Workflows TXT"]:::pipeline
        CromwellEngine["Cromwell Engine"]:::pipeline
        ATACTasks["ATAC-Seq Pipeline Tasks"]:::pipeline
    end

    %% QC Metrics Stage
    subgraph "QC Metrics Generation"
        direction TB
        QCFailed["failed.txt"]:::qc
        QCSummary["qc_summary_stats.csv"]:::qc
    end

    %% DiffBind Analysis Stage
    subgraph "DiffBind Analysis" 
        direction TB
        SamplePrepNB["Prep DiffBind Sheets Notebook"]:::analysis
        SampleSheet["sample_sheet.csv"]:::analysis
        CountMatrixScript["get_matrix.R"]:::analysis
        MatrixTSV["matrix.tsv"]:::analysis
    end

    %% Clustermap Analysis Stage
    subgraph "Clustermap Analysis"
        direction TB
        ClustScript["clustermap.R"]:::analysis
        ClustNB["Clustermap Notebook"]:::analysis
    end

    %% PCA Analysis Stage
    subgraph "PCA Analysis"
        direction TB
        PCANB["PCA Notebook"]:::analysis
        PCAMatrix["annotated_TMM_normalized_counts_matrix.tsv"]:::analysis
        PCAPlotDir["PCA Plots"]:::analysis
    end

    %% Final Reporting Stage
    subgraph "Final Presentation"
        direction TB
        Presentation["Answer ALS Epigenomics Report (PDF)"]:::report
    end

    %% Data Flow
    RawFASTQ --> DataEng --> RenamedFASTQ
    RenamedFASTQ --> CromwellEngine
    CromwellInputs1 --> CromwellEngine
    CromwellInputs2 --> CromwellEngine
    CromwellEngine --> ATACTasks
    ATACTasks --> QCFailed
    ATACTasks --> QCSummary
    QCFailed --> SamplePrepNB
    QCSummary --> SamplePrepNB
    SamplePrepNB --> SampleSheet
    SampleSheet --> CountMatrixScript
    CountMatrixScript --> MatrixTSV
    MatrixTSV --> ClustScript
    ClustScript --> ClustNB
    MatrixTSV --> PCANB
    PCANB --> PCAMatrix
    PCANB --> PCAPlotDir
    ClustNB --> Presentation
    PCAPlotDir --> Presentation
    PCAMatrix --> Presentation

    %% Click Events
    click DataEng "https://github.com/yogionbioinformatics/aals-epigenomics-signal-reproducibility/blob/main/data_engineering/mv_rename_fastqs.sh"
    click CromwellInputs1 "https://github.com/yogionbioinformatics/aals-epigenomics-signal-reproducibility/blob/main/cromwell/input.json"
    click CromwellInputs2 "https://github.com/yogionbioinformatics/aals-epigenomics-signal-reproducibility/blob/main/cromwell/workflows.txt"
    click QCFailed "https://github.com/yogionbioinformatics/aals-epigenomics-signal-reproducibility/blob/main/qc/failed.txt"
    click QCSummary "https://github.com/yogionbioinformatics/aals-epigenomics-signal-reproducibility/blob/main/qc/qc_summary_stats.csv"
    click SamplePrepNB "https://github.com/yogionbioinformatics/aals-epigenomics-signal-reproducibility/blob/main/notebooks/diffbind/1_pre_processing/notebooks/prep_diffbind_sheets.ipynb"
    click SampleSheet "https://github.com/yogionbioinformatics/aals-epigenomics-signal-reproducibility/blob/main/notebooks/diffbind/1_pre_processing/output/sample_sheet.csv"
    click CountMatrixScript "https://github.com/yogionbioinformatics/aals-epigenomics-signal-reproducibility/blob/main/notebooks/diffbind/2_retrieve_count_matrix/scripts/get_matrix.R"
    click MatrixTSV "https://github.com/yogionbioinformatics/aals-epigenomics-signal-reproducibility/blob/main/notebooks/diffbind/2_retrieve_count_matrix/output/matrix.tsv"
    click ClustScript "https://github.com/yogionbioinformatics/aals-epigenomics-signal-reproducibility/blob/main/notebooks/clustermaps/diffbind/scripts/clustermap.R"
    click ClustNB "https://github.com/yogionbioinformatics/aals-epigenomics-signal-reproducibility/blob/main/notebooks/clustermaps/notebooks/clustermap.ipynb"
    click PCANB "https://github.com/yogionbioinformatics/aals-epigenomics-signal-reproducibility/blob/main/notebooks/PCA/notebooks/PCA.ipynb"
    click PCAMatrix "https://github.com/yogionbioinformatics/aals-epigenomics-signal-reproducibility/blob/main/notebooks/PCA/output/matrices/annotated_TMM_normalized_counts_matrix.tsv"
    click PCAPlotDir "https://github.com/yogionbioinformatics/aals-epigenomics-signal-reproducibility/tree/main/notebooks/PCA/output/plots/tmm_normalized"
    click Presentation "https://github.com/yogionbioinformatics/aals-epigenomics-signal-reproducibility/blob/main/presentation/Answer_ALS_Epigenomics_Signal_Reproducibility.pdf"

    %% Styles
    classDef dataPrep fill:#1f77b4,stroke:#000,color:#fff;
    classDef pipeline fill:#2ca02c,stroke:#000,color:#fff;
    classDef qc fill:#ff7f0e,stroke:#000,color:#fff;
    classDef analysis fill:#9467bd,stroke:#000,color:#fff;
    classDef report fill:#7f7f7f,stroke:#000,color:#fff;
```
