# core-bioinformatics/noisyr architecture

> Architecture diagram of the GitHub repository core-bioinformatics/noisyr, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/core-bioinformatics/noisyr
- Repository: https://github.com/core-bioinformatics/noisyr

## Overview

##### Project Type and Purpose
- Project Type: It is an R package developed for bioinformatics analysis, specifically processing HTS (high-throughput sequencing) data.
- Purpose: To capture and remove technical noise from gene expression data (either from count matrices or directly from sequencing BAM files), ensuring that downstream biological signals are preserved.

##### File Structure
- Top-Level Directories/Files: Notice key directories like “R” (where all the core functions reside), “docs” (documentation and generated articles and references), “man” (manual pages), “vignettes” (extended usage examples), and “inst” (extra package files including citation and example data).
- The R directory reveals a modular design where each file implements one part of the pipeline such as expression similarity, noise thresholding, and noise removal.
- The existence of multiple documentation files (e.g., in “docs” and “vignettes”) indicates both detailed explanations of functions and workflows, along with usage examples.

##### Main Components
a. Data Input/Preparation Module:
- Components: Functions like cast_matrix_to_numeric() and cast_gtf_to_genes() are responsible for preparing the raw count matrices and gene annotation files.
- Input Elements: Raw sequencing outputs, count matrices (e.g., from alignment pipelines), and BAM files.
b. Similarity Calculation Module:
- Components: Two approaches are supported here – one using the count matrix (calculate_expression_similarity_counts()) and another using transcript-level data from BAM files (calculate_expression_similarity_transcript()).
- Details: Include the use of sliding windows and selection from multiple similarity metrics.
c. Noise Quantification Module:
- Components: Functions such as calculate_noise_threshold_base(), calculate_first_minimum_density(), and get_methods_calculate_noise_threshold() provide the means to estimate the noise threshold using different strategies (e.g., smoothing or boxplot summarisation).
- Role: This module takes the output from the similarity calculation and identifies an experiment-specific noise threshold.
d. Noise Removal Module:
- Components: Functions remove_noise_from_matrix() and remove_noise_from_bams() that leverage the computed noise thresholds to filter out low expression genes or exons.
- Outcome: Denoted by a noise-reduced count matrix or a set of filtered BAM files.

##### Capturing Relationships and Data Flow
- Sequential processing:
- Next, route the similarity output to the Noise Quantification module.
- Dependencies such as the fact that similarity metrics inform threshold determination, which in turn governs the removal step.

##### Architectural Patterns and Technologies
- That the package is modular with each function or file corresponding to a distinct processing step.
- The design follows a pipeline pattern where data transforms from one stage to the next.
- Supporting utility functions separately from the core pipeline.

- Next, branch out into two parallel paths under a “Similarity Calculation” block – one for the count matrix approach and one for the transcript-based approach.
- Merge these outputs into the “Noise Quantification” block, clearly labeling how similarity assessments feed into threshold determination.
- Follow this with a “Noise Removal” block showing the final step where noisy data (below threshold) is filtered out.
- Use clear labels (e.g., “calculate_expression_similarity_counts()”, “calculate_noise_threshold_base()”) on each block.
- Side blocks/comments for supporting functions (e.g., “get_methods_correlation_distance()”, “optimise_window_length()”) that fine-tune the pipeline.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Input & Preparation
    subgraph "Raw Data Input & Preparation"
        DIN["cast_matrix_to_numeric()"]:::input
        DIG["cast_gtf_to_genes()"]:::input
    end

    %% Similarity Calculation Module
    subgraph "Similarity Calculation"
        SC1["calculate_expression_similarity_counts()"]:::process
        SC2["calculate_expression_similarity_transcript()"]:::process
    end

    %% Merged Similarity Output
    SIM["Similarity Output"]:::process

    %% Noise Quantification Module
    subgraph "Noise Quantification"
        NQ1["calculate_noise_threshold_base()"]:::quant
        NQ2["calculate_first_minimum_density()"]:::quant
        NQ3["get_methods_calculate_noise_threshold()"]:::quant
        NQ4["calculate_noise_threshold_method_statistics()"]:::quant
    end

    %% Noise Removal Module
    subgraph "Noise Removal"
        NR1["remove_noise_from_matrix()"]:::removal
        NR2["remove_noise_from_bams()"]:::removal
    end

    %% Connections
    DIN -->|"providesNumericMatrix"| SC1
    DIN -->|"providesNumericMatrix"| SC2
    DIG -->|"providesAnnotations"| SC1
    DIG -->|"providesAnnotations"| SC2

    SC1 -->|"similarityScores"| SIM
    SC2 -->|"similarityScores"| SIM

    SIM -->|"inputForThresholding"| NQ1
    SIM -->|"inputForThresholding"| NQ2
    SIM -->|"inputForThresholding"| NQ3
    SIM -->|"inputForThresholding"| NQ4

    NQ1 -->|"determinesNoiseLevel"| NR1
    NQ2 -->|"determinesNoiseLevel"| NR1
    NQ3 -->|"determinesNoiseLevel"| NR1
    NQ4 -->|"supportsStatistics"| NR1

    NQ1 -->|"determinesNoiseLevel"| NR2
    NQ2 -->|"determinesNoiseLevel"| NR2
    NQ3 -->|"determinesNoiseLevel"| NR2
    NQ4 -->|"supportsStatistics"| NR2

    %% Click Events for Component Mapping
    click DIN "https://github.com/core-bioinformatics/noisyr/blob/master/R/cast_matrix_to_numeric.R"
    click DIG "https://github.com/core-bioinformatics/noisyr/blob/master/R/cast_gtf_to_genes.R"
    click SC1 "https://github.com/core-bioinformatics/noisyr/blob/master/R/calculate_expression_similarity_counts.R"
    click SC2 "https://github.com/core-bioinformatics/noisyr/blob/master/R/calculate_expression_similarity_transcript.R"
    click NQ1 "https://github.com/core-bioinformatics/noisyr/blob/master/R/calculate_noise_threshold.R"
    click NQ2 "https://github.com/core-bioinformatics/noisyr/blob/master/R/calculate_first_minimum_density.R"
    click NQ3 "https://github.com/core-bioinformatics/noisyr/blob/master/R/get_methods_calculate_noise_threshold.R"
    click NQ4 "https://github.com/core-bioinformatics/noisyr/blob/master/R/calculate_noise_threshold_method_statistics.R"
    click NR1 "https://github.com/core-bioinformatics/noisyr/blob/master/R/remove_noise_from_matrix.R"
    click NR2 "https://github.com/core-bioinformatics/noisyr/blob/master/R/remove_noise_from_bams.R"

    %% Styles
    classDef input fill:#AED6F1,stroke:#1B4F72,stroke-width:2px;
    classDef process fill:#A9DFBF,stroke:#196F3D,stroke-width:2px;
    classDef quant fill:#F9E79F,stroke:#7D6608,stroke-width:2px;
    classDef removal fill:#F1948A,stroke:#922B21,stroke-width:2px;
```
