How core-bioinformatics/noisyr works
Project Type and Purpose
• Project Type: It is an R package developed for bioinformatics analysis, specifically processing HTS (high-throughput sequencing) data.
• Purpose: To capture and remove technical noise from gene expression data (either from count matrices or directly from sequencing BAM files), ensuring that downstream biological signals are preserved.
File Structure
• Top-Level Directories/Files: Notice key directories like “R” (where all the core functions reside), “docs” (documentation and generated articles and references), “man” (manual pages), “vignettes” (extended usage examples), and “inst” (extra package files including citation and example data).
• The R directory reveals a modular design where each file implements one part of the pipeline such as expression similarity, noise thresholding, and noise removal.
• The existence of multiple documentation files (e.g., in “docs” and “vignettes”) indicates both detailed explanations of functions and workflows, along with usage examples.
Main Components
a. Data Input/Preparation Module:
• Components: Functions like cast_matrix_to_numeric() and cast_gtf_to_genes() are responsible for preparing the raw count matrices and gene annotation files.
• Input Elements: Raw sequencing outputs, count matrices (e.g., from alignment pipelines), and BAM files.
b. Similarity Calculation Module:
• Components: Two approaches are supported here – one using the count matrix (calculate_expression_similarity_counts()) and another using transcript-level data from BAM files (calculate_expression_similarity_transcript()).
• Details: Include the use of sliding windows and selection from multiple similarity metrics.
c. Noise Quantification Module:
• Components: Functions such as calculate_noise_threshold_base(), calculate_first_minimum_density(), and get_methods_calculate_noise_threshold() provide the means to estimate the noise threshold using different strategies (e.g., smoothing or boxplot summarisation).
• Role: This module takes the output from the similarity calculation and identifies an experiment-specific noise threshold.
d. Noise Removal Module:
• Components: Functions remove_noise_from_matrix() and remove_noise_from_bams() that leverage the computed noise thresholds to filter out low expression genes or exons.
• Outcome: Denoted by a noise-reduced count matrix or a set of filtered BAM files.
Capturing Relationships and Data Flow
• Sequential processing:
• Next, route the similarity output to the Noise Quantification module.
• Dependencies such as the fact that similarity metrics inform threshold determination, which in turn governs the removal step.
Architectural Patterns and Technologies
• That the package is modular with each function or file corresponding to a distinct processing step.
• The design follows a pipeline pattern where data transforms from one stage to the next.
• Supporting utility functions separately from the core pipeline.
• Next, branch out into two parallel paths under a “Similarity Calculation” block – one for the count matrix approach and one for the transcript-based approach.
• Merge these outputs into the “Noise Quantification” block, clearly labeling how similarity assessments feed into threshold determination.
• Follow this with a “Noise Removal” block showing the final step where noisy data (below threshold) is filtered out.
• Use clear labels (e.g., “calculate_expression_similarity_counts()”, “calculate_noise_threshold_base()”) on each block.
• Side blocks/comments for supporting functions (e.g., “get_methods_correlation_distance()”, “optimise_window_length()”) that fine-tune the pipeline.