How gauravj49/bulkrnaseqde works
Main Components
• Input Data & Annotations:
• “input” folder, highlighting its subdirectories, especially the “annotation” files (with bed files for hg, mm, and sacCer gene symbols).
• Wrapper/Control Scripts:
• Two main shell scripts (01_user_projectname_rnaseq_deseq2_wrapper.sh and 02_user_projectname_rnaseq_ohneOutliers_wrapper.sh) that initiate the pipeline.
• Pre-Processing & Data Normalization:
• Components for normalization (scripts/R_normalize_data.R) and unwanted variation removal.
• Differential Expression Analysis:
• DE analysis module (scripts/R_RUVseq_DESeq2_DEanalysis.R) which runs DESeq2 and integrates RUVseq.
• Post-Processing & Visualization:
• A module for heatmap generation (scripts/R_heatmap.R and scripts/draw_mirs_heatmap.py).
• Gene Annotation:
• Sorting & Logging:
• Additional processing steps like sorting count files (scripts/sort_counts_dir.sh) and generating diagnostic plots/logs as noted in the README.
Relationships and Data Flow
• Data normalization (R script) → unwanted variation removal and DE analysis (R_RUVseq_DESeq2_DEanalysis.R).
• Next, the pipeline splits into two branches: one that goes to statistical result post-processing (gene annotation with Python script) and one that goes to result visualization (heatmaps).
Architectural Patterns and Design Principles
• Modular Pipeline Architecture: The design clearly separates the functions (data normalization, differential expression analysis, annotation, and visualization) into individual, script-based modules.
• Script Orchestration: The wrapper Bash scripts tie everything together and manage the workflow, which is typical in computational pipelines.
• Reusability & Extensibility: Notice the separation between R and Python scripts, which can be updated individually as needed.
• Data Flow (Linear & Branching): Emphasize the sequential steps with branching for multiple outputs (heatmaps, annotated results, and diagnostic plots).
Key Technologies and Frameworks
• R: For normalization, DE analysis (DESeq2), heatmap drawing.
• Python: For adding gene annotations, additional heatmap drawing.
• Shell Scripts: For orchestrating the workflow.
• Cluster Environment: Execution is targeted for an HPCC cluster though the pipeline is adaptable.