How timedreamer/atac-seq_analysis works
This project is an ATAC‑Seq analysis pipeline aimed at processing raw sequencing data through a series of bioinformatics steps. The pipeline starts with downloading FASTQ files, then continues through adapter trimming, mapping with Bowtie2, peak calling with Genrich, and finally calculating the Fraction of Reads in Peaks (FRiP). It’s orchestrated using SLURM job scripts (along with a Snakefile-based workflow), making it a modular and reproducible pipeline designed for distributed high-performance computing environments.
Project Components
• Data Acquisition: Represented by the “01_download_fastq.slurm” script. This component is responsible for fetching FASTQ files from external resources (e.g., via URLs provided by SRA-Explorer).
• Pre-processing and Alignment: Handled by “02_submit_snake.slurm” in combination with the “atac_se_pipeline_bt2.smk” Snakefile and the “bt2_allignment.slurm” script. These perform critical steps such as adapter trimming (with fastp) and mapping (with Bowtie2).
• Peak Calling: Managed by “03_call_peaks.slurm”, this module uses Genrich to call peaks directly from post-alignment data.
• FRiP Calculation: The “calc_FRiP.slurm” script computes the Fraction of Reads in Peaks, a key quality metric in ATAC‑Seq analysis.
• Orchestration and Configuration: The “submit_all.sh” script enables sequential submission of all jobs, while “exp_settings.yaml” holds configuration parameters that influence the workflow.
• Support Data: The file “zmav4.chr.length.txt” provides reference information (e.g., maize contig data) used to exclude certain chromosomal data during analysis.
Relationships & Data Flow
• The pipeline follows a sequential processing model:
• FASTQ Download → Pre-processing & Alignment → Peak Calling → FRiP Calculation.
• Each component’s output serves as the input for the next stage. For example, the downloaded FASTQ files are trimmed and then mapped; the alignment results feed into the peak calling process.
• The “submit_all.sh” acts as the master orchestrator, triggering each step in the correct order on a SLURM-managed cluster.
• Configuration parameters defined in “exp_settings.yaml” should be highlighted as settings influencing multiple stages, ensuring a customizable, reproducible workflow.
Architectural Patterns and Important Tools
• Modularity: Each SLURM script encapsulates a self-contained processing step, facilitating independent monitoring, debugging, and reusability.
• Workflow Automation: The use of a Snakefile (atac_se_pipeline_bt2.smk) underlines a workflow management approach that improves reproducibility.
• External Tools Integration: Key bioinformatics tools (fastp for adapter trimming, Bowtie2 for alignment, and Genrich for peak calling) are core to the processing steps.
• Distributed Processing: The reliance on SLURM scripts stresses the pipeline’s deployment in a parallelized, high-performance computing environment.