How alihkz94/long-chimeric-reads-project works
This project supports a research study titled “Evaluation of chimera detection algorithms on long‐read amplicons” and is organized as a modular analysis pipeline. Its purpose is to generate and analyze both simulated and real datasets to assess the performance of chimera detection algorithms, integrating multiple tools (e.g., BLAST, DADA2) and producing quantitative evaluations (like F1 scores) along with visualizations.
Main Components
• Data Sources:
• Simulated Data: Contains scripts for generating synthetic sequences, creating chimeras, performing quality filtering, and statistical analysis.
• Real Data: Contains scripts for processing actual experimental datasets, including steps like dereplication, tag-jump filtering, and statistical analysis.
• Processing Pipelines:
• BlasCh Pipeline: Scripts (e.g., BLASTn.sh, BlasCh_PipeCraft.py) for running BLAST alignments and processing XML outputs to detect false positive and false negative chimeras.
• DADA2 Pipeline: Contains scripts (Python and R) that either complement or investigate chimera detection via the DADA2 approach.
• Visualization & Reporting:
• Supporting/Utility Scripts:
• Various shell scripts and Python utilities across folders (e.g., make_selfdb.sh, cutadapt.sh) that support pre-processing, database creation, and additional filtering steps.
Relationships and Data Flow
• Data Ingestion: Start with the “Simulated_data” and “Real_data” folders where raw inputs are processed.
• Processing Modules:
• For real data, illustrate a similar pipeline where data is pre-processed using shell scripts, followed by chimera detection and filtering.
• That outputs from BLAST alignments (handled in the BlasCh directory) are passed to specialized modules like BlasCh_false_negative.py and false_negative_info.py for further analysis.
• Integration:
• How the outputs (filtered chimera detections, false positive/negative metrics) from both pipelines feed into common statistical analysis scripts.
• Visualization:
• Finally, direct outputs from the analysis pipelines towards the “Figures_tables” component which generates comprehensive graphs and tables.
Architectural Patterns
• Modular/Layered Architecture: Illustrate how each directory represents a functional module within the overall workflow. For example, separate channels for simulated versus real datasets ensure clarity in the processing flow.
• Integration Points: Emphasize the central role of common output files (like statistical measures and F1 scores) which are computed from multiple upstream modules to provide a holistic analysis.
Relevant Technologies and Tools
• Shell Scripting: Numerous shell scripts manage tasks like self-database building (make_selfdb.sh), data dereplication, and quality filtering.
• Python: Used across many modules (e.g., BlasCh_PipeCraft.py, chimera_generator.py) to handle data processing tasks.
• R: Utilized mainly in the statistical analysis and visualization components (e.g., dada2_chimera.R, barplot scripts, and other figure generation scripts).
• BLAST: Used within the BlasCh pipeline for sequence alignment.