How lilit-nersisyan/fivepseq works
Project Type and Purpose
• This is a Python-based command‐line bioinformatics tool (fivepseq) for analyzing 5′ endpoint distributions in RNA degradome (BAM) datasets.
• It also provides a shell‐based preprocessing pipeline (preprocess_scripts/fivepseq_preprocess.sh) to go from FASTQ → QC → trimming → alignment → deduplication → bedgraph.
File‐Structure Analysis
• Top‐level:
• fivepseq/ – core package (CLI entry, config, logic, util, viz, tests)
• preprocess_scripts/ – shell script pipeline
• setup.py, version.txt, README.rst
• fivepseq/logic:
• structures/ – domain models (Alignment, Annotation, Genome, Count objects)
• algorithms/ – statistical and pipeline modules (count_stats, general_pipelines)
• fivepseq/viz: uses Bokeh, HTML templating for plots and report generation
• fivepseq/util: I/O helpers (readers, writers, formatting)
• Configuration: fivepseq/config.py, CLI entry in __main__.py
• Tests under fivepseq/tests ensure modular correctness.
Key Components
a. User Inputs
• Raw FASTQ (optional) → Preprocessor Script
• Aligned reads (.bam/.bai), Genome (.fa/.fasta), Annotation (.gtf/.gff)
b. Preprocessing Pipeline (shell script)
• FASTQC/MultiQC → Cutadapt (adapter trimming) → UMI-tools → STAR aligner → Deduplication → Bedgraph generation
• Invokes external tools (FASTQC, STAR, Samtools, bedtools, etc.)
• Output: deduplicated BAM + index
c. fivepseq CLI Application
• CLI Parser (fivepseq/__main__.py) → Config Loader (config.py)
• Readers (util/readers.py) parse BAM, FASTA, GFF
• Data Structures populated (logic/structures)
• Statistical Algorithms (logic/algorithms/count_stats)
• Pipelines (logic/algorithms/general_pipelines: count_pipeline, viz_pipeline)
• Visualization (viz/bokeh_plots.py, header_html.py, template.html)
• Writers (util/writers.py) output CSV/TSV, HTML, image files (via phantomjs)
d. External Dependencies & Services
• Python libs: numpy, pysam, cython, bokeh, selenium/pillow/phantomjs
• Optionally run inside Docker
Relationships and Data Flow
• Preprocessor Script → external tool calls → generates BAM → fivepseq CLI
• fivepseq CLI:
Input files → Readers → Data Structures → Statistical Algorithms → Visualization Pipeline → Writers → Output Directory (HTML report, plots, stats)
Architectural Patterns & Principles
• Modular separation: I/O (util) vs. Domain Models (structures) vs. Computation (algorithms) vs. Presentation (viz)
• Pipeline pattern: sequential stages (count vs. viz pipelines)
• Dependency injection via config.py and CLI arguments
• Testable units: full test suite under fivepseq/tests
Technologies & Frameworks
• Python 3.x
• Bokeh (visualization)
• External CLI tools (STAR, Samtools, etc.)
• Sphinx for documentation (doc/source, build)
• Shell scripting for preprocessing
• Docker (optional containerization)