How myersgroup/relate works
Main Components
• Data Input & File Formats:
– Represent components that deal with reading genetic/haplotype data (example/data, include/file_formats).
– Components for converting to intermediate representations (see “ConvertToTreeSequence.cpp”, “FileFormats.cpp” and the “tskit” subdirectory).
• Preprocessing / Input Preparation:
– Modules in scripts/PrepareInputFiles and parts in include/pipeline that prepare the input and segregate the data.
• Core Genealogy Construction:
– The “pipeline” components (e.g., BuildTopology, OptimizeParameters, MakeChunks, Paint) are the heart of the system that build genealogies.
– Include the “tree_builder”, “branch_length_estimator”, and “tree_comparer” modules (located in include/src) showing how the lineage trees are built and refined.
• Analysis Modules:
– Separate C++ modules for computing coalescent rate (include/evaluate/coalescent_rate) and mutation rate (include/evaluate/mutation_rate) along with selection assessment (include/evaluate/selection).
• Post-processing / Visualization:
– Modules dedicated to output formatting and visualization (e.g., include/treeview for the tree view module, scripts/TreeView).
• Build and Execution Infrastructure:
– The build system (cmake, CMakeLists.txt files) and bin directory containing compiled executables.
• Scripting and Job Management:
– Several shell and batch scripts (in scripts/, docs/download/…/scripts) that manage parallel execution (using SGE, LSF, Slurm) and the orchestration of the computational pipelines.
• Documentation & Examples:
– A separate module containing detailed documentation (docs/) and example run scripts (example/, documentation PDFs in docs/figures).
Relationships and Data Flow
• Input data flows from the external genotype/haplotype files (from example/data or user-specified data) into the “Data Input & File Formats” component.
• The “Preprocessing / Input Preparation” component adapts and cleans this data for the analysis pipeline.
• The “Core Genealogy Construction” component uses the preprocessed data to build the genealogical trees by invoking modules like BuildTopology, InferBranchLengths, and MakeChunks.
• Outputs from the genealogy construction feed into the “Analysis Modules” where coalescent rate, mutation rate, and selection analyses are performed.
• Finally, results flow into the “Post-processing / Visualization” component which formats results and provides visual outputs (tree view, plots, and summary files).
Architectural Patterns and Design Principles
• Modular Design:
• Pipeline & Workflow:
• Extensibility and Reusability:
– Notice the clear separation in the “include” directory with self-contained modules (and test support in include/test) that indicate good modular design.
• Use of Scripting for orchestration:
– Include an “orchestration” component to represent cron-like or scheduler interactions (SGE, LSF, Slurm) as there are many dedicated scripts.