How laurapeng/decoder works
Main Components
• Input/Configuration Module:
• Configuration files: config_de_novo.tsv, config_de_novo_parmode.tsv, and config_single_sample.tsv.
• Data files: Located under the “data” directory (e.g., .mat, .tsv, and sometimes .csv files), including demo_data examples.
• Core Processing Modules:
• De novo deconvolution functions: Main entry points are Decon_de_novo.m (integrated mode), Decon_de_novo_parmode_step1.m, and Decon_de_novo_parmode_step2.m (for parallel/HPC mode).
• Single-sample deconvolution: Executed via the Decon_single_sample.m script.
• Utility/Supporting Functions:
• The “utils” directory contains helper scripts (e.g., Load_data.m, Train_NMF_seed.m, nnmf_unmix_quick.m, Plot_factor_tree.m) that support data loading, transformation, processing, plotting, and saving of intermediate results.
• External/Cluster Integration:
• Launch and scheduling scripts such as launch_DECODER_K7.m and launch_sbatch.sh, which facilitate parallel processing on a cluster/HPC environment.
• Output/Result Management:
• The “results” directory where deconvolution outputs are stored (such as factor trees in PDF, sample weights, gene scores, and other text files), representing the analysis results.
Relationships and Data Flow
• The user (or automation script) begins by placing data and setting configurations in the working directory.
• The input component (configuration and data files) is read and preprocessed by the core MATLAB functions (Decon_de_novo.m or Decon_single_sample.m).
• In the integrated or de novo modes, the framework processes the raw matrix data via several algorithmic steps (e.g., non-negative matrix factorization and iterative updates) that are supported by various utility functions.
• In the parallel HPC mode, processing is split into two steps: first (using launch_DECODER_K7.m/launch_sbatch.sh) the system launches multiple jobs for increasing K values, and then a collection step (Decon_de_novo_parmode_step2.m) merges and finalizes the results.
• Once analysis completes, the outputs are generated in the “results” directory (with detailed subfolders especially for multi-dataset results like TCGA and Moffitt microarray data).
Architectural Patterns and Technologies
• Modular design: Main deconvolution functions are separated by analysis mode, and utility functions encapsulate common tasks (data loading, plotting, etc.).
• Batch/parallel processing: Parallel mode implementation demonstrates that the framework has been designed to run on HPC clusters using shell scripts and MATLAB integration.
• Scientific computing: The implementation uses MATLAB-specific functions and scripting for numerical and matrix-based data analysis, including NMF/nnmf operations.
• Clear Labels: Each block should be clearly labeled (e.g., “Config Files (TSV)”, “Data Files (MAT/TSV)”, “Decon_de_novo.m”, “Parallel Dispatch (launch_DECODER_K7.m/launch_sbatch.sh)”, “Aggregate Results (Decon_de_novo_parmode_step2.m)”).
Final Note: