How dm4m/fsru works
1. Repository purpose
- This is a compact PyTorch research implementation of the FSRU model from the AAAI 2024 paper on multimodal rumor detection.
- The repo is not a service or app; it is an experiment pipeline centered on training and evaluating a multimodal classifier.
2. Main execution path
- main_kfold.py is the orchestration entry point.
- It likely drives k-fold cross-validation, wires together argument parsing, dataset loading, model construction, loss computation, and evaluation.
3. Configuration boundary
- arguments.py is the central configuration layer.
- It likely defines CLI flags and experiment hyperparameters such as dataset choice, batch size, learning rate, fold settings, and model dimensions.
- This is the main control boundary between the user and the training pipeline.
4. Data ingestion subsystem
- data_loader.py and datasets.py form the input pipeline.
- datasets.py likely defines dataset objects and sample formatting for multimodal rumor examples.
- data_loader.py likely builds PyTorch DataLoader instances, handles batching, shuffling, and train/validation split logic for each fold.
5. Input data characteristics
- The project depends on external datasets: Weibo and Twitter/MediaEval.
- These datasets are not stored in the repo, so the architecture has a clear external data boundary: raw multimodal samples come from downloaded dataset files and are fed into the local training pipeline.
6. Model subsystem
- model.py contains the FSRU network implementation.
- Based on the paper and repository name, this is the core multimodal fusion architecture, where text/image features are transformed and combined in the frequency spectrum domain.
- This is the central compute boundary of the system.
7. Loss / optimization subsystem
- loss.py encapsulates the objective used during training.
- It likely provides the classification loss and any auxiliary terms needed by FSRU.
- This module sits between model outputs and the optimizer loop in the training script.
8. Training and evaluation flow
- The flow is: arguments → dataset/dataloader → model forward pass → loss computation → backpropagation → fold-level evaluation.
- Because the entry point is main_kfold.py, the architecture emphasizes repeated train/validate cycles rather than a single static train/test run.
9. Experiment structure
- The repository is designed for reproducible academic experiments.
- There is no obvious server, database, or message queue layer; all state is local, with data read from disk and metrics produced by the training run.
10. Outputs and artifacts
- The main outputs are trained model checkpoints, validation/test metrics, and possibly fold-aggregated results.
- The fig/ directory is documentation-only and contains the model illustration used in the README, not a runtime dependency.
11. Important architectural boundaries for a diagram
- External datasets on one side.
- Local experiment runner and configuration in the center.
- Data loading/formatting feeding the FSRU model and loss modules.
- Cross-validation and metrics aggregation wrapping the whole training process.
12. Technology stack
- Python + PyTorch is the core runtime stack.
- The README lists supporting scientific libraries: NumPy, Pandas, SciPy, scikit-learn, and timm, which indicate standard ML preprocessing, metrics, and model utilities.
Main entry points: main_kfold.py K-fold runner · model.py FSRU model · data_loader.py Data loader · datasets.py Datasets
Components 14
External data
Downloaded multimodal rumor datasets used at runtime
- Weibo data · Multimodal rumor dataset loaded from disk, not stored in repo
- Twitter data · MediaEval/Twitter-style multimodal rumor data loaded from disk
Experiment pipeline
Local training, evaluation, and model implementation
- K-fold runner · Main orchestration script for repeated train/validate cycles
main_kfold.py - CLI config · Central argument and hyperparameter definition layer
arguments.py - Data loader · Builds PyTorch loaders and fold-aware batch streams
data_loader.py - Datasets · Dataset definitions and multimodal sample construction
datasets.py - FSRU model · Core FSRU architecture for text-image fusion and classification
model.py - Losses · Training objective and any auxiliary loss terms
loss.py - Train loop · Forward pass, backward pass, and parameter updates inside each fold
main_kfold.py - Evaluation · Validation and test scoring for each fold
main_kfold.py - Fold split · K-fold partitioning and run aggregation across splits
main_kfold.py - Checkpoints · Saved model weights and fold-level experiment outputs
Connections 17
- README → K-fold runner: describes · Documents the main experiment workflow
- README → Figures: includes · References the illustrative architecture figure
- CLI config → K-fold runner: configures · Provides command-line options and experiment settings
- K-fold runner → Fold split: orchestrates · Runs repeated cross-validation cycles
- Fold split → Data loader: builds · Creates fold-specific data streams
- Data loader → Datasets: wraps · Uses dataset objects to format samples
- Datasets → Weibo data: reads · Loads Weibo samples from external storage
- Datasets → Twitter data: reads · Loads Twitter/MediaEval samples from external storage
- Data loader → FSRU model: feeds · Supplies batched multimodal inputs to the network
- K-fold runner → FSRU model: constructs · Instantiates the FSRU model for each run
- FSRU model → Losses: produces · Model outputs are scored by the training objective
- Losses → Train loop: guides · Loss drives backpropagation and updates
- FSRU model → Train loop: updates · Parameters are optimized during training
- Train loop → Evaluation: yields · Completed training cycle is validated and tested
- Evaluation → Fold split: aggregates · Fold metrics are combined across splits
- Train loop → Checkpoints: writes · Produces checkpoints and experiment outputs
- Evaluation → Checkpoints: writes · Stores fold metrics and summary results