How tencent-hunyuan/hunyuan-turbos3 works
Project type and purpose
• This is not a standard application codebase but a research-grade large language model (LLM) project.
• Hunyuan-TurboS is a 56B-activated parameter hybrid Transformer-Mamba MoE model with an adaptive chain-of-thought mechanism.
• The README points to a technical report (PDF) for full details and describes both model architecture and multi-stage training pipeline.
File structure
• Top level contains only README.md and the technical report PDF – no code.
README for additional insights
• Key architecture: Mamba2 blocks for long-sequence efficiency, standard Transformer Attention layers, Mixture-of-Experts (MoE) feed-forward networks with grouped-query attention.
• Adaptive long-short chain-of-thought controller that switches inference modes.
• Training stages: Pre-training on 16T tokens, Supervised Fine-Tuning (SFT), Adaptive CoT Fusion, Multi-round Deliberation Learning, and two-stage RL.