How salesforce/codegen works
CodeGen is a family of pretrained code-generation language models, released across three generations. The principal documented workflow is to load a checkpoint and tokenizer, encode a prompt, generate tokens, and decode a completion; the CodeGen1 sample script additionally truncates and prints the result. The graph also preserves the documented CodeGen2 infill capability and the CodeGen1 training example. CodeGen2 and CodeGen2.5 implementation details are not present in the sampled source, so their links are limited to README-documented relationships.
Main entry points: sample.py Sampling CLI · modeling_codegen.py Transformer block · codegen2 CodeGen2 checkpoints · codegen1 CodeGen1 checkpoints
Components 14
Generation workflow
- Sampling CLI
codegen1/jaxformer/hf/sample.py - GPT-2 tokenizer
codegen1/jaxformer/hf/sample.py - Decoded completion
codegen1/jaxformer/hf/sample.py
Model runtime
- Model configuration
codegen1/jaxformer/hf/codegen/configuration_codegen.py - Causal attention
codegen1/jaxformer/hf/codegen/modeling_codegen.py - Transformer MLP
codegen1/jaxformer/hf/codegen/modeling_codegen.py - Transformer block
codegen1/jaxformer/hf/codegen/modeling_codegen.py - CodeGen transformer
codegen1/jaxformer/hf/codegen/modeling_codegen.py - Causal LM head
codegen1/jaxformer/hf/codegen/modeling_codegen.py
Model releases and training
- CodeGen1 checkpoints
codegen1 - CodeGen2 checkpoints
codegen2 - CodeGen2.5 checkpoints
codegen25 - DeepSpeed training example
codegen1/jaxformer/hf/train_deepspeed.py
Other components
- Model user
Connections 15
- Model user → Sampling CLI: provides prompt
codegen1/jaxformer/hf/sample.py - Sampling CLI → GPT-2 tokenizer: encodes prompt
codegen1/jaxformer/hf/sample.py - Sampling CLI → Causal LM head: generates tokens
codegen1/jaxformer/hf/sample.py - Sampling CLI → GPT-2 tokenizer: decodes tokens
codegen1/jaxformer/hf/sample.py - Sampling CLI → Decoded completion: truncates and prints
codegen1/jaxformer/hf/sample.py - Model configuration → CodeGen transformer: configures
codegen1/jaxformer/hf/codegen/modeling_codegen.py - Transformer block → Causal attention: invokes
codegen1/jaxformer/hf/codegen/modeling_codegen.py - Transformer block → Transformer MLP: invokes
codegen1/jaxformer/hf/codegen/modeling_codegen.py - CodeGen transformer → Transformer block: runs layers
codegen1/jaxformer/hf/codegen/modeling_codegen.py - Causal LM head → CodeGen transformer: runs transformer
codegen1/jaxformer/hf/codegen/modeling_codegen.py - Sampling CLI → CodeGen1 checkpoints: loads checkpoint
codegen1/jaxformer/hf/sample.py - Model user → CodeGen2 checkpoints: loads and generates
README.md - Model user → CodeGen2.5 checkpoints: loads and generates
README.md - CodeGen2 checkpoints → Decoded completion: supports infill generation
README.md - DeepSpeed training example → CodeGen1 checkpoints: loads model
codegen1/jaxformer/hf/train_deepspeed.py