How xampprocky/tokei works
Tokei is a Rust library and CLI for counting files and classifying lines as code, comments, or blanks by language. The CLI accepts paths and optional prior statistics, applies configuration and ignore rules, scans and analyzes source (including embedded languages), aggregates results, then sorts and prints or serializes them. The README documents broad language support, multiple output formats, and library use. Some implementation links rely on README workflow descriptions because the relevant source was not excerpted.
Main entry points: main.rs CLI Entry · mod.rs Language Aggregation · syntax.rs Syntax Counter · input.rs Import and Export
Components 18
CLI and Output
- CLI Entry
src/main.rs - CLI Options
src/cli.rs - Terminal Printer
src/cli_utils.rs - Import and Export
src/input.rs - Display Constants
src/consts.rs
File Discovery
- Configuration
src/config.rs - File Walker
src/utils/fs.rs
Language Analysis
- Language Catalog
src/language/languages.rs - Language Types
src/language/language_type.rs - Syntax Counter
src/language/syntax.rs - Embedded Languages
src/language/embedding.rs - Byte Helpers
src/utils/ext.rs
Statistics and API
- Language Aggregation
src/language/mod.rs - File Statistics
src/stats.rs - Result Sorting
src/sort.rs - Library API
src/lib.rs - Utility Macros
src/utils/macros.rs
Other components
- CLI User
Connections 21
- CLI User → CLI Entry: provides paths
README.md - CLI Entry → CLI Options: parses options
src/main.rs - CLI Entry → Import and Export: loads prior stats
src/main.rs - CLI Entry → Configuration: loads settings
README.md - CLI Entry → Language Aggregation: requests statistics
src/main.rs - Language Aggregation → File Walker: discovers files
- File Walker → Configuration: uses scan rules
- Language Aggregation → Language Catalog: uses language definitions
src/language/mod.rs - Language Catalog → Language Types: defines supported types
- Language Aggregation → Syntax Counter: counts source
src/language/mod.rs - Syntax Counter → Embedded Languages: finds embedded regions
src/language/syntax.rs - Syntax Counter → Byte Helpers: uses byte helpers
src/language/syntax.rs - Syntax Counter → File Statistics: produces counts
src/language/syntax.rs - Language Aggregation → File Statistics: stores reports
src/language/mod.rs - CLI Entry → Result Sorting: sorts results
src/main.rs - CLI Entry → Terminal Printer: prints results
src/main.rs - CLI Entry → Display Constants: sets output widths
src/main.rs - CLI Entry → Import and Export: serializes results
src/main.rs - Import and Export → File Statistics: encodes reports
src/input.rs - Library API → Configuration: exports configuration
src/lib.rs - Library API → Language Aggregation: exports language API
src/lib.rs