How juspay/codetraverse works
1. Repository shape
CODETRAVERSE is primarily a Python static-analysis engine, with a separate TypeScript/Node bridge for invoking it from JavaScript consumers. The Python side owns parsing, component extraction, graph construction, and file outputs; the bridge is a thin integration layer.
2. Main execution entry points
The top-level main.py is the primary CLI for analyzing a repository, while path.py is a second CLI for querying the generated graph. The Python package also has codetraverse/main.py, which appears to be the internal module entry point behind the root script.
3. Analysis pipeline
The core flow is: walk a source repository, detect the target language, parse each file, extract structural components and dependencies, serialize per-file summaries, then unify everything into a global dependency graph. This is a read-only analysis pipeline; it does not modify the analyzed codebase.
4. Language-specific front ends
Support is split across per-language modules in codetraverse/extractors/ and codetraverse/adapters/ for Haskell, ReScript, TypeScript, Rust, Go, and Python. The extractors likely perform language-aware AST/component extraction, while the adapters normalize those results into the common internal model.
5. Shared component model
codetraverse/base/component_extractor.py and codetraverse/registry/extractor_registry.py indicate a central abstraction layer that maps files/languages to the right extractor implementation. This is the main boundary that keeps the rest of the system language-agnostic.
6. Graph construction and storage
codetraverse/utils/networkx_graph.py and codetraverse/utils/graph_partitioner.py suggest the extracted dependencies are assembled into a NetworkX graph. The outputs described in the README are GraphML for interoperability and gpickle for Python-native reuse.
7. Per-file JSON artifacts
The fdep/ output directory contains one JSON summary per source file. These files are the intermediate, file-scoped representation of extracted components before they are unified into the repo-wide graph.
8. Diff and impact-analysis subsystem
The codetraverse/ast_diff/ package contains language-specific diff logic, plus shared helpers like basefilediff.py, Detailedchanges.py, and integration with Git/Bitbucket wrappers. This is a separate analysis path focused on comparing AST-level changes and likely feeding impact or change-tracking workflows.
9. Repository traversal and path utilities
codetraverse/path.py and codetraverse/utils/blackbox.py indicate utility layers for repo path handling and graph query operations. The README’s path CLI uses these outputs to answer shortest-path and neighborhood queries over the dependency graph.
10. Node/TypeScript bridge
The bridge/ directory is a standalone TypeScript package with its own tests and a python-runner abstraction. Architecturally, it wraps the Python engine so JS/TS callers can launch analyses without embedding Python logic directly.
11. Fixture-driven validation
sample_code_repo_test/ contains small example repositories in the supported languages, used to validate extraction behavior across language boundaries. The tests/ tree mirrors the package structure, with focused tests for adapters, extractors, graph utilities, registry behavior, and the CLI entry points.
12. Packaging and deployment boundary
The project is packaged as both a Python distribution (pyproject.toml, setup.py, requirements.txt, uv.lock) and a Node package for the bridge (bridge/package.json, root package.json). The architecture therefore has two consumer surfaces: direct Python CLI/library use and indirect JS/TS invocation via the bridge.
Main entry points: component_extractor.py Component model · networkx_graph.py Graph build · tests Tests · bridge.ts Bridge API
Components 23
Python engine
Static analysis core, CLI entry points, graph construction, and diff workflows.
- CLI · Top-level command for analyzing a repository.
main.py - Module CLI · Internal Python entry point behind the root script.
codetraverse/main.py - Path CLI · CLI for querying the generated dependency graph.
codetraverse/path.py - Component model · Language-agnostic component extraction contract.
codetraverse/base/component_extractor.py - Registry · Maps file types and languages to extractor implementations.
codetraverse/registry/extractor_registry.py - Graph build · Builds the repository-wide dependency graph using NetworkX.
codetraverse/utils/networkx_graph.py - Partitioning · Splits or organizes the constructed graph for downstream use.
codetraverse/utils/graph_partitioner.py - Graph query · Supports neighborhood and path-style queries over the graph.
codetraverse/utils/blackbox.py - Diff orchestration · Coordinates AST-level comparison workflows across languages.
codetraverse/utils/AstDifferOrchestrator.py - Base diff · Shared diff primitives used by language-specific change detectors.
codetraverse/ast_diff/basefilediff.py
Language support
Per-language extractors and adapters that normalize source code into the shared model.
- TypeScript · TypeScript structural extraction front end.
codetraverse/extractors/typescript_extractor.py - Python · Python structural extraction front end.
codetraverse/extractors/python_extractor.py - Go · Go structural extraction front end.
codetraverse/extractors/go_extractor.py - Haskell · Haskell structural extraction front end.
codetraverse/extractors/haskell_extractor.py - Rust · Rust structural extraction front end.
codetraverse/extractors/rust_extractor.py - ReScript · ReScript structural extraction front end.
codetraverse/extractors/rescript_extractor.py
TS bridge
Node/TypeScript wrapper that launches the Python engine for JS consumers.
- Bridge API · TypeScript facade exposed to JavaScript consumers.
bridge/src/bridge.ts - Python runner · Launches the Python analysis engine from Node.
bridge/src/python-runner.ts - Bridge entry · Bridge package entry point and export surface.
bridge/src/index.ts
Validation
Sample repositories and tests used to verify extraction, graphing, and CLI behavior.
- Sample repos · Miniature language repositories used for extraction validation.
sample_code_repo_test - Tests · Repository-wide test suite covering extractors, adapters, graph utilities, and CLIs.
tests
Packaging
Distribution and installation boundaries for Python and Node consumers.
- Python package · Python packaging and install metadata for the analysis engine.
pyproject.toml - Node package · Node packaging metadata for the bridge.
bridge/package.json
Connections 25
- CLI → Module CLI: delegates to · The root CLI routes into the package implementation.
- Module CLI → Registry: selects · Analysis starts by choosing an extractor for each source file.
- Registry → Component model: implements · Concrete extractors conform to the shared extraction contract.
- Component model → TypeScript: specializes · TypeScript analysis is one implementation of the common model.
- Component model → Python: specializes · Python analysis is one implementation of the common model.
- Component model → Go: specializes · Go analysis is one implementation of the common model.
- Component model → Haskell: specializes · Haskell analysis is one implementation of the common model.
- Component model → Rust: specializes · Rust analysis is one implementation of the common model.
- Component model → ReScript: specializes · ReScript analysis is one implementation of the common model.
- TypeScript → Graph build: feeds · Extracted components are assembled into the repo graph.
- Python → Graph build: feeds · Extracted components are assembled into the repo graph.
- Go → Graph build: feeds · Extracted components are assembled into the repo graph.
- Haskell → Graph build: feeds · Extracted components are assembled into the repo graph.
- Rust → Graph build: feeds · Extracted components are assembled into the repo graph.
- ReScript → Graph build: feeds · Extracted components are assembled into the repo graph.
- Graph build → Partitioning: organizes · Graph utilities can partition or reshape the dependency graph.
- Path CLI → Graph query: queries · Path-oriented queries use the graph query utility.
- Diff orchestration → Base diff: uses · Language-specific diff workflows share base diff logic.
- Bridge API → Python runner: invokes · The bridge launches Python analysis as a subprocess.
- Bridge entry → Bridge API: exports · Package entry points surface the bridge API.
- Sample repos → Tests: validates · Fixture repositories support cross-language behavior checks.
- Tests → Component model: covers · Tests exercise the extraction abstraction and implementations.
- Tests → Graph build: covers · Tests exercise graph assembly and related utilities.
- Python package → CLI: packages · Python packaging exposes the CLI and library entry points.
- Node package → Bridge API: packages · Node packaging exposes the bridge for JS/TS consumers.