How glato/emerge works
1. What this repo is
- Emerge is a Python 3 command-line analysis engine that scans a source tree, extracts code structure, computes metrics, and exports graph/data artifacts.
- The output is both machine-readable files and a bundled interactive HTML visualization.
2. Primary entrypoints
- emerge.py is the standalone CLI launcher.
- emerge/main.py and emerge/core.py appear to hold the top-level orchestration for parsing CLI flags, loading YAML config, and running analyses.
3. Configuration boundary
- emerge/config.py and emerge/configs/*.yaml define the analysis contract.
- Configuration drives source root, language/file filters, ignored dependencies/entities, selected metrics, export targets, and app visualization settings.
- This is the main boundary between “what to analyze” and the analysis engine itself.
4. Analysis pipeline
- The pipeline is: load config → traverse filesystem → filter files → parse source files → build graph/result objects → compute metrics/stats → export outputs.
- emerge/files.py likely handles recursive file discovery and path-based filtering.
- emerge/analysis.py, emerge/analyzer.py, emerge/core.py, emerge/results.py, and emerge/stats.py are the orchestration/result layers around that pipeline.
5. Language parsing subsystem
- emerge/languages/ contains one parser per supported language, all built on abstractparser.py.
- Supported parsers include C, C++, Go, Groovy, Java, JavaScript, Kotlin, Objective-C, Python, Ruby, Swift, and TypeScript.
- These parsers are responsible for recognizing files and, where supported, extracting finer-grained entities such as classes or declarative UI constructs.
6. Graph model and code relationships
- emerge/graph.py is the central model for dependency-style structures.
- The repo supports filesystem graphs, file dependency graphs, entity dependency graphs, inheritance graphs, and a “complete” graph that combines dependency and inheritance edges.
- Graph construction is the bridge from parsed source data to visual analysis and metric computation.
7. Metrics subsystem
- emerge/metrics/ is split into isolated metric modules behind abstractmetric.py.
- Implemented metrics include SLOC, whitespace complexity, number of methods, fan-in/fan-out, modularity, TF-IDF semantic keywords, and experimental git-based metrics.
- metrics/git/ indicates optional mining from a Git repository, separate from the source scan itself.
8. Analysis outputs and exports
- emerge/export.py is the output boundary for GraphML, JSON, tabular console/file output, and the D3-oriented data bundle.
- The export layer takes computed graphs and metrics and materializes them into the chosen formats.
- emerge/abstractresult.py and emerge/results.py suggest a structured result object model used across exporters.
9. Interactive HTML application
- emerge/output/html/ is a prebuilt static web app packaged with the tool.
- It uses Bootstrap for UI, D3 for force-directed layout, simpleheat for heatmaps, hull for cluster hulls, and dark-mode-switch for theme control.
- The generated data files feed this app so users can inspect clusters, node relationships, semantic search, and metrics interactively in a browser.
10. Testing and packaging
- emerge/tests/ contains parser and metric tests plus test data per language, showing that parser output is a key contract.
- setup.py, requirements.txt, MANIFEST.in, and Dockerfile indicate the project is packaged for pip and container use.
- The repo is structured as a library-plus-CLI tool, with the UI assets bundled rather than served by a separate backend.
Main entry points: metrics.py Metrics Core · graph.py Graph Model · analyzer.py Analyzer · abstractparser.py Parser Base
Components 21
CLI & Orchestration
Entry points and top-level control flow.
Config & Scan
Configuration boundary and filesystem discovery.
Language Parsing
Per-language source parsers built on a shared parser base.
- Parser Base · Shared parsing contract for all language implementations.
emerge/languages/abstractparser.py - Python Parser · Representative parser for Python source extraction.
emerge/languages/pyparser.py - Java Parser · Representative parser for JVM-style class extraction.
emerge/languages/javaparser.py - C/C++ Parser · Representative parser for native-language structure extraction.
emerge/languages/cppparser.py
Graph & Results
Core analysis model, results, and statistics.
- Graph Model · Central model for filesystem, dependency, inheritance, and combined graphs.
emerge/graph.py - Results · Structured result objects shared across exporters and consumers.
emerge/results.py - Result Base · Common result abstraction used by the output layer.
emerge/abstractresult.py - Stats · Aggregates derived counts and summary statistics.
emerge/stats.py
Metrics
Metric plug-ins and their categories.
- Metrics Core · Coordinates metric modules and metric execution.
emerge/metrics/metrics.py - SLOC · Source-line counting metric.
emerge/metrics/sloc/sloc.py - Fan In/Out · Dependency connectivity metric.
emerge/metrics/faninout/faninout.py - TF-IDF · Semantic keyword extraction metric.
emerge/metrics/tfidf/tfidf.py
Connections 23
- Launcher → Main CLI: starts · The standalone script invokes the main CLI flow.
- Main CLI → Config: loads · CLI parses runtime options and resolves configuration.
- Main CLI → Core Flow: runs · Main hands control to the orchestration layer.
- Core Flow → Analyzer: delegates · Core execution passes into the analysis engine.
- Analyzer → File Scan: scans · Analysis begins by discovering candidate source files.
- Config → Templates: uses · Configuration is seeded by YAML presets and templates.
- File Scan → Parser Base: selects · File types are matched to parser implementations.
- Parser Base → Python Parser: implements · Representative language parser implementation.
- Parser Base → Java Parser: implements · Representative language parser implementation.
- Parser Base → C/C++ Parser: implements · Representative language parser implementation.
- Python Parser → Graph Model: emits · Parsed entities and dependencies are added to graph structures.
- Java Parser → Graph Model: emits · Parsed entities and dependencies are added to graph structures.
- C/C++ Parser → Graph Model: emits · Parsed entities and dependencies are added to graph structures.
- Graph Model → Stats: summarizes · Graph content feeds derived statistics and summaries.
- Graph Model → Results: packages · Graph state is wrapped in result objects.
- Results → Result Base: conforms to · Result objects follow a shared abstraction.
- Analyzer → Metrics Core: computes · The analysis engine runs selected metrics over parsed data.
- Metrics Core → SLOC: includes · Representative size metric module.
- Metrics Core → Fan In/Out: includes · Representative dependency metric module.
- Metrics Core → TF-IDF: includes · Representative semantic metric module.
- Metrics Core → Stats: feeds · Metric outputs contribute to overall statistics.
- Analyzer → Export: exports · Final analysis artifacts are materialized by the exporter.
- Export → HTML App: bundles · Export data feeds the packaged interactive HTML application.