How growgraph/ethexco works
Project type and purpose
• This repository, “EthExCo,” is a Python-based data-processing and model-training pipeline (open-source tool) that ingests raw ethics-related data, breaks it into “frames,” augments and labels those frames, assembles a dataset, and finally trains a model (including a “modal” variant).
• It is not a front-end/back-end web app, but rather a command-line-driven ETL + ML pipeline library with reusable components under ethexco/ and orchestration scripts in run/.
File structure
• Top-level config and metadata:
• .gitignore, .pylintrc, mypy.ini, pyproject.toml
• Pre-commit and virtual-env setup files
• Core library: ethexco/
• frame.py (frame-creation logic)
• util.py (shared utilities)
• Orchestration scripts: run/
• chunk_raw_data.py: splits raw inputs
• generate_frames.py: applies frame logic
• frames_add_text.py: annotates frames with text
• generate_dataset.py: bundles frames into dataset
• train.py & train_modal.py: kick off model training
• testing/: unit tests for pipelines and frames
• Lock file: uv.lock suggests use of the “uv” CLI for environment management.
README for additional insights
• Workflow steps (chunk → frames → dataset → train)
• Dev setup uses “uv” to create and sync virtual environments
• No external services or web-APIs, purely file-based I/O and local compute
Elements
• Clear, self-explanatory labels for each pipeline stage and module
• Blue rectangles for Python modules (ethexco/)
• Green rounded rectangles for orchestration scripts (run/)
• Orange parallelograms for environment/tooling (venv, uv)
Detailed capture of architecture
• Do not oversimplify: represent each script as its own node
• Interactions between scripts and library functions