How picnicsupermarket/dbt-score works
Project type and purpose
• This is a Python-based, open-source command-line tool (“dbt-score”) whose purpose is to lint and score dbt (Data Build Tool) metadata. It reads a dbt manifest JSON, applies a set of user-configurable rules, computes scores, and outputs results in various formats.
File structure
• Top-level directories:
• src/dbt_score: the main package
• docs: documentation (MkDocs)
• tests: pytest test suite
• .github/workflows: CI/CD pipelines
• Key modules under src/dbt_score:
• __main__.py & cli.py — CLI entrypoint (likely built with click/argparse)
• config.py — loading and validating user configuration
• dbt_utils.py — helpers to parse the dbt manifest
• rule_registry.py, rule_catalog.py, rule_filter.py — extensible rule discovery & filtering
• evaluation.py & scoring.py — core logic to apply rules and compute scores
• formatters/… — pluggable output formatters (JSON, human-readable, manifest-annotated, ASCII)
• exceptions.py — custom error types
• Build & packaging: pyproject.toml, pdm.lock, .pre-commit, .prettierrc
• CI workflows define lint, test, release steps.
Insights from README
• Describes dbt-score as a “linter for dbt metadata”
• Documents programmatic invocation, CLI flags, configuration model, rule definition syntax, and output formats.
• No external services—operates entirely on local files (dbt manifest).
a. Main components
• CLI Layer
• Configuration Loader
• Manifest Parser (dbt_utils)
• Rule Registry & Catalog
• Rule Filter
• Evaluation Engine
• Scoring Engine
• Formatter Plugins
• Output Sink (stdout, file)
• Tests & CI (side component)
b. Relationships & data flow
• User invokes CLI →
• CLI parses arguments →
• Config Loader reads config file (+ CLI overrides) →
• Manifest Parser loads and normalizes dbt manifest JSON →
• Rule Registry dynamically discovers available rules →
• Rule Filter selects active rules based on config →
• Evaluation Engine applies each rule to manifest models →
• Scoring Engine aggregates rule results into a score →
• Formatter Plugin transforms results into chosen format →
• CLI writes to stdout or files
c. Architectural patterns / principles
• Modular separation of concerns (parsing, rules, evaluation, formatting)
• Plugin-style extensibility for rules and formatters
• Single responsibility per module
• Clear error handling via custom exceptions
d. Key technologies
• Python 3.x (packaged with PDM)
• pytest for testing
• MkDocs for documentation
• CI via GitHub Actions
• CLI framework (e.g., click or built-in argparse)
Detail
• Under “Formatter,” show individual formatter plugins (JSON, human-readable, ASCII, manifest).