How gridhead/metasource works
Main components
• CLI/Entry Point: At the very top, you have main.go which initializes the service and interprets command line arguments. This component directs the service into one of its two operational modes (“database” for downloading metadata and “dispense” for serving HTTP requests).
• Configuration Module: Under metasource/config, the base configuration settings are maintained. Represent this as a “Configuration Manager” that flows into several functional areas.
• Drivers: Encapsulated in metasource/driver, these files (e.g., cont.go, fish.go, hash.go, pull.go, etc.) implement the core logic required for processing and interacting with the RPM repository metadata. This is arguably the “engine” of the application.
• Lookup: The metasource/lookup directory contains tools for navigating and indexing repository components (branches, sources, relationships, etc.). This can be represented as a “Metadata Lookup Service” that supports the driver and reader functionality.
• Models: Within metasource/models, there are several directories (dict, home, rels, sxml) which define the data structures and relationships for the metadata. Treat this part as the “Domain Model Layer” that represents the structured data.
• Reader: In metasource/reader, the files here handle reading files or other relevant inputs. Represent this as the “Input/ Data Parsing Module” that feeds data into the processing pipeline.
• Routes: Located in metasource/routes, this folder contains the HTTP and HTML endpoint definitions (including a template for "home.html") that expose the processed data to external clients. This is your “HTTP API Layer” or “Web Service Frontend.”
Relationships and interactions
• The main CLI entry point directs execution into either the “database” mode (invoking the driver, lookup, reader, and model layers to fetch and process metadata) or into the “dispense” mode (where the routes come into play to serve HTTP endpoints).
• The Configuration Manager provides the necessary settings (like database location, log level, etc.) to both the driver component and the HTTP service.
• The drivers use both the lookup and reader modules to retrieve and validate repository metadata, then store or update the domain models accordingly.
• The HTTP Routes layer interacts with the domain models to respond to client queries (via GET endpoints as illustrated by the example curl requests in the README).
Important architectural patterns and design principles
• Modular Design: The project organizes business logic, configuration, data models, and presentation (routes) into separate packages.
• Separation of Concerns: There is a clear split between data acquisition duties (database subcommand) and data dissemination (dispense subcommand and HTTP server), making it easier to maintain and scale.
• Extensibility: With distinct layers (Drivers, Lookup, Reader, and Models), it’s easy to add new features such as additional metadata sources or alternative processing techniques.
Relevant technologies and libraries
• Go (Golang) is the primary programming language.
• External dependencies such as createrepo_c-libs and Git are required for fetching and processing metadata.
• The use of cron for scheduling periodic database downloads should be annotated as an external scheduling mechanism that triggers the “database” fetch process.
• From main.go (CLI) to both the “Database” process and the “Dispense” process.
• Data flows between Drivers, Lookup, Reader, and Models.
• The HTTP API (Routes) component should show interactions with external clients.
• Core business logic (Core drivers, lookup, reader, models) and
• Interface layers (CLI and HTTP endpoints).