How awcodify/ragnarock works
Project’s Purpose and Type
• This project is a full-stack analytical service that performs real-time infrastructure analysis using a Retrieval Augmented Generation (RAG) approach.
• It collects Prometheus metrics, performs vector similarity search to find historical patterns, leverages an analysis service that interacts with an LLM via TensorZero, and finally exposes an API using FastAPI to deliver structured JSON responses.
Main Components
• Data Collection:
• The file src/data/prometheus_client.py likely implements the logic of fetching metrics.
• RAG Service and Data Processing:
• Components where metrics are processed include:
• Collect Metrics (collector logic)
• Vector Store (implemented in src/data/vector_store.py, potentially with Qdrant integration)
• Analysis (core logic in src/services/analysis.py)
• Interaction with TensorZero (located in the tensorzero/ directory) which acts as a gateway to various LLM providers.
• LLM Providers:
• API Layer:
• The REST API built with FastAPI (in src/api/main.py) exposes endpoints like /analyze.
• This layer handles incoming HTTP requests and returns JSON responses based on the analysis.
Architectural Patterns and Technologies
• Separation of Concerns: The clear split between data collection (Prometheus), processing components (vector store search and analysis), and API delivery.
• Microservice Elements: Although this is a single project, there’s modularity between services such as analysis, health check (src/services/health.py), and LLM interactions.
• Async Operations: FastAPI is async-friendly.
• Integration with External Tools: Docker, docker-compose, and Prometheus; plus the integration with tensorzero for LLM provider abstraction.
a. A high-level layout:
b. Main Components to Include:
• Prometheus (Data Source)
• Data Collection Module (fetching metrics)
• Historical Data Search module (searching in vector store)
• Vector Store (e.g., Qdrant, as inferred by tests and service naming)
• Analysis Service (computing similar patterns and producing analysis)
• TensorZero Gateway (to interface with the LLM providers)
• LLM Providers (Claude, Ollama, and possibly others)
• API endpoints served by FastAPI (for the REST API)
• Flow from "Collect Metrics" to "Search Historical".
• Then a loop between "Search Historical" and "Vector Store" (for storing new metrics).
• From "Vector Store" to "Analysis" which then leads into the "TensorZero Gateway".
• From TensorZero, data flows to the various "LLM Providers", whose outputs feed back into the analysis.
• Finally, the aggregated processed data is exposed via the "API Endpoint" to the client.
• Use clear labels for each component.
• Rectangles for modules (e.g., Data Collector, Analysis Service)