How arman0604/finops works
1. This repository is a FinOps analytics product split into two major halves: a Python analytics backend that ingests and transforms cloud billing data, and a React + TypeScript frontend that presents command-center style views in the browser.
2. The data domain is centered on cloud cost sources for AWS, Azure, and GCP. The raw inputs live under data/raw/ as CSVs plus anomaly labels, while cleaned/combined outputs live under data/processed/. This separation suggests an offline batch pipeline before anything is visualized.
3. The main Python processing chain is decomposed by responsibility:
- data_generator.py produces synthetic or test billing datasets.
- normalizer.py aligns heterogeneous billing formats into a unified schema.
- anomaly_detector.py and stream_detector.py handle batch and near-real-time anomaly detection.
- forecasting_engine.py generates spend predictions.
- storage.py persists results to SQLite.
These modules are the core analytics engine and likely feed the UI through API endpoints or generated artifacts.
4. Persistence is lightweight and local-first. There is a SQLite database at both finops.db and data/database/finops.db, which implies the app can operate without external infrastructure. The database is probably used to store normalized spend, anomaly results, and forecast outputs.
5. The application exposes a backend interface via api.py, which likely serves processed analytics to the frontend. chatbot.py and dashboard_app.py suggest additional backend entry points or service wrappers for conversational assistance and dashboard serving, rather than a single monolithic server.
6. The frontend is a Vite-based React app in src/. It uses TypeScript, with a clear page-level structure:
- CommandCenter for executive overview
- AnomalyWatch for anomaly monitoring
- SpendForecasting for forecast views
- DataUpload for ingestion workflows
Shared layout components live under src/components/Layout/, and visualization-specific UI lives under src/components/VisualizationSection/.
7. Frontend data access is abstracted through src/data/api.ts, which likely centralizes calls to the Python backend, while src/data/mockData.ts provides local fallback or development data. That implies the UI can run in a partially disconnected mode during development.
8. There is a generated dashboard artifact: dashboard_generator.py and finops_dashboard.html, plus a copy in static/finops_dashboard.html. This indicates a second delivery path besides the React SPA: a prebuilt HTML dashboard that can be served or opened directly, likely containing the same analytics in a static format.
9. The repo also contains an implementation_plan.md.resolved, which suggests the project was developed from a planned multi-step architecture with staged implementation. The presence of test_finops_dataset.csv reinforces that the project is designed around repeatable local testing with bundled data rather than live cloud integrations.
10. Overall architecture flow: raw cloud billing CSVs enter the Python pipeline, are normalized and stored, then analyzed for anomalies and forecasts. Those results are exposed either through the Python API to the React dashboard or through a generated static HTML dashboard. The browser is the primary interaction layer, while Python owns the data engineering and analytics logic.
Main entry points: api.ts API client · processed Processed data · api.py API · Layout.tsx Layout shell
Components 22
Python analytics
Batch analytics, API, and dashboard generation
- Generator · Produces synthetic or test billing datasets
data_generator.py - Normalizer · Converts heterogeneous billing formats into a unified schema
normalizer.py - Batch detect · Detects anomalies from processed billing data
anomaly_detector.py - Stream detect · Handles streaming or incremental anomaly detection
stream_detector.py - Forecasting · Generates spend forecasts from historical data
forecasting_engine.py - Storage · Persists normalized, anomaly, and forecast outputs
storage.py - API · Serves analytics data to the frontend and other clients
api.py - Chatbot · Conversational wrapper around FinOps analytics
chatbot.py - Dashboard app · Backend entry point for dashboard serving
dashboard_app.py - Dashboard gen · Generates the standalone HTML dashboard artifact
dashboard_generator.py
React app
Browser UI and shared components
- Command Center · Executive summary and command-center style dashboard
src/pages/CommandCenter.tsx - Anomaly Watch · Views anomalies and operational alerts
src/pages/AnomalyWatch.tsx - Spend Forecasting · Displays forecast trends and predictions
src/pages/SpendForecasting.tsx - Data Upload · Supports upload-driven ingestion workflows
src/pages/DataUpload.tsx - API client · Centralized frontend calls to backend services
src/data/api.ts - Mock data · Local fallback datasets for development and disconnected use
src/data/mockData.ts - Layout shell · Header, sidebar, and app layout structure
src/components/Layout/Layout.tsx - Viz section · Reusable visualization section for page-level charts
src/components/VisualizationSection/VisualizationSection.tsx
Data assets
Raw inputs, processed outputs, and local persistence
- Raw billing · AWS, Azure, and GCP billing sources plus anomaly labels
data/raw - Processed data · Unified and derived billing outputs used by analytics and UI
data/processed - SQLite store · Local FinOps database for normalized spend and analysis results
finops.db - Static dashboard · Prebuilt dashboard for direct serving or offline use
static/finops_dashboard.html
Connections 29
- Raw billing → Generator: seed data · Synthetic and test data can augment or replace raw inputs
- Raw billing → Normalizer: ingest · Raw cloud billing files are normalized into a unified schema
- Generator → Normalizer: sample input · Generated datasets follow the same normalization path
- Normalizer → Processed data: write cleaned · Unified billing data is materialized as processed outputs
- Processed data → Batch detect: analyze · Batch anomaly detection runs on cleaned billing data
- Processed data → Forecasting: train/forecast · Historical spend drives predictions
- Processed data → Stream detect: incremental feed · Processed data can also support streaming-style checks
- Batch detect → Storage: persist findings · Anomaly results are saved for later retrieval
- Stream detect → Storage: persist events · Streaming detections are stored locally
- Forecasting → Storage: persist forecasts · Forecast outputs are written to the local store
- Storage → SQLite store: store · Results are persisted in SQLite
- API → SQLite store: read/write · Backend interface serves and updates analytics data
- API → Processed data: serve data · API may also expose processed artifacts directly
- Chatbot → API: query · Conversational layer consumes backend analytics endpoints
- Dashboard app → API: serve through · Dashboard server wrapper exposes backend data
- Dashboard gen → Static dashboard: build artifact · Generates the standalone HTML dashboard
- API client → API: fetch · Frontend data access layer calls the Python API
- API client → Mock data: fallback · Local mock data supports development and offline mode
- Command Center → Viz section: compose · Primary overview page uses shared visualization UI
- Anomaly Watch → Viz section: compose · Monitoring page reuses shared charting sections
- Spend Forecasting → Viz section: compose · Forecast page reuses visualization components
- Data Upload → API client: submit · Upload workflows send data through the frontend data layer
- Layout shell → Command Center: wraps · Shared layout hosts the main pages
- Layout shell → Anomaly Watch: wraps · Shared layout hosts the main pages
- Layout shell → Spend Forecasting: wraps · Shared layout hosts the main pages
- Layout shell → Data Upload: wraps · Shared layout hosts the main pages
- API client → Command Center: feeds · Command Center is data-driven from the backend client
- API client → Anomaly Watch: feeds · Anomaly views consume backend data
- API client → Spend Forecasting: feeds · Forecast views consume backend data