# vishwasaichalla/social_media_analytics_dashboard-full_stack_data_visualization_platform architecture

> Architecture diagram of the GitHub repository vishwasaichalla/social_media_analytics_dashboard-full_stack_data_visualization_platform, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vishwasaichalla/social_media_analytics_dashboard-full_stack_data_visualization_platform
- Repository: https://github.com/vishwasaichalla/social_media_analytics_dashboard-full_stack_data_visualization_platform

## Overview

##### Project Type & Purpose
- This is a full-stack social media analytics dashboard.
- Backend: Flask-based REST API, CSV ingestion, data transformation, MongoDB storage, APScheduler for mock data.
- Frontend: Streamlit dashboard with Plotly Express visualizations.
- Deployed via Docker Compose, optionally on AWS EC2.

##### File Structure Analysis
- Top-level:
- Dockerfile & docker-compose.yml: container definitions.
- backend/: Flask app (app.py), ingestion (data_ingest.py), storage (data_store.py), transformation (transformation.py), sample CSV.
- frontend/: Streamlit script (visualization.py).
- Documentation: PowerBI.md, data-ingestion.md, datastore-setup.md, visualization.md.
- CI: .github/workflows/super-linter.yml.
- requirements.txt for Python deps.
- Patterns:
- Client-server separation (frontend vs backend).
- ETL pipeline: data ingestion → transformation → storage.
- Scheduler for periodic data generation.
- Container orchestration via Docker Compose.

##### Key Insights from README
- Architecture section clearly defines components and interactions.
- Specifies technologies: Flask, MongoDB, APScheduler, Streamlit, Plotly.
- Data flow: initial ingestion, real-time API calls, visualization updates.

- Data Sources:
- Social_Media_Engagement.csv (initial load)
- Mock data generator (APScheduler)
- Backend Services:
- Data Ingest Service (data_ingest.py)
- Transformation Layer (transformation.py)
- Data Store (MongoDB)
- Flask API (app.py)
- Frontend:
- Streamlit Dashboard (visualization.py)
- Orchestration & Deployment:
- Docker Compose
- CI Pipeline (GitHub Actions)
- External / Infrastructure:
- AWS EC2 (optional)
- Elastic IP, Security Groups

1. CSV File → Data Ingest Service → Transformation Layer → MongoDB.
2. Scheduler → Data Ingest Service (every 30s) → Transformation → MongoDB.
3. Streamlit Frontend → REST API Calls → Flask API → MongoDB → API Response → Frontend.
4. Docker Compose orchestrates containers for MongoDB, backend, frontend.
5. CI Pipeline → Linter on code push.

c. Architectural Patterns & Principles:
- Layered Architecture: presentation (frontend), application (Flask), data (MongoDB).
- ETL Pipeline: ingestion → transformation → storage.
- Client-Server Model: Streamlit client consumption of REST API.
- Modularization: separate Python modules for ingest, transform, storage.

d. Key Technologies & Libraries:
- Python 3.8+, Flask, PyMongo
- APScheduler for background tasks
- Streamlit, Plotly Express
- Docker, Docker Compose
- GitHub Actions (super-linter)
- (Optional) AWS EC2 for production

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Sources
    subgraph "Data Sources"
        DS_CSV["Social_Media_Engagement.csv"]:::dataFlow
    end

    %% Background Tasks
    Scheduler["APScheduler Task"]:::scheduler

    %% Backend Code Modules inside Flask Container
    subgraph "Docker Compose"
        subgraph "Flask Container"
            Ingest["Data Ingest Service\ndata_ingest.py"]:::dataFlow
            Transform["Transformation Layer\ntransformation.py"]:::dataFlow
            StoreModule["Data Store Module\ndata_store.py"]:::dataFlow
            API["Flask REST API\napp.py"]:::api
        end
        MongoDB["MongoDB"]:::dataFlow
        Frontend["Streamlit Dashboard\nvisualization.py"]:::frontend
    end

    %% DevOps & CI
    subgraph "DevOps"
        CI["GitHub Actions\n(super-linter)"]:::devops
    end

    %% Deployment & Orchestration
    subgraph "Deployment & Orchestration"
        DockerfileNode["Dockerfile"]:::infra
        ComposeFile["docker-compose.yml"]:::infra
        DockerComposeIcon["Docker Compose"]:::infra
    end

    %% Documentation
    subgraph "Documentation"
        DataIngestDoc["data-ingestion.md"]:::docs
        DatastoreDoc["datastore-setup.md"]:::docs
        VisualizationDoc["visualization.md"]:::docs
        PowerBIDoc["PowerBI.md"]:::docs
    end

    %% Infrastructure
    AWS["AWS EC2\nElastic IP & Security Groups"]:::infra

    %% Data Flows
    DS_CSV -->|"load CSV"| Ingest
    Scheduler -->|"trigger every 30s"| Ingest
    Ingest -->|"clean & transform"| Transform
    Transform -->|"store data"| StoreModule
    StoreModule -->|"persist"| MongoDB

    %% API Calls
    Frontend -->|"REST API calls"| API
    API -->|"query"| MongoDB
    MongoDB -->|"response"| API
    API -->|"JSON response"| Frontend

    %% DevOps Relationships
    CI -->|lints code on push| Ingest
    CI -->|lints code on push| Transform
    CI -->|lints code on push| API
    CI -->|lints code on push| Frontend

    %% Deployment Relationships
    DockerfileNode -->|builds image| DockerComposeIcon
    ComposeFile -->|defines services| DockerComposeIcon
    DockerComposeIcon -->|orchestrates| MongoDB
    DockerComposeIcon -->|orchestrates| API
    DockerComposeIcon -->|orchestrates| Frontend

    %% Infrastructure Relationships
    AWS -.->|hosts| DockerComposeIcon

    %% Documentation Links
    DataIngestDoc -->|describes| Ingest
    DatastoreDoc -->|describes| StoreModule
    VisualizationDoc -->|describes| Frontend
    PowerBIDoc -->|additional dashboard docs| VisualizationDoc

    %% Click Events
    click DS_CSV "https://github.com/vishwasaichalla/social_media_analytics_dashboard-full_stack_data_visualization_platform/blob/main/backend/Social_Media_Engagement.csv"
    click Ingest "https://github.com/vishwasaichalla/social_media_analytics_dashboard-full_stack_data_visualization_platform/blob/main/backend/data_ingest.py"
    click Transform "https://github.com/vishwasaichalla/social_media_analytics_dashboard-full_stack_data_visualization_platform/blob/main/backend/transformation.py"
    click StoreModule "https://github.com/vishwasaichalla/social_media_analytics_dashboard-full_stack_data_visualization_platform/blob/main/backend/data_store.py"
    click API "https://github.com/vishwasaichalla/social_media_analytics_dashboard-full_stack_data_visualization_platform/blob/main/backend/app.py"
    click Frontend "https://github.com/vishwasaichalla/social_media_analytics_dashboard-full_stack_data_visualization_platform/blob/main/frontend/visualization.py"
    click DockerfileNode "https://github.com/vishwasaichalla/social_media_analytics_dashboard-full_stack_data_visualization_platform/tree/main/Dockerfile"
    click ComposeFile "https://github.com/vishwasaichalla/social_media_analytics_dashboard-full_stack_data_visualization_platform/blob/main/docker-compose.yml"
    click CI "https://github.com/vishwasaichalla/social_media_analytics_dashboard-full_stack_data_visualization_platform/blob/main/.github/workflows/super-linter.yml"
    click DataIngestDoc "https://github.com/vishwasaichalla/social_media_analytics_dashboard-full_stack_data_visualization_platform/blob/main/data-ingestion.md"
    click DatastoreDoc "https://github.com/vishwasaichalla/social_media_analytics_dashboard-full_stack_data_visualization_platform/blob/main/datastore-setup.md"
    click VisualizationDoc "https://github.com/vishwasaichalla/social_media_analytics_dashboard-full_stack_data_visualization_platform/blob/main/visualization.md"
    click PowerBIDoc "https://github.com/vishwasaichalla/social_media_analytics_dashboard-full_stack_data_visualization_platform/blob/main/PowerBI.md"

    %% Styles
    classDef dataFlow fill:#ADE1F9,stroke:#333,stroke-width:1px
    classDef api fill:#A4F9A1,stroke:#333,stroke-width:1px
    classDef scheduler fill:#F9D89A,stroke:#333,stroke-width:1px
    classDef infra fill:#D3D3D3,stroke:#333,stroke-width:1px
    classDef docs fill:#E3D7FF,stroke:#333,stroke-width:1px
    classDef devops fill:#FFF4A3,stroke:#333,stroke-width:1px
    classDef frontend fill:#C1FFC1,stroke:#333,stroke-width:1px
```
