# mitodl/mit-learn architecture

> Architecture diagram of the GitHub repository mitodl/mit-learn, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mitodl/mit-learn
- Repository: https://github.com/mitodl/mit-learn

## Overview

##### Main Components

a. Frontend Layer:
- Next.js application (in /frontends/main)
- Component library (in /frontends/ol-components)
- API client layer (in /frontends/api)
- Storybook for component documentation

b. Backend Layer:
- Django REST Framework application
- Multiple Django apps for different features:
- learning_resources
- authentication
- profiles
- channels
- news_events
- vector_search
- ai_chat

c. Data Storage:
- PostgreSQL database
- OpenSearch for search functionality
- S3 for media storage
- Redis for caching

d. External Services:
- Keycloak for OpenID Connect authentication
- Mailgun for email services
- PostHog for analytics and feature flags
- Various MIT learning platforms (xpro, OCW, etc.)

##### Integration Points

- Frontend to Backend API communication
- Backend to Database interactions
- Authentication flow with Keycloak
- ETL pipelines from external learning platforms
- Search queries to OpenSearch
- File uploads to S3
- Email sending via Mailgun
- Analytics data to PostHog

##### Key Architectural Patterns

Highlight:
- Microservices-like structure with distinct Django apps
- ETL pipeline architecture for data ingestion
- REST API design with OpenAPI specification
- Component-based frontend architecture
- Event-driven architecture for certain features
- Caching strategy with Redis

##### Specific Implementation Details

- Celery task queue for background processing
- Docker containerization structure
- OpenSearch cluster configuration
- Frontend state management
- API versioning (v0 and v1)

##### Development Environment

Show:
- Docker Compose setup
- Local development services
- Codespaces configuration
- Testing infrastructure

**┌─────────────────────────────────────────────────────────────┐**
- Client Layer                           │
- (Next.js, React Components, API Client, Frontend Routing)  │
- ┤
- API Gateway                            │
- (Django REST Framework, OpenAPI, Authentication)           │
- ┤
- Service Layer                            │
- (Django Apps, Business Logic, ETL Pipelines)              │
- ┤
- Data Layer                               │
- (PostgreSQL, OpenSearch, Redis, S3)                       │
- ┤
- External Services                           │
- (Keycloak, Mailgun, PostHog, MIT Learning Platforms)      │
- ┘

- Blue for frontend components
- Green for backend services
- Yellow for databases and storage
- Purple for external services
- Red for security components
- Gray for development/infrastructure components

- Environment boundaries

## Diagram (Mermaid)

```mermaid
graph TB
    %% Styles and classes
    classDef frontend fill:#3498db,stroke:#2980b9,color:white
    classDef backend fill:#2ecc71,stroke:#27ae60,color:white
    classDef database fill:#f1c40f,stroke:#f39c12,color:black
    classDef external fill:#9b59b6,stroke:#8e44ad,color:white
    classDef security fill:#e74c3c,stroke:#c0392b,color:white
    classDef infra fill:#95a5a6,stroke:#7f8c8d,color:white

    %% Frontend Layer
    subgraph Frontend
        NextApp["Next.js Application"]:::frontend
        CompLib["Component Library"]:::frontend
        APIClient["API Client Layer"]:::frontend
        Storybook["Storybook"]:::frontend
    end

    %% Backend Layer
    subgraph Backend
        DRF["Django REST Framework"]:::backend
        
        subgraph DjangoApps["Django Apps"]
            LR["Learning Resources"]:::backend
            Auth["Authentication"]:::backend
            Profiles["Profiles"]:::backend
            Channels["Channels"]:::backend
            NewsEvents["News & Events"]:::backend
            VectorSearch["Vector Search"]:::backend
            AIChat["AI Chat"]:::backend
        end
        
        Celery["Celery Tasks"]:::backend
    end

    %% Data Layer
    subgraph DataStorage
        Postgres[(PostgreSQL)]:::database
        OpenSearch[(OpenSearch)]:::database
        S3[(S3 Storage)]:::database
        Redis[(Redis Cache)]:::database
    end

    %% External Services
    subgraph External
        Keycloak["Keycloak SSO"]:::external
        Mailgun["Mailgun"]:::external
        PostHog["PostHog"]:::external
        MITPlat["MIT Learning Platforms"]:::external
    end

    %% Development Infrastructure
    subgraph DevInfra
        Docker["Docker Compose"]:::infra
        CI["CI/CD Workflows"]:::infra
        DevContainer["Dev Container"]:::infra
    end

    %% Relationships
    NextApp --> APIClient
    APIClient --> DRF
    CompLib --> NextApp
    DRF --> DjangoApps
    DjangoApps --> Postgres
    DjangoApps --> OpenSearch
    DjangoApps --> S3
    DjangoApps --> Redis
    Auth --> Keycloak
    DjangoApps --> Mailgun
    DjangoApps --> PostHog
    DjangoApps --> MITPlat
    Celery --> DjangoApps

    %% Click Events
    click NextApp "https://github.com/mitodl/mit-learn/tree/main/frontends/main/"
    click CompLib "https://github.com/mitodl/mit-learn/tree/main/frontends/ol-components/"
    click APIClient "https://github.com/mitodl/mit-learn/tree/main/frontends/api/"
    click Storybook "https://github.com/mitodl/mit-learn/tree/main/frontends/ol-components/.storybook/"
    click LR "https://github.com/mitodl/mit-learn/tree/main/learning_resources/"
    click Auth "https://github.com/mitodl/mit-learn/tree/main/authentication/"
    click Profiles "https://github.com/mitodl/mit-learn/tree/main/profiles/"
    click Channels "https://github.com/mitodl/mit-learn/tree/main/channels/"
    click NewsEvents "https://github.com/mitodl/mit-learn/tree/main/news_events/"
    click VectorSearch "https://github.com/mitodl/mit-learn/tree/main/vector_search/"
    click AIChat "https://github.com/mitodl/mit-learn/tree/main/ai_chat/"
    click Docker "https://github.com/mitodl/mit-learn/blob/main/docker-compose.yml"
    click DevContainer "https://github.com/mitodl/mit-learn/tree/main/.devcontainer/"
    click CI "https://github.com/mitodl/mit-learn/tree/main/.github/workflows/"

    %% Legend
    subgraph Legend
        Frontend_L["Frontend Component"]:::frontend
        Backend_L["Backend Service"]:::backend
        Database_L[(Database)]:::database
        External_L["External Service"]:::external
        Security_L["Security Component"]:::security
        Infra_L["Infrastructure"]:::infra
    end
```
