# certunlp/ngen architecture

> Architecture diagram of the GitHub repository certunlp/ngen, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/certunlp/ngen
- Repository: https://github.com/certunlp/ngen

## Overview

##### Project type and purpose
- This is a full-stack Incident Response Management System (Ngen) for CSIRTs.
- The backend is a Django REST API with Celery for background tasks and integrates with external tools (IntelMQ, Cortex, Kintun, MISP).
- The frontend is a React/Vite SPA served via Nginx.
- Docker Compose orchestrates the environment (development and production), including the DB, message broker, Django, Celery, React, and Nginx.

##### File structure
- Top-level folders:
- docker/ – Dockerfiles, compose files, env templates, entrypoints
- frontend/ – React/Vite app, assets, components, services, build configs
- ngen/ & project/ – Django app code, models, serializers, views, URLs, settings, Celery, mailer
- manage.py, requirements.txt – Django entry point and dependencies
- Patterns:
- Separate frontend and backend projects
- Monolithic Django app employing MVC (models, serializers/controllers, views)
- Celery for asynchronous tasks
- Configuration files: docker-compose.yml, Dockerfile.prod/dev, nginx templates, .env files

##### Key points from the README
- Production and development ports (443 vs 3000/8000)
- Default credentials, environment variable requirements
- Integration points: IntelMQ, Cortex, Kintun, MISP
- Emphasis on containerized deployment and environment segregation

##### Main components
a. Frontend Layer
- User’s browser
- Nginx (serving the React SPA)
- React/Vite application
- API client module (axios + token interceptors)
b. API Layer
- Nginx (reverse proxy for /api) → Gunicorn/Uvicorn
- Django REST Framework endpoints (ngen/views)
- Authentication & permissions middleware
c. Data Layer
- PostgreSQL (or configured SQL database)
- Django ORM (models in ngen/models)
- Fixtures for initial data
d. Asynchronous Processing
- Message broker (Redis or RabbitMQ)
- Celery workers (ngen/tasks.py, services)
e. External Services
- SMTP server (EMAIL_HOST)
- IntelMQ, Cortex, Kintun, MISP (via ngen/mailer & ngen/services)
f. Storage & Reporting
- Local/static file storage (attachments, artifacts)
- Report templates (Django templates for PDF/HTML)

##### Relationships and interactions
- Browser → Nginx(frontend) → React SPA → API calls → Nginx(API) → Gunicorn → Django views
- Django views ↔ Django ORM ↔ PostgreSQL
- Long-running tasks or integrations → Celery via message broker → external tools → DB updates
- Django email module → SMTP
- Static assets and uploaded files served via Nginx or Django’s static handler

##### Architectural patterns & design principles
- Three-tier architecture: Presentation (React), Business logic (Django), Persistence (Postgres)
- API-first design with RESTful endpoints and token-based auth
- Layered separation: web server, application server, worker pool
- Containerized micro-services style (each major process in its own container)

##### Technologies, frameworks, and libraries
- Frontend: React, Vite, Redux/store, axios, i18next, SCSS, React Router
- Backend: Django, Django REST Framework, Celery, Gunicorn, django-cortex, django-misp, Redis/RabbitMQ, PostgreSQL
- Infrastructure: Docker, Docker Compose, Nginx, SSL certificates
- CI/CD: GitHub Actions (frontend-dev.yml, backend-dev.yml, push-tag.yml)

##### Essential details without over-splitting
- Docker container as a high-level component
- Only the major external services, not every individual API service file

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Frontend Layer
    UserBrowser["User Browser"]:::frontend
    subgraph "Frontend Layer"
        NginxFE["Nginx (serving SPA)"]:::infra
        ReactApp["React/Vite SPA"]:::frontend
        APIClientMain["API Client Module"]:::frontend
        Interceptors["Token Interceptors"]:::frontend
    end

    %% API Layer
    subgraph "API Layer"
        NginxAPI["Nginx (reverse proxy /api)"]:::infra
        APIprefix["URL Routing (/api)"]:::backend
        Gunicorn["Gunicorn/Uvicorn"]:::backend
        AuthMiddleware["Authentication & Permissions"]:::backend
        DRF["Django REST Framework Endpoints"]:::backend
    end

    %% Django App Components
    subgraph "Django App"
        Apps["App Settings & Initialization"]:::backend
        ORM["Django ORM (models)"]:::backend
        Fixtures["Initial Data Fixtures"]:::backend
        StaticStorage["Static File Storage"]:::infra
        Templates["Report Templates"]:::backend
        AdminNode["Admin & Management Commands"]:::backend
    end

    %% Asynchronous Layer
    subgraph "Asynchronous Processing"
        Broker["Message Broker (Redis/RabbitMQ)"]:::datastore
        Celery["Celery Workers"]:::backend
    end

    %% Data Layer
    subgraph "Data Layer"
        Postgres["PostgreSQL"]:::datastore
        Settings["Database Configuration"]:::backend
    end

    %% Integration Services
    subgraph "Background Services & Integrations"
        ContactLookup["Contact Lookup Service"]:::external
        CortexService["Cortex Integration"]:::external
        KintunService["Kintun Integration"]:::external
        EmailClient["Email Client"]:::external
        EmailHandler["Email Handler"]:::external
        IMAPStrategy["IMAP Strategy"]:::external
        SMTP["SMTP Server"]:::external
        IntelMQ["IntelMQ"]:::external
        MISP["MISP"]:::external
    end

    %% Orchestration & CI/CD
    Orchestrator["Docker Compose & Environment"]:::infra
    CI["CI/CD Workflows"]:::infra

    %% Connections
    UserBrowser --> NginxFE --> ReactApp --> APIClientMain --> Interceptors --> NginxAPI --> APIprefix --> Gunicorn --> AuthMiddleware --> DRF
    DRF --> ORM --> Postgres
    DRF --> Fixtures
    DRF --> StaticStorage
    DRF --> Templates
    DRF --> Broker --> Celery
    Celery --> ORM
    Celery --> ContactLookup
    Celery --> CortexService
    Celery --> KintunService
    Celery --> EmailClient --> SMTP
    EmailHandler --> IMAPStrategy
    Orchestrator --> NginxFE
    Orchestrator --> NginxAPI
    Orchestrator --> Postgres
    Orchestrator --> Broker
    Orchestrator --> Celery
    CI --> Orchestrator

    %% Click Events
    click ReactApp "https://github.com/certunlp/ngen/tree/main/frontend/src"
    click NginxFE "https://github.com/certunlp/ngen/blob/main/frontend/nginx-prod/nginx.conf.template.ssl"
    click NginxFE "https://github.com/certunlp/ngen/blob/main/frontend/nginx-prod/docker-entrypoint.sh"
    click NginxFE "https://github.com/certunlp/ngen/blob/main/frontend/Dockerfile.dev"
    click NginxFE "https://github.com/certunlp/ngen/blob/main/frontend/Dockerfile.prod"
    click APIClientMain "https://github.com/certunlp/ngen/blob/main/frontend/src/api/api.jsx"
    click Interceptors "https://github.com/certunlp/ngen/blob/main/frontend/src/api/setupInterceptors.jsx"
    click DRF "https://github.com/certunlp/ngen/tree/main/ngen/views"
    click AuthMiddleware "https://github.com/certunlp/ngen/blob/main/ngen/middleware.py"
    click AuthMiddleware "https://github.com/certunlp/ngen/blob/main/ngen/backends.py"
    click AuthMiddleware "https://github.com/certunlp/ngen/blob/main/ngen/permissions.py"
    click Gunicorn "https://github.com/certunlp/ngen/blob/main/project/wsgi.py"
    click Gunicorn "https://github.com/certunlp/ngen/blob/main/project/asgi.py"
    click Celery "https://github.com/certunlp/ngen/blob/main/project/celery.py"
    click Celery "https://github.com/certunlp/ngen/blob/main/ngen/tasks.py"
    click ContactLookup "https://github.com/certunlp/ngen/blob/main/ngen/services/contact_lookup.py"
    click CortexService "https://github.com/certunlp/ngen/blob/main/ngen/cortex.py"
    click KintunService "https://github.com/certunlp/ngen/blob/main/ngen/kintun.py"
    click EmailClient "https://github.com/certunlp/ngen/blob/main/ngen/mailer/email_client.py"
    click EmailHandler "https://github.com/certunlp/ngen/blob/main/ngen/mailer/email_handler.py"
    click IMAPStrategy "https://github.com/certunlp/ngen/blob/main/ngen/mailer/email_protocol_strategy/imap_strategy.py"
    click ORM "https://github.com/certunlp/ngen/tree/main/ngen/models"
    click Fixtures "https://github.com/certunlp/ngen/tree/main/ngen/fixtures"
    click StaticStorage "https://github.com/certunlp/ngen/tree/main/ngen/static"
    click StaticStorage "https://github.com/certunlp/ngen/blob/main/ngen/storage.py"
    click Templates "https://github.com/certunlp/ngen/tree/main/ngen/templates/reports"
    click Orchestrator "https://github.com/certunlp/ngen/blob/main/docker/docker-compose.yml"
    click Orchestrator "https://github.com/certunlp/ngen/blob/main/docker/docker-compose-dev.yml"
    click Orchestrator "https://github.com/certunlp/ngen/blob/main/docker/Dockerfile.dev"
    click Orchestrator "https://github.com/certunlp/ngen/blob/main/docker/Dockerfile.prod"
    click Orchestrator "https://github.com/certunlp/ngen/blob/main/docker/.env/ngen.base.env.example"
    click Settings "https://github.com/certunlp/ngen/blob/main/project/settings.py"
    click Broker "https://github.com/certunlp/ngen/blob/main/docker/docker-compose.yml"
    click Broker "https://github.com/certunlp/ngen/blob/main/docker/docker-compose-dev.yml"
    click CI "https://github.com/certunlp/ngen/blob/main/.github/workflows/frontend-dev.yml"
    click CI "https://github.com/certunlp/ngen/blob/main/.github/workflows/backend-dev.yml"
    click AdminNode "https://github.com/certunlp/ngen/blob/main/ngen/management/commands/loaddatafirsttime.py"
    click AdminNode "https://github.com/certunlp/ngen/blob/main/ngen/admin.py"
    click APIprefix "https://github.com/certunlp/ngen/blob/main/project/urls.py"
    click Apps "https://github.com/certunlp/ngen/blob/main/ngen/apps.py"

    %% Styles
    classDef frontend fill:#E3F2FD,stroke:#42A5F5,color:#1E88E5
    classDef backend fill:#E8F5E9,stroke:#66BB6A,color:#2E7D32
    classDef datastore fill:#FFF3E0,stroke:#FFA726,color:#EF6C00
    classDef external fill:#F3E5F5,stroke:#AB47BC,color:#6A1B9A
    classDef infra fill:#ECEFF1,stroke:#90A4AE,color:#37474F
```
