# mandatoryprogrammer/tarnish architecture

> Architecture diagram of the GitHub repository mandatoryprogrammer/tarnish, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mandatoryprogrammer/tarnish
- Repository: https://github.com/mandatoryprogrammer/tarnish

## Overview

##### Project Type & Purpose
- This is a full-stack, distributed static-analysis tool (“tarnish”) for auditing Chrome extensions.
- It comprises a static JavaScript/CSS/HTML frontend hosted on S3, a Python Tornado web/API backend that enqueues analysis jobs, and horizontally scaling Celery workers that perform the analysis.

##### File-Structure Analysis
- site/ → All static frontend assets (HTML, CSS, JS, Ace editor modes, highlight.js themes) intended for an S3 (or simple HTTP) static website.
- tarnish-server/ → The Tornado + Celery API server:
- server.py + celerylib.py: HTTP endpoints to submit scans, fetch results, view manifest.
- Dockerfile: container image for Elastic Beanstalk or local Docker.
- docker-compose.yaml.example: local orchestration of server, Redis, and optional worker.
- tarnish-worker/ (and tarnish-server/tarnishworker/) → Celery worker(s):
- tasks.py + celerylib.py: download .crx, unzip, run static scans (fingerprinting, CSP analysis, Retire.js lookup, dangerous-function detection) using the bundled chromium-docs and libs/.
- Dockerfile & .ebextensions/Dockerrun.aws.json: container and EB config for Spot Instances.
- start_workers.sh: bootstraps worker processes.
- tornado-celery/ → Vendored library providing Celery integration for Tornado I/O loop.
- Top-level scripts (start.sh, update-site.sh, beanstalk_conf_gen.sh): helpers to deploy server and upload static site.

a. Main Components
- User’s Browser (UI) – hosts static site from S3
- S3 Static-Site Bucket – serves site/ files
- API Gateway or Load Balancer – routes “tarnish” AJAX calls to the Tornado server
- Tornado Web/API Container (tarnish-server) – receives scan requests, queries Redis, reads/writes S3 result cache
- Redis Broker – message queue broker for Celery tasks (e.g., Elasticache or container)
- Celery Worker Containers (tarnish-worker) – pull jobs, perform static analysis, push results to S3
- S3 Results Bucket – stores scan reports & cached JSON for fast repeat fetches
- External Chrome Web Store API – source of extension CRXs

b. Relationships & Data Flow
1. Browser → (HTTP GET) → S3 Static Bucket → serves index.html and JS/CSS.
2. Browser → (XHR POST) → API Gateway → Tornado API “submit scan” endpoint.
3. Tornado → (PUBLISH) → Redis broker enqueues scan task.
4. Celery Worker → (CONSUME) → Redis → downloads extension CRX from Chrome Web Store.
5. Worker → PROCESS → static-analysis modules (chromium-docs, Retire.js, Python parsers) → generates JSON report.
6. Worker → (PUT) → S3 Results Bucket (cache & report files).
7. Tornado → (GET) → S3 Results Bucket (on subsequent requests) → returns JSON to Browser.

c. Architectural Patterns & Principles
- Producer/Consumer (asynchronous job queue via Celery + Redis)
- Separation of Concerns:
- Frontend (pure static assets)
- API layer (Tornado HTTP + Celery producer logic)
- Worker layer (Celery consumers executing analysis)
- Horizontal scalability (Elastic Beanstalk + Spot Instances for worker fleet)
- Caching (S3-backed results to avoid reprocessing)

d. Key Technologies & Frameworks
- AWS S3 (static hosting & results storage)
- AWS Elastic Beanstalk (server & worker deployment, auto-scaling Spot Instances)
- Docker containers (tarnish-server, tarnish-worker)
- Python Tornado (web server)
- Celery + Redis (task queue)
- chromium-docs data (IDL/JSON specs for API fingerprinting)
- Retire.js (vulnerable-library detection)
- Ace Editor & highlight.js (frontend code viewer)

## Diagram (Mermaid)

```mermaid
flowchart LR
    %% UI & Static Layer
    subgraph "UI & Static Layer"
        Browser["User Browser"]:::ui
        S3Static["S3 Static Bucket"]:::storage
    end

    %% API Layer
    subgraph "API Layer"
        APIGW["API Gateway / Load Balancer"]:::compute
        TornadoAPI["Tornado API Container\n(Docker, EB)"]:::compute
        TornadoCeleryLib["tornado-celery Library"]:::compute
    end

    %% Worker Layer
    subgraph "Worker Layer"
        Redis["Redis Broker"]:::queue
        Workers["Celery Worker Containers"]:::compute
        S3Results["S3 Results Bucket"]:::storage
    end

    %% External Services
    Chrome["Chrome Web Store API"]:::external

    %% Connections
    Browser -->|"GET index.html, JS/CSS"| S3Static
    Browser -->|"XHR POST scan"| APIGW
    APIGW -->|"submit scan"| TornadoAPI
    TornadoAPI -->|"enqueue task"| Redis
    Redis -->|"consume task"| Workers
    Workers -->|"download CRX"| Chrome
    Workers -->|"process static analysis"| Workers
    Workers -->|"PUT report"| S3Results
    TornadoAPI -->|"GET cached JSON"| S3Results
    TornadoAPI -->|"return JSON"| Browser

    %% Click Events
    click S3Static "https://github.com/mandatoryprogrammer/tarnish/tree/master/site/"
    click TornadoAPI "https://github.com/mandatoryprogrammer/tarnish/blob/master/tarnish-server/server.py"
    click TornadoCeleryLib "https://github.com/mandatoryprogrammer/tarnish/tree/master/tarnish-server/tornado-celery/"
    click Workers "https://github.com/mandatoryprogrammer/tarnish/blob/master/tarnish-worker/tasks.py"

    %% Styles
    classDef ui fill:#cce5ff,stroke:#004085,color:#004085
    classDef compute fill:#d4edda,stroke:#155724,color:#155724
    classDef queue fill:#fff3cd,stroke:#856404,color:#856404
    classDef storage fill:#ffeeba,stroke:#856404,color:#856404
    classDef external fill:#e2e3e5,stroke:#6c757d,color:#6c757d
```
