How nyu-llm-ctf/llm_ctf_dataset_dev works
Project type and purpose
• This is a data‐centric repository (“LLM_CTF_Dataset_Dev”) containing raw Capture-The-Flag (CSAW CTF) challenge materials organized by year and category (crypto, web, pwn, rev, misc). There is no conventional “application” code; the repository’s purpose is to serve as a dataset for ML/CTF research or tooling.
File structure
• Top‐level directories:
• 2013, 2014, 2015, 2016 – each containing subfolders by CTF phase (CSAW-Finals, CSAW-Quals) and challenge categories (crypto, web, pwn, rev, misc).
• README.md at the root (identifying the repo).
• Each challenge folder typically includes: Dockerfile/docker-compose.yml (deployment), challenge.json (metadata), source binaries/scripts, input/output assets, solution code, and sometimes flag files.
• There are no src/, lib/, or tests folders as in an application; instead the file tree itself is the raw dataset.
Diagramming strategy
a. Main components
• Raw File Storage – the on‐disk hierarchy: years → phases → categories → individual challenge directories.
• Metadata Indexer – a component that reads challenge.json files and builds a normalized catalog.
• Processing Pipeline – scripts or services that extract features (e.g., text extraction, binary parsing, container configuration) from raw challenge contents.
• Database – a persistent store (SQL or NoSQL) for processed metadata and extracted features.
• API Layer – RESTful service exposing dataset queries (by year, category, keywords).
• Frontend/UI (optional) – web dashboard to browse/search the dataset.
• Consumer/ML Pipeline – external ML training systems or CTF tooling that consume the API or database.
b. Relationships and data flows
1. Raw File Storage feeds into the Metadata Indexer and Processing Pipeline.
2. Both Indexer and Pipeline write structured records to the Database.
3. API Layer reads from Database to serve requests.
4. Frontend/UI calls API Layer.
5. Consumer/ML Pipeline calls API Layer or reads directly from Database.
c. Architectural patterns/design principles
• Microservices for Indexer, Processing Pipeline, and API Layer to allow independent scaling.
• Event-driven ingestion: when new challenge directory is added, an event triggers indexing and processing.
• Separation of concerns: raw storage vs. processing vs. serving.
d. Key technologies/frameworks (examples)
• File storage: local disk or object store (S3).
• Message bus for events: Kafka or RabbitMQ.
• Processing: Python or Node.js scripts in Docker containers.
• Database: PostgreSQL or MongoDB.
• API: Flask, Express.js, or FastAPI.
• Frontend: React or simple server-rendered pages.