How ghassen-fatnassi/fast works
Project type and purpose
• This is a monorepo of educational labs for two domains:
a. MIT 6.5840 (Distributed Systems) labs under MIT6.5840/src
b. Stanford CS 149 (Parallel Computing) labs under StanfordCS149
• Each subdirectory is a self-contained lab or utility (e.g., kvraft, raft, MapReduce, tasksys, ISPC renderers).
File structure
• Top-level directories: MIT6.5840 and StanfordCS149
• MIT6.5840/src contains:
• kvraft, kvsrv, shardctrler, shardkv – key/value stores & sharding controllers
• raft – consensus algorithm
• mr & main – MapReduce coordinator/worker pipelines
• labrpc, labgob – RPC and serialization libraries
• porcupine – consistency checker/visualizer
• StanfordCS149/LAB1 and LAB2 contain multiple progN_xxx directories with C++ and ISPC code, Makefiles, and object artifacts.
README
• Confirms two course tracks: distributed systems (labs 1–5) and parallel computing (labs 1–5)
• Author’s goal: learn-by-doing; each lab implements a self-contained system or library.
a. Main components
• Distributed-systems track:
1. RPC library (labrpc) & serialization (labgob)
2. Raft servers + persister
3. Key/Value servers (kvraft & kvsrv)
4. Shard controller cluster (shardctrler)
5. Sharded KV clusters (shardkv)
6. MapReduce: Coordinator, Worker, Client scripts
7. Consistency checker (porcupine)
• Parallel-computing track:
1. Task execution library (tasksys cpp)
2. Mandelbrot/CUDA/ISPC renderers (prog1–prog3)
3. Math kernels (sqrt, saxpy)
b. Relationships & interactions
• Clients → RPC layer → Servers (raft, kv, shardctrler)
• Coordinator ↔ Workers via RPC
• Raft servers replicate logs → persister
• Shard controller issues shard maps → KV servers
• Parallel labs: main app → tasksys → worker threads / ISPC kernels
c. Architectural patterns & principles
• Modular service layers: RPC / serialization separated from business logic
• Consensus (leader election, log replication)
• Sharding for scalability
• Master-worker for MapReduce
• Thread pool and data parallel kernels for compute labs
d. Key technologies/frameworks
• Go modules for MIT labs (go.mod)
• C++/ISPC + Makefiles for Stanford labs
• Shell scripts for test harnesses
Elements
• Labels: component name + primary language/runtime
• Rectangles for services/binaries
• Distinct hues for MIT vs. Stanford domains
Detailed capture of essential elements
• Within Distributed lane, sub-lanes for RPC, consensus, KV, sharding, MapReduce, and checker
• Within Parallel lane, sub-lanes for tasksys, renderer apps, and math kernels
• Lab folder as a subsystem, with input/output artifacts (e.g., .ppm images)