How dabeaz/curio works
Project type and purpose
• From the file tree and README, this project (“Curio”) is an open-source Python library implementing a coroutine‐based asynchronous runtime and set of concurrency primitives. It is *not* a full-stack web app or compiler but a standalone library whose core function is to provide async tasks, scheduling, I/O, networking, synchronization primitives, and structured concurrency.
File structure
• Top-level directories:
• curio/ ← The library’s source code
• docs/ ← Documentation (Sphinx)
• examples/ ← Sample programs demonstrating usage
• tests/ ← Unit and integration tests
• Key source modules in curio/:
• kernel.py ← The event‐loop (task scheduler)
• sched.py ← Scheduling policies and ready/wait queues
• task.py ← Task object abstraction (lifecycle, cancellation)
• traps.py ← Async “syscalls” interface to kernel services
• io.py, file.py, network.py, socket.py, ssl.py
← Async I/O and networking layers built on traps
• sync.py, queue.py, channel.py
← Async synchronization and communication primitives
• thread.py, workers.py
← Bridges to thread‐ and process‐based execution
• No front-end/back-end split—this is purely a library offering components.
README for additional insights
• Emphasis on *separation* of synchronous vs asynchronous environments
• Core features: tasks, cancellation, locks, events, queues, subprocess/thread integration, structured concurrency
• No external dependencies—interfaces directly with OS via low-level I/O
a. Main components to represent
1. User code (async functions & tasks)
2. Curio Kernel / Event Loop (curio/kernel.py)
3. Scheduler (curio/sched.py)
4. Trap layer (curio/traps.py) – the syscalls interface
5. Async I/O subsystems
• File I/O (file.py)
• Network sockets & SSL (socket.py, ssl.py, network.py)
6. Sync primitives & channels (sync.py, queue.py, channel.py)
7. Worker subsystem for threads/processes (thread.py, workers.py)
8. Underlying OS / system calls
b. Relationships and interactions
• User code invokes await on tasks & primitives
• Tasks are managed by the Kernel which uses the Scheduler to enqueue/dequeue task runs
• When a task performs I/O or waits on a lock, it issues a Trap call
• The Trap layer marshals requests to the appropriate subsystem (I/O, sync, etc.)
• I/O subsystems interact with the OS (selectors, epoll/kqueue, socket APIs)
• Sync primitives may block tasks by re‐queuing them until an event/unlock occurs
• Worker threads or processes are used when user code opts in—tasks can be off‐loaded via workers
c. Architectural patterns & design principles
• Layered design: user code → traps → kernel/scheduler → OS
• Separation of concerns: pure async code never mixes sync primitives except through defined traps
• Event‐driven loop tackling concurrency without OS threads
• Structured concurrency support (task groups, cancellation scopes)
d. Key technologies & frameworks
• Python 3.7+ async/await syntax
• Selector‐based event loop (platform‐specific multiplexing)
• Zero external dependencies beyond Python stdlib