How aliyan-navaid/parallelpixels works
Project Type and Purpose
• This is a CLI-based, concurrent image-processing pipeline written in C (C11) with POSIX Threads.
• Purpose: Monitor an input directory, chunk images, apply filters in parallel, reconstruct images, and write them out—displaying real‐time stats and supporting graceful shutdown.
File Structure
• Top-level:
• main.c, image.py (utility/script)
• CMakeLists.txt, clean.sh
• pipeline/… broken into chunking, filter, reconstruction sub-dirs
• shared/… generic data structures, thread pool, image I/O, hash, macros
• vendors/xxHash
• Each pipeline stage has its own include/ and src/ folders.
• Shared folder supplies thread-safe queues, dynamic arrays, dictionaries, image routines, threading primitives.
• Build: CMake + make producing ppxl executable.
Extract Architecture from README
• Stages: Watcher → Chunker → Filter → Reconstruction → Output
• Communication via thread-safe queues: name_queue, chunker_filtering_queue, filtering_reconstruction_queue.
• Auxiliary threads: statistics display, user-input listener (for shutdown).
• Graceful shutdown on signals or user input.
a. Main Components
• Input Directory (filesystem)
• Watcher Thread (directory_monitor)
• Name Queue (image_queue)
• Chunker Thread Pool (image_chunker + file_tracker)
• Chunker→Filter Queue (shared queue)
• Filter Thread Pool (chunk_threader + filter)
• Filter→Reconstruction Queue
• Reconstruction Thread
• Output Directory (filesystem)
• Stats & Shutdown Threads (thread_pool, atomic counters)
• Shared Utilities (th pool, darray/dlist/dict)
• External Libraries (stb_image, stb_image_write, xxHash)
b. Relationships & Data Flow
c. Architectural Patterns & Principles
• Pipeline / Producer–Consumer model
• Thread-safe queues as bounded buffers
• Thread pools per stage maximize CPU usage
• Separation of concerns: each stage in its own module
• Graceful teardown via broadcast shutdown flags (stdatomic.h + pthreads)
d. Key Technologies & Frameworks
• Language: C11, POSIX Threads (pthread)
• Build System: CMake
• Image I/O: stb_image.h, stb_image_write.h (or libpng/jpeg)
• Utilities: uthash, xxHash, custom dynamic array/dlist/dict in shared/