How znasif/hacktemplate works
Main Components
• Client Application:
– Components include two variants: the primary client (main.py with screen_capture.py) and a cross-platform beta (main_beta.py with screen_capture_beta.py).
– Responsible for capturing the video feed (note the dual-display requirement) and initiating the stream.
– Establishes the connection (via WebSockets) to the server.
• Server Application:
– The entry point (stream.py) runs as an asynchronous server (using uvicorn) to accept the incoming stream.
– Contains several processing modules under the “processors” folder: base_processor, Groq processor, MediaPipe processor, OCR processor, OpenAI processor, and YOLO processor.
– Additional modules (llama.py and llava.py) likely wrap or interface with more advanced language/vision models.
– Utility scripts (utils.py), and a models directory (which houses for example the YOLO segmentation model, yolo11n-seg.pt).
• External Dependencies and Environment Setup:
• Data Flow:
– Video capture from the client’s monitor is streamed to the server.
– The server processes the stream via the selected processor (user-selected from options such as “Dense Region Caption,” “OCR,” etc.).
– The processed results are then output, with descriptions being read aloud or potentially routed to another system (e.g., WhatsApp desktop app for audio sharing).
Interactions and Relationships
– Emphasize that these modules are plugins, making it easy for extension.
• Optionally, include callouts for additional components such as “LLama Module” and “LLava Module” to indicate integration with large language/vision models.
Architectural Patterns and Design Principles
• That the application follows a client–server architecture with a separation of concerns:
– The client is solely responsible for capturing video and handling local dependencies.
– The server acts as a processing hub and orchestrates multiple processing pipelines.
• Modular design of the processor components that can be easily swapped or extended based on user selection.
• Asynchronous, event-driven nature of processing (using websockets and uvicorn).
Relevant Technologies
• Client with labels for Windows/Cross-Platform differences (indicating pywin32 for Windows, pyautogui/screeninfo for cross-platform).
• Optionally, mention the use of virtual audio cables for routing audio (as part of the remote sharing feature).
Specific Elements
– "Client Application" – Include subcomponents "Screen Capture (main.py / screen_capture.py)" and "Cross-Platform Variant (main_beta.py)".
– "Connection Layer" – Indicate the WebSocket link established by the client.
– "Server Application / Stream Handler" – Represented by stream.py.
• Base Processor
• Groq Processor
• MediaPipe Processor
• OCR Processor
• YOLO Processor
• OpenAI Processor
– "LLama/LLava Integration Modules" – Representing advanced model integrations.
• Annotations that indicate configuration points (such as the .env file entries) that enable inter-component communication.