How pvlbzn/latai works
Main Components
• Command Line Entry Points:
• The project includes two entry points: a top-level main.go and one in cmd/main.go. They initialize the application and set up the main loop.
• Terminal UI (TUI) Module:
• Located in internal/tui, it is responsible for rendering the UI and includes the three distinct views (Table View, Information Panel, and Events Panel).
• Prompts Manager:
• Separated in internal/prompt. It handles default prompts (stored in internal/prompt/prompts) and supports loading custom prompts from the user’s home directory (~/.latai/prompts).
• Evaluator:
• Found in internal/evaluator. This module likely orchestrates the measurement of latency by coordinating between sending prompts and receiving responses.
• Providers:
• Built-in providers (OpenAI, Groq, AWS Bedrock) reside in internal/provider. A common Provider interface (defined in provider.go) abstracts the implementations. Each provider is responsible for verifying API keys, sending requests, and measuring latency.
Data Flow and Interactions
• User actions (starting the TUI application) are captured by the command line entry point.
• The TUI module displays three panels:
• Table View: shows the list of AI models loaded upon verifying API keys (via provider’s VerifyAccess functions).
• Information Panel: displays real-time performance metrics (e.g., latency, jitter, etc.) provided by the evaluator.
• Events Panel: logs non-blocking events and any errors, especially during asynchronous evaluations.
• The Evaluator module leverages default or custom prompts (managed by the prompt module) and iterates through the provider modules.
• Providers perform API key validation via their VerifyAccess methods, fetch available models, and measure latency (using the Measure method). They have two levels of API calls – one for verification and one for sending latency testing prompts.
• The TUI is updated with responses and test results from the Evaluator and Providers.
Important Architectural/Design Patterns and Principles
• Modularity and Separation of Concerns:
• The architecture cleanly separates UI (internal/tui), evaluation logic (internal/evaluator), prompt management (internal/prompt) and provider integration (internal/provider).
• Abstraction via Interfaces:
• A common Provider interface (in provider.go) ensures each provider (OpenAI, Groq, Bedrock) can be interchanged and extended.
• Parallel Execution:
• The README mentions parallel non-blocking measurements; so envision asynchronous operations managed likely within the evaluator and then logged in the events panel.
• Configuration via Environment Variables:
• API keys and settings (like AWS_PROFILE/REGION) are managed externally as shown in the README.