fatihmakes/mark-xlvi

MARK XLVI is a local, cross-platform assistant with three major surfaces: voice/desktop control, a remote dashboard for phone/browser access, and persistent memory. The runtime is centered on Gemini-backed reasoning, with local OS automation, screen understanding, and speech I/O wrapped around it.

1. Primary orchestration path

- main.py is the likely main entrypoint for launching the assistant.

- ui.py appears to provide the local user interface layer.

- The core loop is: user input → LLM reasoning → tool/action execution → response output.

2. LLM and prompt layer

- core/llm_client.py is the API boundary to the model provider, and the README says Gemini is the main integration.

- core/prompt.txt likely contains the system instructions / assistant persona.

- This layer is the decision-making center that chooses whether to answer directly or invoke a tool.

3. Speech pipeline

- core/stt.py handles speech-to-text input from the microphone.

- core/tts.py handles spoken output back to the user.

- This is the real-time voice interface described in the README and is separate from the action execution layer.

4. Tool/action subsystem

- The actions/ package is the assistant’s operational boundary: each file is a capability module.

- Core device control lives in computer_control.py, computer_settings.py, desktop.py, and open_app.py.

- Web/browser automation is split into browser_control.py and web_search.py.

- File-centric operations are handled by file_controller.py and file_processor.py.

- Visual/contextual input comes from screen_processor.py.

- Specialized task modules include weather_report.py, flight_finder.py, youtube_video.py, send_message.py, reminder.py, game_updater.py, code_helper.py, and dev_agent.py.

5. OS and environment control boundary

- The assistant is explicitly cross-platform, so OS-specific behavior is likely isolated inside the actions layer rather than the core loop.

- actions/computer_control.py and related modules are the abstraction over Windows/macOS/Linux differences.

- core/installer.py suggests a bootstrap/setup path for platform dependencies and runtime preparation.

6. Visual awareness path

- actions/screen_processor.py and actions/desktop.py indicate screen capture / desktop state inspection.

- This provides the “see” capability mentioned in the README and likely feeds contextual information back into the LLM before action selection.

- In the architecture diagram, this should be shown as a feedback loop from the desktop into the reasoning engine.

7. Persistent memory subsystem

- memory/memory_manager.py is the main persistence abstraction.

- memory/config_manager.py likely manages memory-related settings and storage configuration.

- This subsystem preserves long-term context across sessions, which the README describes as advanced persistent memory.

8. Remote dashboard / phone control

- dashboard/server.py is a separate server process or embedded web server for remote access.

- dashboard/static/app.html and login.html are the browser-facing UI assets.

- config/certs/jarvis.crt and jarvis.key show the dashboard uses TLS, which matters for the secure remote-control story.

- This is a distinct boundary from the local desktop UI: remote clients talk to the dashboard, not directly to the assistant internals.

9. Security and transport

- The presence of certificates in config/certs/ indicates HTTPS/TLS is part of the deployment model.

- This is important because the README emphasizes secure mobile control and file sharing.

- The security boundary is therefore the dashboard server, not the action modules themselves.

10. External dependencies and integrations

- The README indicates Gemini API usage for reasoning.

- playwright install in the setup instructions implies browser automation depends on Playwright.

- The assistant also depends on local microphone/speaker hardware and standard OS automation capabilities.

11. Likely end-to-end data flow

- Local voice path: microphone → STT → LLM client → action modules / memory → TTS → user.

- Remote path: mobile/browser client → dashboard server → assistant core/actions → response back to dashboard.

- Visual path: screen/desktop capture → screen processor → LLM context → actions.

12. What to show in a diagram

- Put core/llm_client.py at the center.

- Surround it with three major systems: speech I/O, actions/tooling, and memory.

- Add a separate remote access lane for dashboard/server.py and static web assets.

- Draw the OS as the execution target for the actions/ package, and Gemini as the external reasoning service.