tejasrp24/dbms_proj_backend

Purpose and entry points

AG-OS is a cognitive-memory assistant for users with short-term memory loss. It combines camera-based face recognition, recorded-audio transcription, LLM-generated interaction summaries, persistent memory, and reminders/calendar data. The documented production API entry point is app/services/face_recognition/main.py; an alternate/duplicated backend layout also exposes backend/app/main.py. The React/Vite client starts at frontend/src/main.jsx and renders frontend/src/App.jsx.

HTTP/API boundary

FastAPI routes organize the core domain API: people, caregivers, sessions, interactions, memory, notes, emotions, audio, and calendar events. Route handlers validate request/response payloads through schemas and delegate domain work to services. Key paths: app/api/routes/persons.py, app/api/routes/interactions.py, app/api/routes/memory.py, app/schemas/interaction.py.

Persistence and domain model

PostgreSQL is the system of record, configured through application database/session modules. SQLAlchemy-style domain models represent users, known persons, caregivers, conversations, face encodings, notes, emotional records, and calendar events; association tables encode relationships. Paths: app/db/session.py, app/models/person.py, app/models/conversation.py, app/models/junction_tables.py. A separate SQL schema exists at database/schema.sql, indicating possible legacy/direct-SQL persistence alongside the ORM-oriented structure.

Recognition and interaction pipeline

The primary assistant flow is camera input → face detection/recognition → person identity or unknown-person handling → interaction creation and memory context. Recognition is isolated behind the face-recognition service and controllers, using bundled OpenCV/Caffe and YOLO assets. Paths: app/services/face_recognition/main.py, app/services/face_recognition/face_service.py, app/controllers/face_controller.py, app/models/face_encoding.py. Material ML assets include yolov8n.pt, deploy.prototxt, and res10_300x300_ssd_iter_140000.caffemodel.

Audio, transcription, and summarization

Audio endpoints/controllers accept or manage recordings, then invoke Whisper/faster-whisper transcription. Interaction and conversation services combine transcripts, recognized-person context, and stored memory; the LLM layer produces summaries or assistant output. Paths: app/api/routes/audio.py, app/controllers/audio_controller.py, app/services/whisper_service.py, app/services/conversation_summarizer.py, app/services/llm_service.py. LLM providers are runtime-selectable: Groq by default, with OpenAI and local Ollama options.

Memory and care workflows

Dedicated services maintain long-lived memory, notes, emotional observations, caregiver relationships, and calendar events. This separates medical/care context from transient recognition and transcription requests. Paths: app/services/memory_service.py, app/services/note_service.py, app/services/emotion_service.py, app/services/caregiver_service.py, app/services/calendar_service.py.

Reminders, scheduling, and external integrations

APScheduler supports scheduled application work, while a Celery-oriented reminders subsystem provides asynchronous reminder tasks. Google Calendar and Google Tasks integrations form an external authorization/API boundary. Paths: app/core/scheduler.py, app/ai_models/reminders/tasks.py, app/ai_models/reminders/celery_config.py, app/services/google_calendar.py, app/services/google_tasks.py.

Runtime and repository shape

The intended runtime is Python 3.11/FastAPI on port 8004 plus a Node 18 React/Vite frontend on port 5173; Docker support is rooted at Dockerfile. PostgreSQL may be local or Neon-hosted. The repository contains overlapping app/ and backend/app/ implementations; use the README’s app/services/face_recognition/main.py entry point unless deployment wiring explicitly targets backend/run.py.