# kevinjose102/context-aware-agent architecture

> Architecture diagram of the GitHub repository kevinjose102/context-aware-agent, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kevinjose102/context-aware-agent
- Repository: https://github.com/kevinjose102/context-aware-agent

## Overview

Lenni is a conversational mental-health support and medical-triage agent. A user sends text or recorded audio through the chat interface; the Flask backend processes the message, uses conversation context and safety-oriented triage to choose a response, and returns it to the interface. Its documented capabilities also include voice and text emotion analysis, guideline retrieval, and short- and long-term memory. The backend excerpts show orchestration and Pinecone/LLM calls; unsampled code limits certainty about internal audio and emotion wiring.

Main entry points: [`backend/server.py`](https://github.com/kevinjose102/context-aware-agent/blob/main/backend/server.py) (Message orchestration), [`frontend/app/components/Start.jsx`](https://github.com/kevinjose102/context-aware-agent/blob/main/frontend/app/components/Start.jsx) (Chat interface), [`frontend/app/routes.ts`](https://github.com/kevinjose102/context-aware-agent/blob/main/frontend/app/routes.ts) (App routes)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ui["User interface"]
  node_chat["Chat interface<br/>[Start.jsx]"]
  node_routes["App routes<br/>[routes.ts]"]
end

subgraph group_backend["Conversation backend"]
  node_api["Flask API<br/>[server.py]"]
  node_message_flow["Message orchestration<br/>[server.py]"]
  node_triage["Safety triage<br/>[server.py]"]
  node_speech["Speech processing<br/>[server.py]"]
  node_emotion["Emotion analysis<br/>[server.py]"]
end

subgraph group_memory["Memory and retrieval"]
  node_short_memory["Short-term cache<br/>[server.py]"]
  node_long_memory[("Long-term memory<br/>[server.py]")]
  node_guidelines["Guideline retrieval<br/>[server.py]"]
end

subgraph group_integrations["External services"]
  node_pinecone[("Pinecone")]
  node_llm["Chat model"]
  node_stt["Speech recognition"]
  node_emotion_model["Emotion model"]
end

node_user(("User"))

node_user -->|"sends message"| node_chat
node_routes -->|"routes to chat"| node_chat
node_chat -->|"submits text or audio"| node_api
node_api -->|"dispatches message"| node_message_flow
node_message_flow -->|"evaluates risk"| node_triage
node_message_flow -->|"adds turn"| node_short_memory
node_message_flow -->|"retrieves context"| node_long_memory
node_message_flow -->|"requests guidance"| node_guidelines
node_message_flow -->|"requests response"| node_llm
node_guidelines -->|"queries guidelines"| node_pinecone
node_long_memory -->|"fetches vectors"| node_pinecone
node_message_flow -->|"stores conversation"| node_pinecone
node_api -->|"processes audio"| node_speech
node_speech -.->|"transcribes audio"| node_stt
node_api -.->|"analyzes emotion"| node_emotion
node_emotion -.->|"scores vocal tone"| node_emotion_model
node_message_flow -.->|"detects text emotion"| node_emotion_model
node_api -->|"returns reply"| node_chat

click node_chat "https://github.com/kevinjose102/context-aware-agent/blob/main/frontend/app/components/Start.jsx"
click node_routes "https://github.com/kevinjose102/context-aware-agent/blob/main/frontend/app/routes.ts"
click node_api "https://github.com/kevinjose102/context-aware-agent/blob/main/backend/server.py"
click node_message_flow "https://github.com/kevinjose102/context-aware-agent/blob/main/backend/server.py"
click node_triage "https://github.com/kevinjose102/context-aware-agent/blob/main/backend/server.py"
click node_speech "https://github.com/kevinjose102/context-aware-agent/blob/main/backend/server.py"
click node_emotion "https://github.com/kevinjose102/context-aware-agent/blob/main/backend/server.py"
click node_short_memory "https://github.com/kevinjose102/context-aware-agent/blob/main/backend/server.py"
click node_long_memory "https://github.com/kevinjose102/context-aware-agent/blob/main/backend/server.py"
click node_guidelines "https://github.com/kevinjose102/context-aware-agent/blob/main/backend/server.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_chat,node_routes,node_user toneBlue
class node_api,node_message_flow,node_triage,node_speech,node_emotion toneAmber
class node_short_memory,node_long_memory,node_guidelines toneMint
class node_pinecone,node_llm,node_stt,node_emotion_model toneRose
```

## Components

### User interface

- [Chat interface](https://github.com/kevinjose102/context-aware-agent/blob/main/frontend/app/components/Start.jsx): `frontend/app/components/Start.jsx`
- [App routes](https://github.com/kevinjose102/context-aware-agent/blob/main/frontend/app/routes.ts): `frontend/app/routes.ts`

### Conversation backend

- [Flask API](https://github.com/kevinjose102/context-aware-agent/blob/main/backend/server.py): `backend/server.py`
- [Message orchestration](https://github.com/kevinjose102/context-aware-agent/blob/main/backend/server.py): `backend/server.py`
- [Safety triage](https://github.com/kevinjose102/context-aware-agent/blob/main/backend/server.py): `backend/server.py`
- [Speech processing](https://github.com/kevinjose102/context-aware-agent/blob/main/backend/server.py): `backend/server.py`
- [Emotion analysis](https://github.com/kevinjose102/context-aware-agent/blob/main/backend/server.py): `backend/server.py`

### Memory and retrieval

- [Short-term cache](https://github.com/kevinjose102/context-aware-agent/blob/main/backend/server.py): `backend/server.py`
- [Long-term memory](https://github.com/kevinjose102/context-aware-agent/blob/main/backend/server.py): `backend/server.py`
- [Guideline retrieval](https://github.com/kevinjose102/context-aware-agent/blob/main/backend/server.py): `backend/server.py`

### External services

- **Pinecone**
- **Chat model**
- **Speech recognition**
- **Emotion model**

### Other components

- **User**

## Connections

- User → Chat interface: sends message
- App routes → Chat interface: routes to chat
- Chat interface → Flask API: submits text or audio
- Flask API → Message orchestration: dispatches message
- Message orchestration → Safety triage: evaluates risk
- Message orchestration → Short-term cache: adds turn
- Message orchestration → Long-term memory: retrieves context
- Message orchestration → Guideline retrieval: requests guidance
- Message orchestration → Chat model: requests response
- Guideline retrieval → Pinecone: queries guidelines
- Long-term memory → Pinecone: fetches vectors
- Message orchestration → Pinecone: stores conversation
- Flask API → Speech processing: processes audio
- Speech processing → Speech recognition: transcribes audio
- Flask API → Emotion analysis: analyzes emotion
- Emotion analysis → Emotion model: scores vocal tone
- Message orchestration → Emotion model: detects text emotion
- Flask API → Chat interface: returns reply