How ncp-1st/ncp-avatar works
Mediary is an AI diary and avatar-video application with calendar/history browsing, location-based memories, and personalized supportive chat. The main workflow is mixed-media entry, preprocessing, AI interpretation and clarification, draft review and approval, then narration/video rendering and storage. Separate branches cover counseling with diary context, calendar retrieval, and map messages. The graph combines sampled implementation links with explicitly documented README flows; unsampled integrations are represented only where documented.
Main entry points: diary_orchestrator.py Diary orchestration · diary_media.py Narration and video · diary.py Diary API · diary_pipeline.py Input preprocessing
Components 27
User experience
- Diary screen
frontend/pages/diary.py - Calendar screen
frontend/pages/calendar.py - Counsel screen
frontend/pages/counsel.py - Map screen
frontend/pages/map.py
Diary workflow
- Diary API
backend/api/diary.py - Input preprocessing
backend/orchestration/diary_pipeline.py - Diary orchestration
backend/orchestration/diary_orchestrator.py - Multimodal diary agent
backend/agents/diary_chatbot/hcx.py - Diary workflow models
backend/agents/diary_chatbot/models.py - Script generation
backend/agents/diary_chatbot/write_script/hcx.py - Narration and video
backend/orchestration/diary_media.py
Memory and discovery
- Calendar API
backend/api/calendar.py - Calendar queries
backend/repositories/calendar_repository.py - Maps API
backend/api/maps.py - Map message service
backend/services/maps/store.py
Counseling
- Counsel API
backend/api/counsel.py - Counsel orchestration
backend/orchestration/counsel_flow.py - Counsel agent
backend/agents/counsel_chatbot/context_agent.py - Diary knowledge search
backend/services/knowledge/diary_vector.py
Persistence and services
- Diary persistence
backend/repositories/sqlalchemy.py - Relational database
database/models.py - Object storage
backend/services/storage/ncp_object.py - Speech recognition
backend/services/speech/clova.py - Voice synthesis
backend/services/voice/clova_voice.py - Avatar renderer
backend/services/avatar/huggingface_space.py - Embedding service
backend/services/embedding/service.py
Other components
- User
Connections 30
- User → Diary screen: records media
README.md - Diary screen → Diary API: submits entries
frontend/pages/diary.py - Diary API → Input preprocessing: preprocesses inputs
backend/api/diary.py - Input preprocessing → Object storage: stores media
backend/orchestration/diary_pipeline.py - Input preprocessing → Speech recognition: transcribes audio
backend/orchestration/diary_pipeline.py - Diary API → Diary orchestration: handles chat and review
backend/api/diary.py - Diary orchestration → Multimodal diary agent: interprets context
backend/orchestration/diary_orchestrator.py - Diary orchestration → Diary workflow models: uses workflow state
backend/orchestration/diary_orchestrator.py - Diary orchestration → Script generation: generates draft
- Diary orchestration → Diary persistence: saves approved version
- Diary orchestration → Embedding service: indexes approved diary
backend/orchestration/diary_orchestrator.py - Diary orchestration → Narration and video: renders approved diary
backend/orchestration/diary_orchestrator.py - Narration and video → Voice synthesis: synthesizes narration
backend/orchestration/diary_media.py - Narration and video → Avatar renderer: renders avatar video
backend/orchestration/diary_media.py - Narration and video → Object storage: stores outputs
backend/orchestration/diary_media.py - Narration and video → Diary persistence: updates render records
backend/orchestration/diary_media.py - Diary persistence → Relational database: reads and writes
backend/repositories/sqlalchemy.py - User → Calendar screen: browses history
README.md - Calendar screen → Calendar API: requests diary history
frontend/pages/calendar.py - Calendar API → Calendar queries: fetches filtered entries
backend/api/calendar.py - Calendar queries → Relational database: queries diary records
backend/repositories/calendar_repository.py - User → Counsel screen: starts a conversation
README.md - Counsel screen → Counsel API: sends message
- Counsel API → Counsel orchestration: routes counseling
- Counsel orchestration → Counsel agent: applies support guidance
- Counsel orchestration → Diary knowledge search: retrieves diary context
- User → Map screen: places a memory
README.md - Map screen → Maps API: sends location message
- Maps API → Map message service: stores map message
backend/api/maps.py - Maps API → Relational database: persists location record
backend/api/maps.py