How med-air/medgame works
MedGAME turns clinical summaries into structured, interactive medical-education stories, then generates multimodal story assets for learners to play through acts, scenes, and question interactions. The pipeline has distinct narrative-design, story-direction, and audio/video-execution stages; the UI serves generated case content and tracks player progress. The UI/API flow and selected services are sampled; other endpoint-to-service wiring and parts of the generation pipeline are described only in the README, so uncertain connections are omitted.
Main entry points: gameStore.ts Game state · GamePage.tsx Game screen · router.py API routes · scene_service.py Scene media
Components 23
Story generation
- Narrative designer
MedGame_Pipeline/generation_factory.py - Story schema
MedGame_Pipeline/pydantic_models.py - Story director
MedGame_Pipeline/movie_builder.py - Audio generation
MedGame_Pipeline/audio_video_pipeline.py - Video generation
MedGame_Pipeline/infinitetalk_runner.py
Game backend
- FastAPI entry
MedGAME_UI/BackEnd/app/main.py - API routes
MedGAME_UI/BackEnd/app/api/router.py - Case loading
MedGAME_UI/BackEnd/app/services/case_service.py - Scene media
MedGAME_UI/BackEnd/app/services/scene_service.py - Subtitle parsing
MedGAME_UI/BackEnd/app/services/subtitle_service.py - Case files
Player experience
- UI routes
MedGAME_UI/Frontend/src/App.tsx - Case selection
MedGAME_UI/Frontend/src/pages/HomePage.tsx - Game screen
MedGAME_UI/Frontend/src/pages/GamePage.tsx - Game state
MedGAME_UI/Frontend/src/stores/gameStore.ts - HTTP client
MedGAME_UI/Frontend/src/api/client.ts - Question interactions
MedGAME_UI/Frontend/src/components/interaction/InteractionContainer.tsx - Story presentation
MedGAME_UI/Frontend/src/components/common/ImmersiveDialogueDisplay.tsx
Player state
- Progress service
MedGAME_UI/BackEnd/app/services/player_service.py - Progress records
MedGAME_UI/BackEnd/app/services/player_service.py
Other components
- Learner
- Patient summary
- Story assets
Connections 29
- Patient summary → Narrative designer: provides input
- Narrative designer → Story schema: constrains output
- Narrative designer → Story director: passes storyline
- Story director → Audio generation: plans tasks
- Story director → Video generation: plans tasks
- Audio generation → Story assets: generates audio
- Video generation → Story assets: generates video
- Story assets → Case files: materializes as
- Learner → Case selection: selects case
- UI routes → Case selection: routes to
- UI routes → Game screen: routes to
- Case selection → Game state: starts case
- Game screen → Game state: reads game state
- Game state → HTTP client: requests data
- HTTP client → FastAPI entry: sends HTTP
- FastAPI entry → API routes: registers routes
- API routes → Case loading: dispatches case routes
- API routes → Scene media: dispatches game routes
- API routes → Progress service: dispatches player routes
- Case loading → Case files: reads case data
- Scene media → Case files: reads scene media
- Scene media → Subtitle parsing: parses subtitles
- Scene media → Game state: returns scene media
- Case loading → Game state: returns story data
- Game state → Game screen: updates view state
- Game screen → Story presentation: presents narrative
- Game screen → Question interactions: presents questions
- Learner → Question interactions: answers questions
- Progress service → Progress records: stores progress