How ashishbamania/open-scribe works
Main Components
• Frontend: The React-based user interface (located in the “open-scribe-frontend” directory) that handles audio recording (via MediaRecorder API), playback, and user interactions.
• Backend: The Node.js/Express.js server (located in the “open-scribe-backend” directory) responsible for processing audio uploads, interacting with OpenAI’s APIs (for transcription and text formatting), and handling file uploads (with Multer).
• External Services: OpenAI’s Whisper model for speech-to-text transcription and GPT-4 for formatting transcriptions into structured medical notes.
• User’s Browser/Client: The source interface where users interact with the application.
Relationships and Interactions
• The User’s Browser interacts with the Frontend: Users record audio, review recordings, and send a “Process Recording” request.
• The Frontend sends audio file data and any related requests to the Backend through HTTP API endpoints.
• The Backend receives these requests, uses Multer to handle audio file uploads, and then processes the file by calling external APIs:
• First, sending the audio to OpenAI Whisper for transcription.
• Next, sending the transcribed text to GPT-4 to format into organized medical notes.
• The Backend returns the processed results to the Frontend for display and possible download.
Architectural Patterns and Design Principles
• Separation of Concerns: Clear division between the frontend (user interface/interaction) and the backend (data processing and API calls).
• API-driven Communication: Use of HTTP endpoints for cross-component communication, ensuring loose coupling between frontend and backend.
• External Integration: Clear dependency on external AI services (OpenAI API) which are abstractly integrated into the backend’s workflow.
• Modular Design: With separate directories for frontend and backend, each component is self-contained and maintainable.
Technology and Framework Details
• Frontend Technologies: React.js for building components (e.g., ConsultationRecorder), CSS3 for styles, MediaRecorder API for handling audio capture.
• Backend Technologies: Node.js with Express.js to set up server routes, Multer for file uploading, and integration with OpenAI APIs (Whisper for transcription and GPT-4 for text formatting).