How haiderali780/ai-classroom-xr works
Project Type & Purpose
• The project is clearly a full-stack immersive 3D classroom application that leverages Three.js and React Three Fiber for the frontend, along with an Express.js-based proxy server to securely handle API requests to the OpenAI API.
• The focus is on creating an interactive 3D learning environment with an AI-powered teacher, incorporating advanced 3D models, realistic animations, and voice interactivity.
Analyzing the File Structure
• Top-level files include index.html and package.json (frontend entry points), indicating a React-based application.
• The “src” directory holds the main frontend code: components (like Experience.jsx, CameraManager.jsx, chat/Chat.jsx), styles (App.css, index.css), and assets – all contributing to the 3D scene and user interface.
• A “public” directory contains 3D models (GLB, JSX representations of models) suggesting that 3D assets are served at runtime.
• The separate “proxy-server” directory, with its own package.json, .env, and proxy.js file, represents the backend part. This server handles secure API interactions with the OpenAI API.
• Configuration files (like vite.config.js, .eslintrc.cjs) indicate build and development setups.
Key Components
a. Frontend Components:
• User Interface: Built using React and React Three Fiber, which renders the 3D classroom environment.
• Components:
• Main Scene Components (e.g., App.jsx, Experience.jsx) to manage the 3D scene.
• Interaction Components like Chat (chat/Chat.jsx, Chat.css) for conversation and chat history.
• Camera Management (CameraManager.jsx) to control user navigation in the 3D space.
• Assets & Models: 3D assets stored in the “public/models” folder (classroom and avatar models).
b. Backend Components:
• Express.js Proxy Server: Located in “proxy-server”, this component securely handles API calls to external services (i.e., the OpenAI API) and likely manages middleware (evidenced by the Express.js code in proxy.js) along with environment configurations.
• Environment & Configuration: The use of an .env file in the proxy-server provides necessary credentials and configuration for the API integration.
c. External Services:
• OpenAI API: Used for providing the AI-powered teacher functionality.
• Animation/3D Tools: Blender and Mixamo, which, although external to the runtime environment, are important for creating realistic models and animations.
Relationships & Data Flow
• User Interaction Flow: The user navigates the 3D classroom (via UI components in the React app) and interacts through chat (handled by Chat.jsx).
• API Requests: The React frontend sends conversational or speech triggers that are forwarded to the backend proxy server.
• Back-End Processing: The Express.js proxy processes these requests (using secure API keys from .env) and communicates with the OpenAI API.
• Feedback Loop: The response from the OpenAI API (delivered through the proxy) returns to the frontend, where the AI teacher’s text and voice responses update the user interface and chat history.
Architectural Patterns & Design Principles
• Separation of Concerns: The project clearly separates frontend (UI, 3D rendering, user interactions) and backend (secure API proxy) responsibilities.
• Modular Design: Each React component manages a distinct part of the application’s functionality (scene setup, camera handling, chat).
• Loose Coupling: The proxy server decouples the direct interaction with the OpenAI API from the client, enhancing security and maintainability.
– Begin by sketching two major sections: the Frontend and the Backend.
– Inside the Frontend block, detail the 3D scene components (Experience.jsx, CameraManager.jsx, Chat components) and link them to asset files (models in the “public/models” folder).