How beginnercoder52/asl_website works
Project type and purpose
• This is a full-stack web application for real-time American Sign Language (ASL) recognition.
• It consists of three major parts:
• An AI training and model-conversion pipeline (Python/Keras → TensorFlow.js).
• A backend REST API server (Node.js/Express) that hosts the converted model files and exposes inference endpoints (and potentially persists usage data).
• A React-based single-page frontend that captures live video from the user’s camera, calls the backend API for predictions, and displays results.
File structure
Top level:
• AI/ – Python scripts, dataset folders (A–Z, 0–9), Keras model, preprocessing and training code, plus a conversion script to TFJS.
• backend/ –
– models/asl_models/ (model.json + weights.bin delivered by convert_to_tfjs.py)
– package.json & requirements.txt (Node and any Python/other deps)
– src/db.js (database connection), src/routes.js (Express routes), src/server.js (HTTP server)
• frontend/ –
– public/index.html
– src/ (React components CameraFeed, PredictionDisplay, App.js, index.js)
– Tailwind CSS config
README insights
(Empty in this case – structure speaks for itself.)
a. Offline AI pipeline
• Data preprocessing (preprocess.py)
• Model training (train.py → asl_model.keras)
• Model conversion (convert_to_tfjs.py → model.json + weights.bin)
• Artifact storage (AI/asl_model.keras and AI/data for versioning)
b. Backend services
• Express server (server.js)
• Routing layer (routes.js) – e.g., POST /predict
• Database connector (db.js) – logs, user info, etc.
• Static hosting of TFJS model assets (models/asl_models/)
c. Frontend UI
• CameraFeed component captures frames
• PredictionDisplay shows labels & confidence
• API client inside App.js calls backend → receives JSON prediction
• Tailwind for styling
Relationships & Data flow
1. Offline: raw images → preprocess.py → train.py → Keras model → convert_to_tfjs.py → tfjs artifacts → check into backend/models
2. Runtime:
User Camera → Frontend Component → HTTP Request (/predict) → Backend routes.js → loads TFJS model → runs inference → (optionally logs to DB via db.js) → returns JSON → Frontend updates UI
Architectural patterns & principles
• Layered separation:
Presentation (React) ↔ API (Express) ↔ Inference engine (TFJS) ↔ Data store (DB)
• Single Responsibility: each folder handles one concern (AI, backend, frontend).
• Stateless API: inference calls carry image data, server does not maintain session state except via the DB if needed.
Relevant technologies
• Python/Keras + TensorFlow for training
• TensorFlow.js for browser/server inference
• Node.js + Express for REST endpoints
• React + Tailwind CSS for SPA frontend
• Any DB (configured in db.js) – e.g., MongoDB or PostgreSQL
Summary
• Break the system into AI pipeline, backend API, frontend UI, and datastore.