How tangweichiang/fyp works
Project Type & Purpose
• This is an Android mobile application called “FYP-286841 SignLearn” that appears to focus on sign language learning, translation, and recognition.
• The project integrates machine learning models (in TensorFlow Lite format in the assets) for sign language recognition and utilizes an OpenCV library module for image processing.
• It also has several user-oriented pages including tutorials, quizzes, a translation activity, and standard authentication flows (login, register, forgot password).
Analyzing the File Structure
• Source code (app/src/main/java) containing many activities (CameraActivity, MainActivity, TranslatorActivity, etc.), fragments (RecognitionFragment, TutorialFragment, QuizFragment), and helper classes for sign language detection and quizzes.
• Resources including layouts (XML files under app/src/main/res/layout), drawable resources, fonts, and raw data (e.g., JSON for questions).
• Assets that include a variety of TensorFlow Lite models (e.g., sign_model.tflite, signlearn_model.tflite) and label files, which hints at on-device inference for gesture recognition.
• Native libraries inside “app/src/main/jnilibs” that support performance-sensitive operations.
• A separate module “openCVLibrary3413” is included, containing its own gradle script and source code which clearly provides opencv support for advanced image processing. This module is meant to be integrated with the app to perform tasks that likely precede or complement the ML-based sign language recognition.
Key System Components
• User Interface (UI) Layer:
– Includes Android activities (MainActivity, CameraActivity, TranslatorActivity, etc.) and fragments (RecognitionFragment, TutorialFragment, QuizFragment).
– Responsible for capturing user interactions, camera input, and displaying tutorial/quizzes.
• Business Logic & Controller Layer:
– Embedded within the activities and helper classes where the main processing is done (e.g., handling the object detection, gesture processing, and sign language recognition logic).
• ML Inference and Assets:
– Contains TensorFlow Lite models and label files stored in the assets folder. This is where the machine learning computation for sign language detection happens.
• External/Native Libraries:
– The OpenCV module that processes camera input images.
– Native libraries (via JNI) used for performance-critical image processing tasks.
• Authentication & User Management:
– A set of activities for user login, registration, password recovery, etc.
• Supporting Infrastructure:
– Gradle scripts and configuration files that set up dependencies, including linking the OpenCV module with the main application.
Relationships and Interactions
• The UI layer (activities and fragments) interacts with the business logic to capture camera input and user commands.
• On receiving input (say from the camera), control flows to:
– The OpenCV module (via JNI or directly if integrated as a module) to preprocess or enhance image data.
– Then, the processed images are fed into the machine learning inference engine using the TensorFlow Lite models from the assets.
• Results from ML inference can then trigger actions in the UI (for example, updating a quiz, triggering a tutorial, or displaying the sign language translation).
• The authentication screens manage user sessions and may interact with external services if authentication is offloaded (even if not explicit in the file tree, the presence of multiple activities shows that security and user state are managed).
Recommended Elements
– UI Layer (Activities/Fragments)
– Business Logic/Controller (e.g., gesture recognition, translation handlers)
– ML Inference Engine (TensorFlow Lite models, asset management)
– External Library Modules (OpenCV module, native code integration)
– Authentication Module
– Native/External modules (OpenCV and JNI-libraries)
– Core Android components (activities, fragments)
– Assets/Model components.