How galis/opentiktok works
• The “app” module, which contains the Android front-end, including activities, layout resources, and UI logic.
• The “avcore” module, which contains the video/audio processing engine. This module uses native C++ code (with shader files, JNI bridges, and build scripts for CMake) and integrates external libraries (like OpenCV, libyuv, and GLM) to implement filters, rendering effects, frame manipulation, and other core editing functions.
Overall Structure
a. Two main layers:
– The UI/Presentation Layer (from “app”) that handles user interaction, preview, and editing controls.
– The Core Video Processing Layer (from “avcore”) that includes lower-level native processing, filter application, video composition, and JNI bridging.
Main Components
• User Interface (UI):
– Activities and UI components (e.g., VideoEditActivity, VideoExportActivity, LogoActivity, and various custom views).
– Layouts and resources (layouts, menus, drawable assets).
• Application Logic:
– Java helper classes (e.g., BaseActivity, utility classes contained in “app/src/main/java/com/galix/opentiktok/ui”).
• Native Video Editing/Processing Engine:
– JNI Bridge Layer (files under avcore/src/main/cpp/jni and corresponding Java wrappers in avcore/src/main/java/com/galix/avcore/avcore).
– Core Processing Modules:
• AVEngine, AVFrame, AVVideo, AVSync, etc. (classes handling audio/video processing and composition).
– Rendering and Filter Modules:
• Submodules under “avcore/src/main/java/com/galix/avcore/render” (e.g., filters like BeautyFilter, LutFilter, OesFilter, PagFilter, etc.).
• Native C++ Code:
– Core implementation files and shader resources (located in avcore/src/main/cpp, including the shader files in res/raw for different effects).
• External Libraries & Dependencies:
– OpenCV, libyuv, GLM, and others (visible from the extensive include directories and integration code).
• Build and Configuration:
– Gradle build scripts and CMakeLists.txt, which direct how the Java and native layers are compiled and integrated.
• Assets:
– Video test files, LUTs, animation files, etc. (present in app/src/main/assets and app/src/main/res).
Relationships and Interactions
• That user interactions in the UI trigger events in the application layer that call into the AVCore’s Java wrappers.
• That these Java components call the JNI methods to invoke native C++ processing functions.
• Integration boundaries:
• How external libraries (like OpenCV) are “wrapped” inside the native processing engine, and how shader code is employed to implement rendering effects.
Architectural Patterns and Principles
• Modular design: Separation between UI and processing layers.
• Encapsulation: The AVCore module provides clean interfaces (via JNI and Java wrappers) to hide native implementation details.
• Use of a layered approach:
– Presentation layer (Android activities, layouts)
– Application logic (Java classes managing user actions)
– Processing layer (native video/audio processing, filter pipelines)
• Integration and dependency inversion: The UI layer depends on abstractions provided by AVCore, which in turn relies on lower-level native libraries.
• Link this block to an “AVCore Engine” block that further breaks down into:
– “Java Wrapper & AV Interface” (handling calls from the app)
– “Native C++ Processing Core” (filtering, compositing, video rendering, and shader management)
• A side component for “External Libraries” that feeds into the native layer.
• Every edge has labels showing data flow (e.g., “User Command”, “Frame Data”, “Processing Results”) and every component is clearly delineated.