How singo01/app-miner works
Main Components
a. Input & Data Sources:
• Source bitcode files that are obtained (e.g., from cloning the Linux kernel in deadline/code/srcs/).
• That these LLVM bitcode files serve as the raw input to the system.
b. Preprocessor/Analyzer:
• Preprocessor component found under “preprocessor/analyzer/”.
• That this module (built via make) is a modified version of the “Crix” tool and prepares the bitcode for further analysis.
c. Deadline Analysis Module:
• “deadline” directory as a core analysis engine subdivided into several areas:
– “deadline/code”: Manages different stages and outputs from bitcode handling (subdirectories like bcfs, bins, srcs, etc.).
– “deadline/pass” (especially the KSym submodule): This is where many LLVM passes and symbolic analysis routines (using components like Alias, Fetch, Logger, Unroll, etc.) are implemented.
– “deadline/deps”: Encompasses essential libraries like Z3 and additional utilities required for constraint solving and analysis.
– “deadline/llvm”: Prepares the environment or handles initialization aspects of LLVM-based operations.
– “deadline/unit” and “deadline/work”: Represent testing and orchestration scripts (e.g., main.py, app.py, util scripts) that drive the analysis pipeline.
• That the Deadline module carries out heavy lifting in terms of static analysis, LLVM pass execution, and symbolic evaluation.
d. APP-Miner Coordinator:
• “app-miner” (with main.py and its own README) as the orchestration layer.
• This module serves as the interface for users to configure and start the analysis process (providing configuration commands, build commands, etc.) and to eventually direct the flow to process results.
• It aggregates the analysis results (saved in directories like app-miner/result/<target>), including output XML files containing the reported misuses.
Relationships and Data Flow
– One branch can go towards the “deadline/code” (managing file flows and intermediate representations).
– A separate branch can represent the symbolic analysis and LLVM pass execution (within “deadline/pass/KSym”).
• Finally, have the “APP-Miner Coordinator” component (from “app-miner/main.py”) orchestrate the entire workflow:
– It should be depicted as invoking both the preprocessor and deadlines modules.
Architectural Patterns and Technologies
• That the system follows a pipeline architecture where data flows through discrete stages: data ingestion → preprocessing → analysis (static & symbolic) → result aggregation.
• Underneath, various technologies are involved:
– Python scripts (in app-miner and deadline/work) for orchestration.
– C/C++ modules implementing LLVM passes and symbolic routines (in deadline/pass/KSym).
– Docker is used for containerizing the build and runtime environment.
– External libraries such as Z3 are integrated for constraint solving.