How virustotal/yara works
Main Components
• Command-Line Interface (CLI): Represented by the “cli” directory. This is the entry point for user interactions where command-line options are parsed (files such as cli/yarac.c, cli/yara.c, etc.).
• Core Library (libyara): The heart of YARA, found in the “libyara” directory. This includes components for rule compilation, matching, scanning, and rule execution (e.g., compiler.c, rules.c, scanner.c, exec.c). Also, note that this component includes many submodules (modules/) that extend functionality for different file formats (PE, ELF, Mach-O, dotnet, etc.).
• Modules: Located under “libyara/modules”, these represent extensibility points that handle various file types and specialized scanning logic (for example, modules/magic, modules/hash, modules/pe, etc.).
• Sandbox: Contains isolated or experimental components (in the “sandbox” directory) which include alternate entry points and testing variations (e.g., sandbox/collect_matches.cc). This can be treated as an isolated environment to exercise secure or restricted rule execution.
• Build and Configuration Systems: The presence of Bazel build files (BUILD.bazel, WORKSPACE.bazel, and the “bazel” directory), Makefile.am, configure.ac, and scripts (bootstrap.sh, build.sh) means that the build infrastructure is a significant component. While not part of the runtime architecture, you may optionally include an “Infrastructure” layer detailing how the source is built and tested.
• Tests & CI: With directories “tests” and CI files in “.github/workflows”, these support quality assurance and continuous integration, helping validate the scanning engine and rule execution.
• Python Integration: Although not a separate directory, the README mentions yara-python, which implies an additional Python binding layer that wraps around the core library for integration with Python scripts.
Relationships and Interactions
• The CLI component interacts directly with the core library. When a user runs a YARA command, the CLI reads rule files and passes them to the libyara compiler.
• Libyara handles rule compilation, then leverages various helper functions and the rule matching engine to scan files. Internally, it communicates with individual modules (invoked as needed) to process specific file types.
• The modules are dynamically invoked by libyara based on the contents of the rules and files being scanned. They are part of the extension mechanism.
• Data flow typically starts from an input file (or set of files) – processed by the CLI, passed to the rule compiler, then into the matching engine, with results flowing back to the UI.
• Optional interactions include hooking into Python scripts via yara-python for in-process scanning.
Architectural Patterns and Design Principles
• Separation of Concerns: There is a clear separation between the interface (CLI), core execution engine (libyara), modular extensions (modules), and testing/infrastructure.
• Modularity & Extensibility: The design supports loading modules for specialized scanning, facilitating easier updates and extension.
• Multi-Platform Build & Deployment: With multiple build files (Bazel, Makefiles, Visual Studio projects under “windows”), the project emphasizes cross-platform compatibility.
• Layered Architecture: The system can be conceptually divided into a user interaction layer (CLI), processing layer (libyara and its modules), and build/test infrastructure.
Technologies, Frameworks, and Libraries
• Languages: Primarily C (as indicated by .c and .h files) with some build configuration in shell scripts, Bazel, and Makefiles.
• Build Tools: Bazel, Make, autoconf; Visual Studio solutions indicate support for Windows-specific builds.
• CI/CD: GitHub Workflows, Appveyor, and Coverity integration.
• Documentation: Sphinx-based documentation (docs/conf.py) and ReadTheDocs configuration.
Elements
• “CLI Interface” (cli/)
• “Core Rule Engine & Compiler” (libyara)
• “Module Loader/Extensions” (libyara/modules)
• “Sandbox Environment” (sandbox)
• “Testing & CI Infrastructure” (tests, .github/workflows, CI badges)
• “Build & Configuration Layer” (Bazel files, Makefiles)
• “Python Integration” (yara-python, as referenced in the README)
• Core business logic (e.g., solid rectangles for libyara)
• Infrastructure/tools (possibly shaded or in a separate section)
• External interfaces (like Python or external CI/CD tools).