How tuhinshubhra/extanalysis works
Project Type and Purpose
• ExtAnalysis is a full-stack analysis tool that focuses on downloading, analyzing, and scanning browser extensions (Chrome, Firefox, Opera, etc.) for details such as manifest information, permissions, embedded URLs, vulnerabilities (including using RetireJS for JavaScript files), and even integrates VirusTotal scans.
• It provides both web-based visualization (via a Flask webserver) and command-line usage. The emphasis is on analyzing extension files (like .crx, .xpi, and .zip) and presenting results on a friendly user interface.
File Structure for Architectural Clues
• Top-level files like Dockerfile, docker-compose.yml, requirements.txt, and settings.json point towards containerized deployments and Python dependency management.
• The core directory contains main functional modules (e.g., analyze.py, downloader.py, scans.py) that likely implement most of the business logic for extension analysis.
• The frontend directory holds API definitions and view code (like viewfile.py, viewgraph.py) suggesting a Flask-based API that renders templates and sends data to the client browser.
• The plugins directory is used for extensibility (e.g., retirejs.py and vulnerabilities.py) giving hooks for additional analysis features.
• The static and templates directories indicate a web UI component where static resources (CSS, JavaScript, images) and HTML templates form the user interface.
• The db directory, containing the GeoIP database and permission configurations, hints at external data sources that are utilized during analysis.
• Additional files (e.g., extanalysis.py) act as an entry-point to the application, invoking both the core and frontend functionalities.
a. Main Components:
• User Interface: Represented by the static resources and HTML templates in the “templates” and “static” directories.
• Web Server (Frontend): The Flask application contained in the frontend directory that handles incoming HTTP requests (API endpoints and views).
• Core Analysis Engine: The “core” modules that implement the business logic including downloading, scanning, analyzing files, and processing settings.
• Plugins Module: Extensible plugins for specialized functionality such as vulnerability scanning and RetireJS analysis.
• Database/Local Data: Files under “db” (GeoIP database, permissions JSON) serve as local data resources used by the analysis engine.
• External Services: Integrations with external APIs such as VirusTotal for scans, WhoIs for domain info, and possibly web resources for downloading extensions.
b. Relationships and Data Flow:
• The User accesses the system via a web browser; the browser communicates with the Flask-based Web Server (Frontend).
• The Web Server receives requests (to analyze an extension, view a manifest, etc.) and calls upon the Core Analysis Engine.
• The Core Analysis Engine processes the extension files—invoking modules in core and, where applicable, handing off tasks to Plugins.
• When needed, the Core Analysis Engine queries the Local Data (from the db folder) and external services (like VirusTotal or WhoIs lookups).
• Results are pushed back through the frontend to be rendered on the User Interface. Static assets support the interactive and visually rich UI (like network graphs, chiptune player, etc.).
c. Architectural Patterns and Design Principles:
• Separation of Concerns: UI (frontend) is clearly separated from business logic (core) and extension points (plugins).
• Extensibility: The plugins architecture allows adding new scan routines or analyzing additional file types.
• Micro-modular design: Different modules (e.g., downloader, analyzes, network scanning) are encapsulated into individual files.
• Containerization: Dockerfile and Docker Compose configurations facilitate easy deployments and consistent environments.
d. Technologies and Frameworks:
• Python3 is the base programming language.
• Flask: Forms the backbone of the webserver in the frontend.
• External Python Modules: python-whois, maxminddb, and requests are used for external and network interactions.
• Frontend Libraries: The static/js directory includes libraries such as jQuery, Codemirror, and tools for rendering interactive graphics (graph.js).
• Docker: Ensures containerized deployment.
Final Elements
• Major components are clearly labeled. For instance "User Browser", "Flask Web Server (Frontend)", "Core Analysis Engine", "Plugins Module", "Local Data (db)", "External Services".