How francisconeves97/jxscout works
Project Type & Purpose
• jxscout is an open-source CLI tool (written in Go) for capturing, organizing, and analyzing JavaScript assets (HTML, JS, source maps, bundles) flowing through a proxy (Burp or Caido).
• It exposes an HTTP server to receive requests, persists assets to disk/SQLite, and runs analysis modules (chunk discovery, AST analysis, beautifier, sourcemap reverse, ingestion, overrides).
High-Level Components
a. External Components
• Proxy Plugin (Burp / Caido) – forwards HTTP requests/responses to jxscout
• Bun & Prettier – JavaScript runtime & formatter for certain modules
• VSCode Extension – WebSocket client for AST visualization
b. jxscout Core
• HTTP Server (internal/core/websocket + cmd/jxscout/http.go)
• CLI/TUI Prompt (internal/core/tui + cmd/jxscout/main.go)
• Configuration & Logging (pkg/jxscout/*.go + SQLite under ~/.jxscout/)
• Persistence
– Disk storage: ~/jxscout/<project>/…
– Database: SQLite (internal/core/database)
• Event Bus
– DB eventbus (internal/core/dbeventbus)
– Core eventbus (internal/core/eventbus)
c. Processing Modules (internal/modules & pkg/*)
• Asset Fetcher & Service
• Chunk Discoverer
• Sourcemap Reverser
• Beautifier
• AST Analyzer
• HTML/JS Ingestion
• Git Committer, Overrides, Custom Ingestion, etc.
Relationships & Data Flow
1. Proxy Plugin → HTTP Server
– HTTP POST of request/response metadata & bodies
2. HTTP Server → Asset Service
– Saves raw asset to disk, records metadata in SQLite
– Emits “new asset” event on Core EventBus
3. Core EventBus → Module Orchestrator
– Listens for new‐asset events
– Dispatches jobs to various Modules (chunk‐discoverer, sourcemaps, AST analyzer, beautifier) via concurrent queues
4. Modules → Disk & Database
– Write enriched/processed artifacts back to disk (~/jxscout/...)
– Insert analysis results into SQLite
– Some modules invoke Bun (JS) or WASM code (sourcemaps) under the hood
5. CLI/TUI & VSCode Extension → Core
– CLI/TUI: Direct function calls into core services for commands (list assets, override, config)
– VSCode Extension: WebSocket client connects to internal/core/websocket to fetch AST result tree in real time
Architectural Patterns & Principles
• Event-Driven Pipeline: Core EventBus and DB EventBus decouple asset ingestion from processing modules.
• Modular Plugin Architecture: Each analysis/discovery task lives in its own internal/modules/* or pkg/* package.
• Concurrency Controls: Concurrency limits (flags) govern parallelism when fetching, beautifying, analyzing.
• Separation of Concerns: Clear layers for HTTP/API, persistence, business logic (asset service), and presentation (CLI/TUI, WebSocket).
Technologies & Frameworks
• Go standard library (HTTP server, CLI prompt)
• SQLite (via internal/core/database)
• Bun (JavaScript runtime) + Prettier for JS formatting
• WASM (sourcemaps parser)
• WebSocket (internal/core/websocket) for VSCode integration
• Concurrent queues & event buses (internal/core/eventbus, dbeventbus)