# facebook/pyrefly architecture

> Architecture diagram of the GitHub repository facebook/pyrefly, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/facebook/pyrefly
- Repository: https://github.com/facebook/pyrefly

## Overview

##### Project Type and Purpose
- Pyrefly is a multi‐platform, full‐stack static type checker and IDE for Python.
- It consists of a core type‐checking engine in Rust (library and CLI), a Language Server Protocol (LSP) extension for editors (VSCode), a WebAssembly build for in‐browser sandboxing, a Python‐package wrapper (pip), a Docusaurus‐based documentation website, and a conformance‐test suite.

##### File‐Structure Analysis
- Top‐level Rust crates under `pyrefly/`, `pyrefly_wasm/`, `pyrefly_derive/`, `pyrefly_util/`.
- LSP extension in `lsp/` (TypeScript + Node.js).
- Documentation site in `website/` (Docusaurus, React).
- Conformance tests in `conformance/` and `test/`.
- CI/CD workflows under `.github/workflows/` for building binaries, publishing to PyPI, deploying website and extension.

##### Key Components
a. Core Engine (Rust crates)
- pyrefly (binding, graph, solver, state, module, config, commands)
- pyrefly_util (utility crate)
- pyrefly_derive (derive macros)
b. CLI & Python Wrapper
- `pyrefly/bin/main.rs` (CLI front-end)
- `pyrefly.dotslash.py` & `pyproject.toml` (Python pip package)
c. Language Server & Editor Extension
- LSP server hook in `pyrefly` (command: `lsp.rs`)
- VSCode extension in `lsp/src/extension.ts`
d. Web Frontend & WASM Sandbox
- pyrefly_wasm crate → Compiles engine to WebAssembly
- `website/src/sandbox/*` → React components + `pythonWorker.ts`
- Docusaurus site in `website/` (docs, sandbox launcher)
e. Conformance Test Suite
- Python scripts in `conformance/` driving the engine
- `scripts/fetch_typeshed.py` and `test/glean/` JSON fixtures
f. CI/CD Pipelines
- GitHub Actions flows for building/testing/deploying all artifacts

##### Relationships and Data Flows
- The Core Engine (Rust crates) exposes an API used by three “clients”:
1. CLI & Python Wrapper (invoked from shell or `pip install pyrefly`)
2. LSP Server (spawned by VSCode extension)
3. WASM Module (embedded in the website sandbox)
- The VSCode Extension communicates over the LSP protocol to a server process that delegates to the engine.
- The Website loads the WASM module into the browser sandbox; user code is type‐checked in-browser.
- Conformance tests feed Python snippets to the CLI or engine API and verify JSON/console outputs.
- CI/CD orchestrates builds: Rust binaries, Python wheels, WASM artifacts, npm packages, and documentation deployment.

##### Architectural Patterns and Design Principles
- Modular “crate” structure in Rust for separation of concerns (binding, solving, state management).
- Command pattern for CLI commands (`init`, `check`, `run`, `lsp`).
- Single‐module incrementality: each Python module is fully bound and solved in isolation, then combined.
- Polyglot clients: the same engine library supports native CLI, LSP, WebAssembly, and Python binding.
- Conventional separation of front‐end (website + VSCode UI) and back‐end (engine).

##### Technologies, Frameworks, and Libraries
- Rust & Cargo (core engine, CLI, WASM)
- wasm-pack / wasm-bindgen (WASM build)
- Python packaging (pip, `pyproject.toml`)
- Node.js, TypeScript, ESLint (VSCode extension)
- Docusaurus, React (documentation site)
- GitHub Actions (CI/CD)
- Third-party typeshed stubs under `third_party/typeshed` for standard library typing

##### Detailed Capture of Essential Elements
- Multi‐client architecture (CLI, LSP, WASM, Python).
- Website both as documentation and as host for the sandbox.
- CI/CD flows driving artifact production and publishing (PyPI, npm, website host).
- Conformance test subsystem invoking the engine via Python scripts.
- Every component from the file tree (crates, directories) is placed in context to reflect actual dependencies.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Core Engine Rust Crates
    subgraph "Core Engine (Rust Crates)"
        direction TB
        PyreflyCore["pyrefly"]:::rust
        PyreflyUtil["pyrefly_util"]:::rust
        PyreflyDerive["pyrefly_derive"]:::rust
    end

    %% CLI & Python Wrapper
    subgraph "CLI & Python Wrapper"
        direction TB
        CLI["pyrefly/bin/main.rs"]:::rust
        PyProject["pyrefly/pyproject.toml"]:::python
        DotSlash["pyrefly/pyrefly.dotslash.py"]:::python
    end

    %% Language Server & VSCode Extension
    subgraph "LSP & VSCode Extension"
        direction TB
        LSPServer["lsp.rs (pyrefly/lib/commands)"]:::rust
        VSCodeExt["extension.ts (lsp/src/extension.ts)"]:::ts
    end

    %% WASM & Web Sandbox
    subgraph "Web Frontend & WASM Sandbox"
        direction TB
        WASM["pyrefly_wasm"]:::rust
        SandboxUI["Sandbox.tsx"]:::web
        PythonWorker["pythonWorker.ts"]:::web
        WasmJS["pyrefly_wasm.js"]:::web
    end

    %% Documentation Site
    subgraph "Documentation Website"
        direction TB
        Docusaurus["Docusaurus Site"]:::web
        Docs["Docs (*.mdx)"]:::web
        Components["Components & Pages"]:::web
    end

    %% Conformance Test Suite
    subgraph "Conformance Test Suite"
        direction TB
        ConformanceHarness["conformance harness"]:::python
        FetchTypeshed["scripts/fetch_typeshed.py"]:::python
        TestFixtures["test/glean/*.json & *.py"]:::python
    end

    %% CI/CD Pipelines
    subgraph "CI/CD Pipelines"
        direction TB
        CI["GitHub Actions"]:::ci
    end

    %% Relationships
    PyreflyCore -->|"exposes API"| CLI
    PyreflyCore -->|"LSP protocol"| LSPServer
    LSPServer <-->|"communicates via LSP"| VSCodeExt
    PyreflyCore -->|"WebAssembly calls"| WASM
    WASM -->|"loaded in"| SandboxUI
    WASM -->|"worker script"| PythonWorker
    WASM -->|"JS bindings"| WasmJS
    ConformanceHarness -->|"runs tests against"| PyreflyCore
    FetchTypeshed -->|"provides stubs to"| ConformanceHarness
    TestFixtures -->|"used by"| ConformanceHarness
    CI -->|"builds & tests"| PyreflyCore
    CI -->|"publishes pip"| PyProject
    CI -->|"builds WASM"| WASM
    CI -->|"builds extension"| VSCodeExt
    CI -->|"deploys site"| Docusaurus

    %% Click Events
    click PyreflyCore "https://github.com/facebook/pyrefly/blob/main/pyrefly/Cargo.toml"
    click PyreflyUtil "https://github.com/facebook/pyrefly/blob/main/pyrefly_util/Cargo.toml"
    click PyreflyDerive "https://github.com/facebook/pyrefly/blob/main/pyrefly_derive/Cargo.toml"
    click CLI "https://github.com/facebook/pyrefly/blob/main/pyrefly/bin/main.rs"
    click PyProject "https://github.com/facebook/pyrefly/blob/main/pyrefly/pyproject.toml"
    click DotSlash "https://github.com/facebook/pyrefly/blob/main/pyrefly/pyrefly.dotslash.py"
    click LSPServer "https://github.com/facebook/pyrefly/blob/main/pyrefly/lib/commands/lsp.rs"
    click VSCodeExt "https://github.com/facebook/pyrefly/blob/main/lsp/src/extension.ts"
    click WASM "https://github.com/facebook/pyrefly/blob/main/pyrefly_wasm/Cargo.toml"
    click SandboxUI "https://github.com/facebook/pyrefly/blob/main/website/src/sandbox/Sandbox.tsx"
    click PythonWorker "https://github.com/facebook/pyrefly/blob/main/website/src/sandbox/pythonWorker.ts"
    click WasmJS "https://github.com/facebook/pyrefly/blob/main/website/src/sandbox/pyrefly_wasm.js"
    click Docusaurus "https://github.com/facebook/pyrefly/blob/main/website/docusaurus.config.ts"
    click Docs "https://github.com/facebook/pyrefly/tree/main/website/docs/"
    click Components "https://github.com/facebook/pyrefly/tree/main/website/src/components/"
    click ConformanceHarness "https://github.com/facebook/pyrefly/blob/main/conformance/README.md"
    click FetchTypeshed "https://github.com/facebook/pyrefly/blob/main/scripts/fetch_typeshed.py"
    click TestFixtures "https://github.com/facebook/pyrefly/tree/main/test/glean/"
    click CI "https://github.com/facebook/pyrefly/blob/main/.github/workflows/build_and_test_website.yml"

    %% Styles
    classDef rust fill:#ADD8E6,stroke:#000,stroke-width:1px
    classDef python fill:#90EE90,stroke:#000,stroke-width:1px
    classDef ts fill:#D8BFD8,stroke:#000,stroke-width:1px
    classDef web fill:#FFDAB9,stroke:#000,stroke-width:1px
    classDef ci fill:#D3D3D3,stroke:#000,stroke-width:1px
```
