# programming-paradox/fadex architecture

> Architecture diagram of the GitHub repository programming-paradox/fadex, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/programming-paradox/fadex
- Repository: https://github.com/programming-paradox/fadex

## Overview

##### Main Components Identification

The system consists of three main layers:
- Python Interface Layer
- Rust Core Layer
- External Web Resources

##### Key Architectural Elements

a. Core Components (based on src/ directory):
- Crawler Module (crawler.rs)
- Parser Module (parser.rs)
- Library Interface (lib.rs)

**┌─────────────────────┐**
- Python Interface  │
- (User-facing API)  │
- ┘

**┌─────────────────────┐**
- PyO3         │
- (Python-Rust      │
- Bridge)          │
- ┘

**┌─────────────────────────────────────┐**
- Rust Core                 │

- ┌─────────┐  ┌─────────┐          │
- Crawler │  │ Parser  │          │
- ┘  └─────────┘          │

- ┘

**┌─────────────────────┐**
- Web Resources    │
- (HTTP/HTTPS)     │
- ┘

##### Flow Directions

- User Python code → Python Interface
- Python Interface → PyO3 Bridge
- PyO3 Bridge → Rust Core
- Crawler → Web Resources
- Web Resources → Parser
- Parser → PyO3 Bridge
- PyO3 Bridge → Python Interface

##### Component Details

Annotations for key functionalities:
- Crawler: Async web page fetching
- Parser: HTML parsing and extraction
- PyO3: FFI binding and type conversion
- Python Interface: High-level API methods

##### Performance Flow

A separate section showing the performance pipeline:
**Web Request → Async Fetch → Rust Processing → Python Results**

##### Key Features to Highlight

- Asynchronous operations
- Memory safety (Rust)
- High-performance parsing
- Python integration
- Error handling flow

##### Implementation Notes

- The separation between Python and Rust layers
- The role of PyO3 as a bridge
- The async nature of web requests
- The data transformation pipeline

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Define styles
    classDef python fill:#3776AB,color:white
    classDef rust fill:#F74C00,color:white
    classDef bridge fill:#4CAF50,color:white
    classDef external fill:#808080,color:white
    
    %% Legend
    subgraph Legend
        P[Python Component]:::python
        R[Rust Component]:::rust
        B[Bridge Component]:::bridge
        E[External Component]:::external
    end

    %% Python Layer
    subgraph PythonLayer[Python Interface Layer]
        PythonAPI[Python Interface]:::python
    end

    %% Bridge Layer
    subgraph BridgeLayer[FFI Bridge Layer]
        PyO3[PyO3 Bridge]:::bridge
    end

    %% Rust Core Layer
    subgraph RustCore[Rust Core Layer]
        LibInterface[Library Interface]:::rust
        subgraph CoreComponents[Core Components]
            Crawler[Crawler Module]:::rust
            Parser[Parser Module]:::rust
        end
    end

    %% External Resources
    subgraph External[External Layer]
        WebResources[Web Resources]:::external
    end

    %% Connections
    PythonAPI --> PyO3
    PyO3 --> LibInterface
    LibInterface --> Crawler
    LibInterface --> Parser
    Crawler --> WebResources
    WebResources --> Parser
    Parser --> PyO3
    PyO3 --> PythonAPI

    %% Click events for component mapping
    click Crawler "https://github.com/programming-paradox/fadex/blob/master/src/crawler.rs"
    click Parser "https://github.com/programming-paradox/fadex/blob/master/src/parser.rs"
    click LibInterface "https://github.com/programming-paradox/fadex/blob/master/src/lib.rs"
    click PythonAPI "https://github.com/programming-paradox/fadex/blob/master/fadex_lxml_results.py"
    click PyO3 "https://github.com/programming-paradox/fadex/blob/master/Cargo.toml"

    %% Notes
    note1[Async web page fetching]
    note2[HTML parsing and extraction]
    note3[FFI binding and type conversion]
    note4[High-level API methods]

    Crawler --- note1
    Parser --- note2
    PyO3 --- note3
    PythonAPI --- note4
```
