# radumarias/syncoxiders architecture

> Architecture diagram of the GitHub repository radumarias/syncoxiders, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/radumarias/syncoxiders
- Repository: https://github.com/radumarias/syncoxiders

## Overview

##### Project Type and Purpose
- The project is a cloud-based synchronization and sharing solution focused on syncing files (and emails) between various providers (Google Drive, Dropbox, S3, SFTP, etc.) and local storage, with encryption provided via rencfs.
- The README also explains that the solution is currently in a proof-of-concept phase (with CLI support) and envisions an MVP with additional features such as a browser UI, enhanced sync functionality, and future direct API integrations (with Kafka & Flink).

##### File Structure
- Multiple Rust crates indicate a modular design:
– The root includes top-level configuration files (Cargo.toml, Cargo.lock) plus GitHub workflows and configuration folders.
– Key modules include:
◦ file-change-consumer: Likely responsible for ingesting changes.
◦ file-change-router: Possibly handling the routing or decision logic of change events.
◦ file-tree-merge: Contains logic for merging changes from different sources, conflict resolution, and managing directory trees.
◦ file-watcher: Likely monitors file system events and triggers syncs.
- The presence of tests (including shell scripts in file-tree-merge/tests) shows that there is a focus on reliable behavior via integration and script-based system tests.

##### Main Components
- Frontend & User Interaction:
- CLI application (as per the README “syncoxiders” binary command) and a planned browser UI.
- Input/Change Detection:
- File Watcher: Monitors local filesystem changes.
- File Change Consumer: Consumes detected events for further processing.
- Coordination and Routing:
- File Change Router: Determines how and where to route changes (e.g., local folder to cloud, inter-cloud routing).
- Core Processing Engine:
- File Tree Merge Module: Contains the algorithms to merge file trees, resolve conflicts, and apply necessary changes.
- External Integrations (Adapters):
- Cloud Providers: Currently relying on third-party tools/libraries such as rclone, gdrive-rs, and using rencfs for encryption.
- Future plans include developing native clients for these providers.
- Messaging & Future Data Pipeline:
- The envisioned incorporation of Kafka (for change streaming) and Flink (for windowing/grouping) to feed merged changes into the system.

##### Relationships and Interactions
- The user initiates an action (via CLI or later via the browser app), which triggers the file watcher and/or file change consumer.
- The file change consumer routes events to the file change router that then categorizes the event type (create, modify, delete, move) for further processing.
- The file tree merge module then gets involved to compare the current file state with previous snapshots, resolve conflicts, and compute a unified changes tree.
- Once the merge is complete, the system pushes the changes to the respective targets (local files and/or remote cloud storages).
- External services (rclone, rencfs, gdrive-rs) are used to interact with the storage providers, with encryption happening as data flows between cloud and local systems.
- Optionally, future components will integrate real-time messaging (Kafka) to manage change propagation across distributed systems.

##### Specific Architectural Patterns and Key Points
- Modularity: The use of separate Rust crates implies a microservice-like or modular monolith design. Each major function (watching, routing, merging) is a separate module that can be developed and tested independently.
- Data Flow and Event-Driven Design: The file-change consumer and router suggest an event-driven design.
- Extensibility: The architecture leaves room for plugging in new external providers and for evolving the way changes are processed (moving from rclone-run solutions to custom API clients).
- Future Scaling: The inclusion of Kafka and Flink in the planned MVP indicates a design that is prepared to scale out using robust messaging and stream processing frameworks.

##### Final Recommendations for the Principal Software Engineer
- Parts where future improvements (like Kafka/Flink integrations) are planned, so the design remains flexible.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Interface Layer
    subgraph "User Interface Layer"
        UI1["CLI Application"] 
        UI2["Browser UI (Planned)"]
    end

    %% Documentation
    subgraph "Documentation"
        DOC["User Documentation"]
    end

    %% Input/Events Layer
    subgraph "Input/Events Layer"
        FW["File Watcher"]
        FCC["File Change Consumer"]
    end

    %% Core Processing Layer
    subgraph "Core Processing Layer"
        FCR["File Change Router"]
        FTM["File Tree Merge Module"]
    end

    %% External Integrations
    subgraph "External Integrations"
        CA["Cloud Adapters<br/>(rclone, gdrive-rs, etc.)"]
        ENC["Encryption<br/>(rencfs)"]
        FP["Future Messaging Pipeline<br/>(Kafka/Flink)"]
    end

    %% Connections
    UI1 -->|"triggers"| FCC
    UI2 -->|"triggers"| FCC
    FW -->|"detects changes"| FCC
    FCC -->|"routes event"| FCR
    FCR -->|"processes event"| FTM
    FTM -->|"syncs files"| CA
    FTM -->|"applies encryption"| ENC
    FP -->|"feeds into"| FTM
    DOC -->|"informs about"| UI1

    %% Click Events
    click FW "https://github.com/radumarias/syncoxiders/blob/main/file-watcher/src/main.rs"
    click FCC "https://github.com/radumarias/syncoxiders/blob/main/file-change-consumer/src/main.rs"
    click FCR "https://github.com/radumarias/syncoxiders/blob/main/file-change-router/src/main.rs"
    click FTM "https://github.com/radumarias/syncoxiders/tree/main/file-tree-merge/"
    click DOC "https://github.com/radumarias/syncoxiders/tree/main/website/"

    %% Styles
    classDef ui fill:#cfe2f3,stroke:#000,stroke-width:2px;
    classDef event fill:#f4cccc,stroke:#000,stroke-width:2px;
    classDef core fill:#d9ead3,stroke:#000,stroke-width:2px;
    classDef external fill:#fff2cc,stroke:#000,stroke-width:2px;
    classDef docs fill:#ead1dc,stroke:#000,stroke-width:2px;
    
    class UI1,UI2 ui;
    class FW,FCC event;
    class FCR,FTM core;
    class CA,ENC,FP external;
    class DOC docs;
```
