# karasowl/biblioperson architecture

> Architecture diagram of the GitHub repository karasowl/biblioperson, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/karasowl/biblioperson
- Repository: https://github.com/karasowl/biblioperson

## Overview

##### Project type and purpose
- This is a full-stack personal knowledge management system (“Biblioteca de Conocimiento Personal”) with:
- A data ingestion/processing pipeline in Python
- A Flask-based REST API backend (Python + SQLite)
- A modern JavaScript frontend (Vite) for exploration and analysis

##### File structure
- Top-level:
– backend/ : Flask app, DB schema, Python scripts for import and DB maintenance
– dataset/ : standalone data-processing modules (cleaning, converters, processors)
– frontend/ : Vite-based UI (HTML, CSS, JS)
– scripts/ : convenience Python scripts (duplicate of backend scripts for ad-hoc runs)
– requirements.txt & package.json at root for bootstrapping both sides in dev
- Patterns:
– “dataset” directory handles raw data extraction and transformation
– “backend” directory exposes REST endpoints and interacts with the SQLite DB
– “frontend” directory serves the client app that consumes the API
- Config & build files:
– backend/requirements.txt, nodemon.json
– frontend/package.json, vite.config.js

##### README for additional insights
- Data sources include Facebook, Twitter, Telegram, documents
- Features: import → index → explore → generate new content
- Dev mode runs Flask on :5000 and Vite on :5173

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Sources
    EXT["External Data Sources\n(Facebook, Twitter,\nTelegram, Documents)"]:::external

    %% Data Ingestion & Processing Pipeline
    subgraph "Data Ingestion & Processing Pipeline"
        direction TB
        DSCONF["dataset/config.json"]:::pipeline
        DSDOC["dataset/documentacion.md"]:::pipeline
        subgraph "dataset/scripts"
            direction TB
            DS1["app_depuracion.py"]:::pipeline
            DS2["converters.py"]:::pipeline
            DS3["processors.py"]:::pipeline
            DS4["utils.py"]:::pipeline
        end
        subgraph "backend/scripts"
            direction TB
            BS1["importar_completo.py"]:::pipeline
            BS2["importar_datos.py"]:::pipeline
            BS3["limpiar_base_datos.py"]:::pipeline
            BS4["preparar_importacion.py"]:::pipeline
            BS5["api_conexion.py"]:::pipeline
        end
        subgraph "scripts"
            direction TB
            S1["importar_datos.py"]:::pipeline
            S2["api_conexion.py"]:::pipeline
        end
    end

    %% Persistence Layer
    DB[(SQLite Database)]:::db

    %% REST API Service
    subgraph "REST API Service (Flask)"
        direction TB
        REQ["backend/requirements.txt"]:::api
        NOD["backend/nodemon.json"]:::api
        API["api_conexion.py"]:::api
    end

    %% Frontend Application
    subgraph "Frontend Application (Vite)"
        direction TB
        FI["frontend/index.html"]:::frontend
        FP1["frontend/package.json"]:::frontend
        FP2["frontend/package-lock.json"]:::frontend
        VITE["frontend/vite.config.js"]:::frontend
        subgraph "frontend/src"
            direction TB
            FJS["app.js"]:::frontend
            FCSS["styles.css"]:::frontend
        end
    end

    %% Project-wide Config
    subgraph "Project-wide Configuration"
        direction TB
        ROOT1["requirements.txt"]:::config
        ROOT2["package.json"]:::config
    end

    %% Connections
    EXT -->|"ingest data"| DSCONF
    DSCONF -->|"run ETL"| DSDOC
    DSDOC --> DS1
    DS4 --> BS2
    BS2 --> DB
    S1 --> DB
    DB <-->|"queries"| API
    FI -->|"HTTP/JSON"| API

    %% Click Events
    click DSCONF "https://github.com/karasowl/biblioperson/blob/main/dataset/config.json"
    click DSDOC "https://github.com/karasowl/biblioperson/blob/main/dataset/documentacion.md"
    click DS1 "https://github.com/karasowl/biblioperson/blob/main/dataset/scripts/app_depuracion.py"
    click DS2 "https://github.com/karasowl/biblioperson/blob/main/dataset/scripts/converters.py"
    click DS3 "https://github.com/karasowl/biblioperson/blob/main/dataset/scripts/processors.py"
    click DS4 "https://github.com/karasowl/biblioperson/blob/main/dataset/scripts/utils.py"
    click BS1 "https://github.com/karasowl/biblioperson/blob/main/backend/scripts/importar_completo.py"
    click BS2 "https://github.com/karasowl/biblioperson/blob/main/backend/scripts/importar_datos.py"
    click BS3 "https://github.com/karasowl/biblioperson/blob/main/backend/scripts/limpiar_base_datos.py"
    click BS4 "https://github.com/karasowl/biblioperson/blob/main/backend/scripts/preparar_importacion.py"
    click BS5 "https://github.com/karasowl/biblioperson/blob/main/backend/scripts/api_conexion.py"
    click S1 "https://github.com/karasowl/biblioperson/blob/main/scripts/importar_datos.py"
    click S2 "https://github.com/karasowl/biblioperson/blob/main/scripts/api_conexion.py"
    click DB "https://github.com/karasowl/biblioperson/blob/main/backend/data/db_schema.sql"
    click REQ "https://github.com/karasowl/biblioperson/blob/main/backend/requirements.txt"
    click NOD "https://github.com/karasowl/biblioperson/blob/main/backend/nodemon.json"
    click API "https://github.com/karasowl/biblioperson/blob/main/backend/scripts/api_conexion.py"
    click FI "https://github.com/karasowl/biblioperson/blob/main/frontend/index.html"
    click FP1 "https://github.com/karasowl/biblioperson/blob/main/frontend/package.json"
    click FP2 "https://github.com/karasowl/biblioperson/blob/main/frontend/package-lock.json"
    click VITE "https://github.com/karasowl/biblioperson/blob/main/frontend/vite.config.js"
    click FJS "https://github.com/karasowl/biblioperson/blob/main/frontend/src/app.js"
    click FCSS "https://github.com/karasowl/biblioperson/blob/main/frontend/src/styles.css"
    click ROOT1 "https://github.com/karasowl/biblioperson/blob/main/requirements.txt"
    click ROOT2 "https://github.com/karasowl/biblioperson/blob/main/package.json"

    %% Styles
    classDef external fill:#97CAEF,stroke:#036,stroke-width:2px;
    classDef pipeline fill:#FFC66D,stroke:#E55,stroke-width:2px;
    classDef db fill:#8CD790,stroke:#2A4,stroke-width:2px;
    classDef api fill:#FF8A80,stroke:#A00,stroke-width:2px;
    classDef frontend fill:#D0C8FF,stroke:#552,stroke-width:2px;
    classDef config fill:#CCCCCC,stroke:#666,stroke-width:1px;
```
