# yusufkaraaslan/skill_seekers architecture

> Architecture diagram of the GitHub repository yusufkaraaslan/skill_seekers, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yusufkaraaslan/skill_seekers
- Repository: https://github.com/yusufkaraaslan/skill_seekers

## Overview

##### Project type and purpose
- This is a Python-based CLI tool with an optional “MCP server” component. It is not a classical web app but a developer tool that automates:
- Scraping HTML/PDF documentation websites
- Organizing and AI-enhancing content
- Packaging it into a Claude AI “skill” zip file
- Optionally uploading it via an API or manual process
- The README confirms it’s a full-featured ETL + AI-pipeline tool, with both CLI scripts (`cli/`) and an MCP-compatible micro-service (`mcp/`).

##### File structure
- Top‐level directories:
- cli/ – all entry-point scripts (scraper, enhancer, packager, uploader, utils)
- mcp/ – a small server implementing Model Context Protocol tools for Claude Code integration
- configs/ – JSON presets for different frameworks (React, Django, Godot, etc.)
- docs/ – extensive user and technical documentation
- tests/ – unit and integration tests
- Supporting files: setup scripts, requirements, GitHub workflows, README, CONTRIBUTING, etc.
- No traditional database: persistent state is file‐system based (output/ cache, checkpoints).

##### README for additional insights
- Key external dependencies: websites or PDFs, Claude Code locally (via MCP) or Anthropic API, optional OCR tools.

- Blue for core pipeline stages (Scraper, Enhancer, Packager)
- Green for external systems (Websites, AI services)
- Orange for configuration and caching
- Purple for the MCP server

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Sources
    subgraph "External Sources"
        style External_Website fill:#90ee90,stroke:#333,stroke-width:1px
        style External_PDF fill:#90ee90,stroke:#333,stroke-width:1px
        External_Website["Documentation Website\n(HTML)"] 
        External_PDF["PDF Files"] 
    end

    %% CLI Pipeline
    subgraph "CLI Pipeline"
        style CLI_Frontend fill:#add8e6,stroke:#333,stroke-width:1px
        style Config_Manager fill:#ffa500,stroke:#333,stroke-width:1px
        style Scraper_Module fill:#add8e6,stroke:#333,stroke-width:1px
        style OCR_Subcomponent fill:#add8e6,stroke:#333,stroke-width:1px
        style Cache_Store fill:#ffa500,stroke:#333,stroke-width:1px
        style Enhancement_Module fill:#add8e6,stroke:#333,stroke-width:1px
        style Packager_Module fill:#add8e6,stroke:#333,stroke-width:1px
        style Uploader_Module fill:#add8e6,stroke:#333,stroke-width:1px

        CLI_Frontend["CLI Front-end\n(entry scripts)"]
        Config_Manager["Config Manager\n(loads JSON presets)"]
        Scraper_Module["Scraper Module\n(cli/doc_scraper.py,\ncli/pdf_scraper.py)"]
        OCR_Subcomponent["OCR Sub-component\n(cli/pdf_extractor_poc.py)"]
        Cache_Store["Local Cache / Checkpoint Store\n(output/, cache/)"]
        Enhancement_Module["AI Enhancement Module\n(cli/enhance_skill.py,\ncli/enhance_skill_local.py)"]
        Packager_Module["Packager Module\n(cli/package_skill.py,\ncli/package_multi.py)"]
        Uploader_Module["Uploader Module\n(cli/upload_skill.py)"]
    end

    %% MCP Server
    subgraph "MCP Microservice"
        style MCP_Server fill:#dda0dd,stroke:#333,stroke-width:1px
        MCP_Server["MCP Server\n(mcp/server.py)"]
    end

    %% External AI Services
    subgraph "External AI Services"
        style Claude_Code fill:#90ee90,stroke:#333,stroke-width:1px
        style Anthropic_API fill:#90ee90,stroke:#333,stroke-width:1px
        Claude_Code["Claude Code Max\n(via MCP)"]
        Anthropic_API["Anthropic API\n(optional)"]
    end

    %% External Upload Target
    subgraph "Claude AI Platform"
        style Claude_AI fill:#90ee90,stroke:#333,stroke-width:1px
        Claude_AI["Claude AI Platform"]
    end

    %% Pipeline Connections
    CLI_Frontend -->|"commands"| Config_Manager
    Config_Manager -->|"config JSON"| Scraper_Module
    External_Website -->|"HTML pages"| Scraper_Module
    External_PDF -->|"PDF files"| Scraper_Module
    Scraper_Module -->|read/write| Cache_Store
    Scraper_Module -->|"raw data"| Enhancement_Module
    Scraper_Module -->|"OCR needed"| OCR_Subcomponent
    OCR_Subcomponent -->|"text output"| Enhancement_Module
    Enhancement_Module -->|"augmented data"| Packager_Module
    Packager_Module -->|".zip skill"| Uploader_Module
    Uploader_Module -->|"upload API"| Claude_AI

    %% CLI to MCP Server
    CLI_Frontend -->|"MCP commands"| MCP_Server
    MCP_Server -->|"local protocol"| Claude_Code

    %% Enhancement Module optional API
    Enhancement_Module -->|"API calls"| Anthropic_API

    %% Click Events
    click CLI_Frontend "https://github.com/yusufkaraaslan/skill_seekers/tree/development/cli/"
    click Config_Manager "https://github.com/yusufkaraaslan/skill_seekers/tree/development/configs/"
    click Scraper_Module "https://github.com/yusufkaraaslan/skill_seekers/blob/development/cli/doc_scraper.py"
    click Scraper_Module "https://github.com/yusufkaraaslan/skill_seekers/blob/development/cli/pdf_scraper.py"
    click OCR_Subcomponent "https://github.com/yusufkaraaslan/skill_seekers/blob/development/cli/pdf_extractor_poc.py"
    click Enhancement_Module "https://github.com/yusufkaraaslan/skill_seekers/blob/development/cli/enhance_skill.py"
    click Enhancement_Module "https://github.com/yusufkaraaslan/skill_seekers/blob/development/cli/enhance_skill_local.py"
    click Packager_Module "https://github.com/yusufkaraaslan/skill_seekers/blob/development/cli/package_skill.py"
    click Packager_Module "https://github.com/yusufkaraaslan/skill_seekers/blob/development/cli/package_multi.py"
    click Uploader_Module "https://github.com/yusufkaraaslan/skill_seekers/blob/development/cli/upload_skill.py"
    click MCP_Server "https://github.com/yusufkaraaslan/skill_seekers/blob/development/mcp/server.py"

    %% Styles
    classDef core fill:#add8e6,stroke:#333,stroke-width:1px
    classDef external fill:#90ee90,stroke:#333,stroke-width:1px
    classDef config fill:#ffa500,stroke:#333,stroke-width:1px
    classDef mcp fill:#dda0dd,stroke:#333,stroke-width:1px

    class CLI_Frontend,Scraper_Module,Enhancement_Module,Packager_Module,Uploader_Module core
    class External_Website,External_PDF,Claude_Code,Anthropic_API,Claude_AI external
    class Config_Manager,Cache_Store config
    class MCP_Server mcp
```
