# leoforster10010/dependencycrawler architecture

> Architecture diagram of the GitHub repository leoforster10010/dependencycrawler, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/leoforster10010/dependencycrawler
- Repository: https://github.com/leoforster10010/dependencycrawler

## Overview

##### Project type and purpose
- It comprises:
- A **backend host** (`DependencyCrawler.AppHost`) orchestrating discovery, processing and storage
- A **Blazor web UI** (`DependencyCrawler.BlazorClient.Server` + shared `BlazorClient` library)
- Two **CLI clients** (`DependencyCrawler.ConsoleClient`, `DependencyCrawler.QueryClient`)
- Multiple **data providers** (JSON file, MongoDB, PostgreSQL)
- A **DataDiscovery** sub‐system (C# analyzer + REST microservice)
- A **core processing engine** (`DependencyCrawler.DataCore`) with contracts, DTOs and repository implementations

##### File structure
- Top‐level “src” subdivides into:
- **AppHost** – the ASP.NET Core Generic Host that wires up services, logging, configuration, DI
- **BlazorClient.Server** – server‐hosted Blazor that serves the UI and likely exposes REST endpoints
- **BlazorClient** – shared Razor components, layouts, dialogs
- **ConsoleClient** & **QueryClient** – worker‐style clients that invoke the host’s APIs
- **Data.Contracts** – common entity and interface definitions (caching, configuration, module models)
- **DataJson**, **Data.MongoDB**, **Data.Postgresql** (plus migration service) – concrete `IDataSource` implementations
- **DataDiscovery.CSharp** & **.REST** & **.RESTClient** – code‐analysis engine, REST wrapper and client
- **DataCore** family – core processing, providers, converters, read‑only and value access abstractions
- **Implementations** – repository factories, loaders, evaluators, linked/unlinked model mappers
- **Framework** – logging, extensions and common helpers

##### Key insights from README
- The product exposes both CLI and web front ends
- It persists crawl results in JSON (file‐sync) and MongoDB, with optional PostgreSQL migration
- It is driven by a graph model: discovery → processing (DataCore) → storage → query/exposure

##### Main components & interactions (to include
a. Presentation layer
- Blazor Web UI ←→ Backend Host (HTTP/REST or gRPC)
- CLI Clients (ConsoleClient, QueryClient) ←→ Backend Host (HTTP/REST)
b. API / Orchestration
- **DependencyCrawler.AppHost** wires up:
- DataDiscovery service (uses either in‐process `DataDiscovery.CSharp` or calls `DataDiscovery.CSharp.REST`)
- Core engine (`DataCore`)
- Repository layer (implements caching, evaluation, module loading)
- Data sources (`JsonDataSource`, `MongoDbDataSource`, `PostgresqlDataSource`)
c. Business logic
- **DataCore** takes raw discovery DTOs → builds graph, applies evaluations → emits domain models
d. Data access & storage
- JSON File system (local cache)
- MongoDB (primary datastore)
- PostgreSQL (optional via migration service)
e. External microservice
- `DataDiscovery.CSharp.REST` exposes code‐analysis over HTTP; clients may use `RESTClient` library

##### Architectural patterns & principles
- Layered architecture: Presentation → API/Host → Business (DataCore) → Data access → Storage
- Dependency Injection via Microsoft.Extensions.Hosting / ServiceCollectionExtensions
- Adapter pattern for multiple data sources (JSON, Mongo, Postgres) and discovery engines
- Repository & Factory patterns for module loading and evaluation
- Separation of concerns: contracts vs implementations vs clients

## Diagram (Mermaid)

```mermaid
graph TB
  subgraph "Clients/UI"
    BlazorServer["Blazor UI (Server)"]:::client
    BlazorShared["Blazor UI Shared"]:::client
    ConsoleClient["ConsoleClient (CLI)"]:::client
    QueryClient["QueryClient (CLI)"]:::client
  end

  subgraph "Host/API"
    AppHost["DependencyCrawler.AppHost\n(.NET Generic Host)"]:::host

    subgraph "AppHost Modules"
      DataContracts["Data.Contracts"]:::host
      Framework["Framework & Utilities"]:::host
      Implementations["Implementations"]:::host
      Contracts["Shared Contracts"]:::host

      subgraph "Discovery Services"
        DiscInProc["DataDiscovery.CSharp\n(In‑process)"]:::host
        DiscREST["DataDiscovery.CSharp.REST\n(Microservice)"]:::host
        RESTClient["RESTClient Library"]:::host
      end

      subgraph "Core Processing"
        CoreMain["DataCore Engine"]:::core
        CoreDataAccess["DataAccess Abstractions"]:::core
        CoreReadOnly["Read‑Only Access Abstractions"]:::core
        CoreValue["Value‑Access / DTO Converters"]:::core
      end

      Repos["Repository & Model Mappers"]:::host
      DataSources["IDataSource Providers"]:::host
    end
  end

  subgraph "Data Sources & Storage"
    JsonDS["JsonDataSource\n(File System)"]:::storage
    MongoDS["MongoDbDataSource\n(MongoDB)"]:::storage
    PgDS["PostgresDataSource\n(PostgreSQL)"]:::storage
    PgMig["PostgreSQL Migration Service"]:::storage
  end

  %% Client to Host
  BlazorServer -->|"HTTP/REST"| AppHost
  ConsoleClient -->|"HTTP/REST"| AppHost
  QueryClient -->|"HTTP/REST"| AppHost

  %% Host internal flows
  AppHost -->|"DI injects"| DataContracts
  AppHost -->|"DI injects"| Framework
  AppHost -->|"DI injects"| Implementations
  AppHost -->|"DI injects"| Contracts

  AppHost -->|"calls in‑process"| DiscInProc
  AppHost -->|"calls HTTP"| DiscREST
  AppHost -->|"uses"| RESTClient

  AppHost -->|"calls"| CoreMain
  CoreMain -->|"uses"| CoreDataAccess
  CoreMain -->|"uses"| CoreReadOnly
  CoreMain -->|"uses"| CoreValue

  AppHost -->|"injects"| Repos
  AppHost -->|"injects"| DataSources

  DataSources -->|"read/write"| JsonDS
  DataSources -->|"read/write"| MongoDS
  DataSources -->|"read/write"| PgDS
  PgMig -->|"migrates data to"| PgDS

  %% Styles
  classDef client fill:#AED6F1,stroke:#21618C,color:#0B2447;
  classDef host fill:#A9DFBF,stroke:#196F3D,color:#0B3D91;
  classDef core fill:#F5CBA7,stroke:#A04000,color:#1B2631;
  classDef storage fill:#D7BDE2,stroke:#512E5F,color:#1B2631;

  %% Click Events
  click AppHost "https://github.com/leoforster10010/dependencycrawler/blob/main/src/DependencyCrawler.AppHost"
  click BlazorServer "https://github.com/leoforster10010/dependencycrawler/blob/main/src/DependencyCrawler.BlazorClient.Server"
  click BlazorShared "https://github.com/leoforster10010/dependencycrawler/blob/main/src/DependencyCrawler.BlazorClient"
  click ConsoleClient "https://github.com/leoforster10010/dependencycrawler/blob/main/src/DependencyCrawler.ConsoleClient"
  click QueryClient "https://github.com/leoforster10010/dependencycrawler/blob/main/src/DependencyCrawler.QueryClient"
  click DataContracts "https://github.com/leoforster10010/dependencycrawler/blob/main/src/DependencyCrawler.Data.Contracts"
  click Framework "https://github.com/leoforster10010/dependencycrawler/blob/main/src/DependencyCrawler.Framework"
  click Implementations "https://github.com/leoforster10010/dependencycrawler/tree/main/src/DependencyCrawler/Implementations"
  click Contracts "https://github.com/leoforster10010/dependencycrawler/tree/main/src/DependencyCrawler/Contracts"
  click DiscInProc "https://github.com/leoforster10010/dependencycrawler/blob/main/src/DependencyCrawler.DataDiscovery.CSharp"
  click DiscREST "https://github.com/leoforster10010/dependencycrawler/blob/main/src/DependencyCrawler.DataDiscovery.CSharp.REST"
  click RESTClient "https://github.com/leoforster10010/dependencycrawler/blob/main/src/DependencyCrawler.DataDiscovery.CSharp.RESTClient"
  click CoreMain "https://github.com/leoforster10010/dependencycrawler/blob/main/src/DependencyCrawler.DataCore"
  click CoreDataAccess "https://github.com/leoforster10010/dependencycrawler/blob/main/src/DependencyCrawler.DataCore.DataAccess"
  click CoreReadOnly "https://github.com/leoforster10010/dependencycrawler/blob/main/src/DependencyCrawler.DataCore.ReadOnlyAccess"
  click CoreValue "https://github.com/leoforster10010/dependencycrawler/blob/main/src/DependencyCrawler.DataCore.ValueAccess"
  click JsonDS "https://github.com/leoforster10010/dependencycrawler/blob/main/src/DependencyCrawler.Data.Json"
  click MongoDS "https://github.com/leoforster10010/dependencycrawler/blob/main/src/DependencyCrawler.Data.MongoDB"
  click PgDS "https://github.com/leoforster10010/dependencycrawler/blob/main/src/DependencyCrawler.Data.Postgresql"
  click PgMig "https://github.com/leoforster10010/dependencycrawler/blob/main/src/DependencyCrawler.Data.Postgresql.MigrationService"
```
