How leoforster10010/dependencycrawler works
Project type and purpose
• It comprises:
• A backend host (DependencyCrawler.AppHost) orchestrating discovery, processing and storage
• A Blazor web UI (DependencyCrawler.BlazorClient.Server + shared BlazorClient library)
• Two CLI clients (DependencyCrawler.ConsoleClient, DependencyCrawler.QueryClient)
• Multiple data providers (JSON file, MongoDB, PostgreSQL)
• A DataDiscovery sub‐system (C# analyzer + REST microservice)
• A core processing engine (DependencyCrawler.DataCore) with contracts, DTOs and repository implementations
File structure
• Top‐level “src” subdivides into:
• AppHost – the ASP.NET Core Generic Host that wires up services, logging, configuration, DI
• BlazorClient.Server – server‐hosted Blazor that serves the UI and likely exposes REST endpoints
• BlazorClient – shared Razor components, layouts, dialogs
• ConsoleClient & QueryClient – worker‐style clients that invoke the host’s APIs
• Data.Contracts – common entity and interface definitions (caching, configuration, module models)
• DataJson, Data.MongoDB, Data.Postgresql (plus migration service) – concrete IDataSource implementations
• DataDiscovery.CSharp & .REST & .RESTClient – code‐analysis engine, REST wrapper and client
• DataCore family – core processing, providers, converters, read‑only and value access abstractions
• Implementations – repository factories, loaders, evaluators, linked/unlinked model mappers
• Framework – logging, extensions and common helpers
Key insights from README
• The product exposes both CLI and web front ends
• It persists crawl results in JSON (file‐sync) and MongoDB, with optional PostgreSQL migration
• It is driven by a graph model: discovery → processing (DataCore) → storage → query/exposure
Main components & interactions (to include
a. Presentation layer
• Blazor Web UI ←→ Backend Host (HTTP/REST or gRPC)
• CLI Clients (ConsoleClient, QueryClient) ←→ Backend Host (HTTP/REST)
b. API / Orchestration
• DependencyCrawler.AppHost wires up:
• DataDiscovery service (uses either in‐process DataDiscovery.CSharp or calls DataDiscovery.CSharp.REST)
• Core engine (DataCore)
• Repository layer (implements caching, evaluation, module loading)
• Data sources (JsonDataSource, MongoDbDataSource, PostgresqlDataSource)
c. Business logic
• DataCore takes raw discovery DTOs → builds graph, applies evaluations → emits domain models
d. Data access & storage
• JSON File system (local cache)
• MongoDB (primary datastore)
• PostgreSQL (optional via migration service)
e. External microservice
• DataDiscovery.CSharp.REST exposes code‐analysis over HTTP; clients may use RESTClient library
Architectural patterns & principles
• Layered architecture: Presentation → API/Host → Business (DataCore) → Data access → Storage
• Dependency Injection via Microsoft.Extensions.Hosting / ServiceCollectionExtensions
• Adapter pattern for multiple data sources (JSON, Mongo, Postgres) and discovery engines
• Repository & Factory patterns for module loading and evaluation
• Separation of concerns: contracts vs implementations vs clients