How adm-l/data_profiler works
Go Data Profiler is an API-first service that accepts profiling jobs, runs them asynchronously against PostgreSQL, MySQL, or SQL Server sources, and persists results in a separate PostgreSQL metadata store. The principal workflow is request submission, queued profiling, quality and sensitive-data analysis, drift comparison, and profile retrieval/history. The graph includes documented profiling capabilities, but some internal calls are omitted where excerpts do not show their invocation; unsampled implementation details are not assumed.
Main entry points: manager.go Job Manager · store.go Metadata Store · router.go HTTP API · main.go Server Startup
Components 15
Service API
- Server Startup
cmd/server/main.go - Configuration
internal/config/config.go - HTTP API
internal/api/router.go - Job Manager
internal/worker/manager.go - Profile Models
internal/domain/models.go
Profiling
- Source Adapters
internal/adapters/adapter.go - Table Profiler
internal/adapters/profile.go
Quality And History
- Quality Rules
internal/quality/quality.go - PII Detection
internal/pii/detect.go - Drift Comparison
internal/drift/drift.go
Persistence
- Metadata Store
internal/store/store.go - Profile Results
internal/domain/models.go
Other components
- Client
- Source Databases
- Metadata PostgreSQL
Connections 19
- Server Startup → Configuration: loads settings
cmd/server/main.go - Server Startup → Metadata Store: creates store
cmd/server/main.go - Server Startup → Job Manager: starts workers
cmd/server/main.go - Server Startup → HTTP API: serves routes
cmd/server/main.go - Client → HTTP API: submits job
README.md - HTTP API → Job Manager: creates job
internal/api/router.go - HTTP API → Metadata Store: reads job and profiles
internal/api/router.go - Job Manager → Metadata Store: manages job state
internal/worker/manager.go - Job Manager → Source Adapters: runs profiling
internal/worker/manager.go - Source Adapters → Source Databases: connects and discovers
README.md - Table Profiler → Source Databases: queries table metrics
README.md - Source Adapters → Table Profiler: profiles table
internal/adapters/adapter.go - Job Manager → Quality Rules: evaluates profile
internal/worker/manager.go - Job Manager → PII Detection: detects sensitive data
internal/worker/manager.go - Job Manager → Drift Comparison: compares snapshots
internal/worker/manager.go - Job Manager → Metadata Store: saves profile
internal/worker/manager.go - Metadata Store → Metadata PostgreSQL: persists jobs and profiles
README.md - Metadata Store → Profile Results: loads profile history
internal/store/store.go - HTTP API → Client: returns status and results
README.md