How kiyst/jobgoblin works
JobGoblin is an early-stage job aggregation and market-intelligence backend. Its principal workflow is to plan a saved-search query for providers, execute discovery, and persist ingestion outcomes and job identity; normalization is a distinct market-data capability. The README documents the FastAPI server and health/readiness endpoints, while source-index references expose the ingestion, provider, persistence, and saved-search connections. Several planned product areas are represented by models or services but are not shown wired into the main runtime flow.
Main entry points: provider_execution.py Provider execution · persistence.py Job persistence · query_planner.py Query planner · base.py Provider interface
Components 22
API entry
- FastAPI app
backend/app/main.py - Health endpoints
backend/app/api/health.py
Search planning
- Saved searches
backend/app/services/saved_searches.py - Query planner
backend/app/discovery/query_planner.py - Query contracts
backend/app/schemas/provider.py
Job collection
- Collection orchestrator
backend/app/ingestion/orchestrator.py - Ingestion pipeline
backend/app/ingestion/pipeline.py - Provider registry
backend/app/providers/registry.py - Provider interface
backend/app/providers/base.py - Fixture provider
backend/app/providers/fixture.py - Provider execution
backend/app/ingestion/provider_execution.py - Discovered job
backend/app/schemas/discovered_job.py
Domain data
- Saved-search models
backend/app/db/models/saved_search.py - Job persistence
backend/app/ingestion/persistence.py - Identity resolution
backend/app/ingestion/identity.py - Natural keys
backend/app/ingestion/natural_key.py - Domain models
backend/app/db/models - Candidate profiles
backend/app/services/candidate_profiles.py - User job actions
backend/app/services/user_jobs.py
Market intelligence
- Job normalization
backend/app/normalization - Normalization results
backend/app/normalization/types.py
Other components
- API client
Connections 19
- API client → FastAPI app: requests API
README.md - FastAPI app → Health endpoints: mounts routes
backend/app/main.py - Saved searches → Saved-search models: uses models
backend/app/services/saved_searches.py - Query planner → Saved-search models: reads search
backend/app/discovery/query_planner.py - Query planner → Query contracts: builds query
backend/app/discovery/query_planner.py - Collection orchestrator → Query planner: plans queries
backend/app/ingestion/orchestrator.py - Collection orchestrator → Provider registry: gets providers
backend/app/ingestion/orchestrator.py - Provider registry → Provider interface: registers providers
backend/app/providers/registry.py - Fixture provider → Provider interface: implements contract
backend/app/providers/fixture.py - Ingestion pipeline → Provider execution: executes query
backend/app/ingestion/pipeline.py - Provider execution → Provider interface: calls provider
backend/app/ingestion/provider_execution.py - Provider execution → Discovered job: handles results
backend/app/ingestion/provider_execution.py - Provider execution → Job persistence: persists jobs
backend/app/ingestion/provider_execution.py - Job persistence → Identity resolution: resolves identity
backend/app/ingestion/persistence.py - Job persistence → Natural keys: computes keys
backend/app/ingestion/persistence.py - Job persistence → Domain models: writes records
backend/app/ingestion/persistence.py - Candidate profiles → Domain models: uses models
backend/app/services/candidate_profiles.py - User job actions → Domain models: uses models
- Job normalization → Normalization results: returns typed results
backend/app/normalization/types.py