How kendelja/engineerd works
Engineerd aggregates Built In and Wellfound postings, processes them through a shared ingestion pipeline, stores them in PostgreSQL, and serves job listings to a React dashboard. The graph preserves source-specific scraping/parsing, validation and deduplication, API retrieval and ingestion, and dashboard search, filtering, saving, and market selection. Edges follow shown calls or README-documented capabilities; unsampled scraper, validator, deduplicator, and UI details limit some implementation-level connections.
Main entry points: runIngestion.py Ingestion runner · pipeline.py Shared pipeline · repositories.py Job repository · jobs.py Jobs API
Components 21
Job ingestion
- Built In scraper
Backend/app/ingestion/scrapers/BuiltInScraper.py - Wellfound scraper
Backend/app/ingestion/scrapers/WellfoundScraper.py - Built In parser
Backend/app/ingestion/parsers/BuiltInParser.py - Wellfound parser
Backend/app/ingestion/parsers/WellfoundParser.py - Ingestion runner
Backend/scripts/runIngestion.py - Shared pipeline
Backend/app/ingestion/pipeline.py - Normalizer
Backend/app/ingestion/normalizer.py - Validator
Backend/app/ingestion/validator.py - Deduplicator
Backend/app/ingestion/deduplicator.py
API and storage
- Job model
Backend/app/models/job.py - Job repository
Backend/app/database/repositories.py - DB connection
Backend/app/database/connection.py - PostgreSQL
database/schema.sql - Jobs API
Backend/app/api/jobs.py - FastAPI app
Backend/app/main.py
Dashboard
- React bootstrap
frontend/src/main.jsx - App shell
frontend/src/App.jsx - Job dashboard
frontend/src/pages/Dashboard.jsx
Other components
- Built In
- Wellfound
- Dashboard user
Connections 26
- Built In → Built In scraper: provides listings
README.md - Wellfound → Wellfound scraper: provides listings
README.md - Built In scraper → Built In parser: supplies HTML
- Wellfound scraper → Wellfound parser: supplies HTML
- Ingestion runner → Built In scraper: configures source
Backend/scripts/runIngestion.py - Ingestion runner → Built In parser: configures parser
Backend/scripts/runIngestion.py - Ingestion runner → Wellfound scraper: configures source
Backend/scripts/runIngestion.py - Ingestion runner → Wellfound parser: configures parser
Backend/scripts/runIngestion.py - Ingestion runner → Shared pipeline: runs twice
Backend/scripts/runIngestion.py - Shared pipeline → Normalizer: normalizes jobs
Backend/app/ingestion/pipeline.py - Shared pipeline → Validator: validates jobs
Backend/app/ingestion/pipeline.py - Shared pipeline → Deduplicator: checks duplicates
Backend/app/ingestion/pipeline.py - Shared pipeline → Job repository: inserts jobs
Backend/app/ingestion/pipeline.py - Normalizer → Job model: creates model
Backend/app/ingestion/normalizer.py - Validator → Job model: validates model
Backend/app/ingestion/validator.py - Job repository → Job model: reads fields
Backend/app/database/repositories.py - Job repository → DB connection: opens connection
Backend/app/database/repositories.py - Job repository → PostgreSQL: reads and writes
Backend/app/database/repositories.py - Jobs API → Job repository: gets and clears jobs
Backend/app/api/jobs.py - Jobs API → Ingestion runner: starts ingestion
Backend/app/api/jobs.py - FastAPI app → Jobs API: mounts router
Backend/app/main.py - Dashboard user → React bootstrap: opens dashboard
- React bootstrap → App shell: renders
frontend/src/main.jsx - App shell → Job dashboard: renders
frontend/src/App.jsx - Job dashboard → Jobs API: requests listings
frontend/src/pages/Dashboard.jsx - Jobs API → Job dashboard: returns listings
Backend/app/api/jobs.py