How fardinrehman/linkedin-research-data-collection-tool works
The product is a LinkedIn research and data-collection application with a browser UI, a backend collection workflow, and a standalone browser simulation fallback. Users configure a search and fields, launch a job, monitor progress, inspect projected results and failures, then export data. The README documents simulation and authorized-provider adapters, dynamic projection, deduplication, SSE progress, and CSV/XLSX/JSON export. Several server internals are indexed but not excerpted; edges to those stages are shown only where the job-manager excerpt or README supports them.
Main entry points: App.jsx Research workspace · jobManager.js Job manager · api.js API and fallback client · server.js HTTP server
Components 28
Research interface
- Research workspace
client/src/App.jsx - Search form
client/src/components/search/SearchQueryForm.jsx - Field selector
client/src/components/search/FieldSelector.jsx - Custom field editor
client/src/components/search/CustomFieldModal.jsx - Field presets
client/src/types/fields.js - Job stream hook
client/src/hooks/useJobStream.js - Progress monitor
client/src/components/monitor/JobStatusCard.jsx - API and fallback client
client/src/services/api.js - Browser simulation
client/src/services/clientProvider.js
Job API
- HTTP server
server/src/server.js - Job routes
server/src/routes/jobRoutes.js - Job controller
server/src/controllers/jobController.js - Result controller
server/src/controllers/resultController.js - Export controller
server/src/controllers/exportController.js
Collection processing
- Job manager
server/src/services/jobManager.js - Record normalization
server/src/services/normalizationService.js - Composite deduplication
server/src/services/deduplicationService.js - In-memory job state
server/src/services/jobManager.js
Data adapters
- Provider selection
server/src/providers/ProviderFactory.js - Provider contract
server/src/providers/BaseDataProvider.js - Simulation adapter
server/src/providers/MockDataProvider.js - Authorized adapter
server/src/providers/LinkedInAuthorizedProvider.js
Results and export
- Dynamic results table
client/src/components/results/DynamicDataTable.jsx - Failed-record drawer
client/src/components/error/FailedRecordsDrawer.jsx - Export toolbar
client/src/components/results/ExportToolbar.jsx - Export generation
server/src/services/exportService.js
Other components
- Researcher
- Authorized data service
Connections 34
- Researcher → Research workspace: uses
README.md - Research workspace → Search form: renders
client/src/App.jsx - Search form → Field selector: renders
client/src/components/search/SearchQueryForm.jsx - Field selector → Custom field editor: opens
client/src/components/search/FieldSelector.jsx - Research workspace → Field presets: loads presets
client/src/App.jsx - Research workspace → API and fallback client: creates and fetches jobs
client/src/App.jsx - Research workspace → Job stream hook: tracks job
client/src/App.jsx - Research workspace → Progress monitor: renders status
client/src/App.jsx - Job stream hook → API and fallback client: polls status
client/src/hooks/useJobStream.js - API and fallback client → Browser simulation: falls back to
client/src/services/api.js - API and fallback client → HTTP server: calls HTTP API
client/src/services/api.js - Research workspace → Dynamic results table: renders results
client/src/App.jsx - Research workspace → Failed-record drawer: renders diagnostics
client/src/App.jsx - HTTP server → Job routes: mounts
server/src/server.js - HTTP server → Result controller: mounts results
server/src/server.js - HTTP server → Export controller: mounts exports
server/src/server.js - Job routes → Job controller: routes requests
server/src/routes/jobRoutes.js - Job controller → Job manager: creates and reads jobs
server/src/controllers/jobController.js - Job controller → Provider selection: lists providers
server/src/controllers/jobController.js - Job manager → In-memory job state: stores job state
server/src/services/jobManager.js - Job manager → Provider selection: selects adapter
server/src/services/jobManager.js - Job manager → Record normalization: projects records
server/src/services/jobManager.js - Job manager → Composite deduplication: checks duplicates
server/src/services/jobManager.js - Provider selection → Simulation adapter: selects simulation
server/src/providers/ProviderFactory.js - Provider selection → Authorized adapter: selects authorized
server/src/providers/ProviderFactory.js - Simulation adapter → Provider contract: implements contract
server/src/providers/MockDataProvider.js - Authorized adapter → Provider contract: implements contract
server/src/providers/LinkedInAuthorizedProvider.js - Authorized adapter → Authorized data service: requests authorized data
README.md - Result controller → Job manager: reads job results
server/src/controllers/resultController.js - Export controller → Job manager: reads job records
server/src/controllers/exportController.js - Export controller → Export generation: generates files
server/src/controllers/exportController.js - Dynamic results table → Export toolbar: renders toolbar
client/src/components/results/DynamicDataTable.jsx - Export toolbar → API and fallback client: requests export
client/src/components/results/ExportToolbar.jsx - API and fallback client → Export generation: downloads via API
README.md