How vgpastor/batchactions works
@batchactions is a set of TypeScript libraries for record-by-record batch processing, with optional import validation, distributed workers, and pluggable persistence. The principal workflow configures a job, reads records from a source, processes them in batches, and tracks job state and lifecycle events. Import workflows add parsing, schema validation, and preview; distributed workflows add batch claiming and worker processing. README capabilities are represented where source excerpts are incomplete; uncertain internal wiring is omitted.
Main entry points: BatchEngine.ts BatchEngine · BulkImport.ts BulkImport · PreviewImport.ts Import preview · StateStore.ts State store port
Components 21
Core processing
- BatchEngine
packages/core/src/BatchEngine.ts - Data sources
packages/core/src/infrastructure/sources - Job use cases
packages/core/src/application/usecases - Job context
packages/core/src/application/JobContext.ts - Event bus
packages/core/src/application/EventBus.ts - State store port
packages/core/src/domain/ports/StateStore.ts - Distributed store port
packages/core/src/domain/ports/DistributedStateStore.ts
Import workflow
- BulkImport
packages/import/src/BulkImport.ts - Format parsers
packages/import/src/infrastructure/parsers - Import preview
packages/import/src/application/usecases/PreviewImport.ts - Schema validator
packages/import/src/domain/services/SchemaValidator.ts
Distributed workers
- Distributed import
packages/distributed/src/DistributedImport.ts - Prepare batches
packages/distributed/src/PrepareDistributedImport.ts - Process batch
packages/distributed/src/ProcessDistributedBatch.ts
State persistence
- Memory store
packages/core/src/infrastructure/state/InMemoryStateStore.ts - File store
packages/core/src/infrastructure/state/FileStateStore.ts - Sequelize adapter
packages/state-sequelize/src/SequelizeStateStore.ts - Prisma adapter
packages/state-prisma/src/PrismaStateStore.ts
Other components
- Application
- Record processor
- SQL database
Connections 27
- Application → BatchEngine: configures
- BatchEngine → Data sources: reads records
- BatchEngine → Job use cases: delegates
- BatchEngine → Job context: creates context
- Job use cases → Record processor: processes records
- Job context → State store port: saves state
- Job context → Event bus: emits events
- Application → BulkImport: configures import
- BulkImport → Data sources: reads records
- BulkImport → Format parsers: parses data
- BulkImport → Schema validator: validates records
- Application → Import preview: requests preview
- Import preview → Data sources: reads chunks
- Import preview → Format parsers: parses chunks
- Import preview → Schema validator: validates sample
- Application → Distributed import: starts distributed job
- Distributed import → Prepare batches: prepares batches
- Distributed import → Process batch: dispatches batches
- Process batch → Schema validator: validates records
- Process batch → Distributed store port: claims and saves
- BatchEngine → State store port: uses store
- Memory store → State store port: implements
- File store → State store port: implements
- Sequelize adapter → Distributed store port: implements
- Prisma adapter → Distributed store port: implements
- Sequelize adapter → SQL database: reads and writes
- Prisma adapter → SQL database: reads and writes