How spinoco/connector works
Project Type and Purpose
• This is an open-source, single‐service data connector written in TypeScript/Node.js.
• Its job is to poll the Spinoco Platform HTTP API for tasks and related data (recordings, transcriptions, metadata, etc.) and write them to a storage backend (local FS or cloud).
File Structure
• Top level:
• Dockerfile, docker-compose-example.yml → containerization/deployment
• src/ → service code
• src/config.ts → environment-variable config loader
• src/http.ts → HTTP client for Spinoco API
• src/sync.ts → orchestration of fetch + storage
• src/storage/ → pluggable providers: local, AWS S3, GCS, Azure
• src/interfaces/ → TypeScript contracts
• build.sh, webpack.*.js → bundling into dist/bundle.js
• README describes environment variables, supported data types, and how to run via Docker.
Main Components & Their Interactions
a. Configuration Loader
• Reads env vars (SP_API_TOKEN, SP_TASK_SYNC_*, storage provider config).
b. HTTP Client
• Authenticates to the Spinoco REST API, retrieves task lists and per-task data streams.
c. Sync Controller
• Applies filters (skills, hashtags, date), iterates tasks, invokes HTTP client, delegates to storage.
d. Storage Abstraction Layer
• Defines IStorage interface.
• Four implementations:
• LocalFileSystem (writes to disk)
• AWS S3 (uploads to S3)
• Google Cloud Storage
• Azure Blob
e. Logging
• Centralized logger (log level controlled by SP_LOG_LEVEL)
f. Deployment Unit
• Bundled Node.js binary (bundle.js) inside a Docker image, orchestrated via docker-compose or Kubernetes.
Extensibility
• Webpack build step—source TS → bundled JS.
• Environment variables as configuration injection points.