How harit3009/docker-fs works
docker-FS is a NestJS cloud file system: authenticated users manage hierarchical files and folders, upload or download S3 objects, and search or chat over indexed document content. The documented processing stages include S3-triggered Kafka work for metadata creation, zip extraction, document parsing, embeddings, and indexing; trash cleanup and group membership are additional capabilities. Sampled excerpts establish API-to-storage and chat retrieval calls. Kafka/Lambda event wiring and several processing links are documented but their implementations were not sampled, so those edges are limited to documented relationships.
Main entry points: prisma.service.ts Prisma service · ai-retrieval.service.ts AI retrieval · kafka.service.ts Kafka bridge · embedding.service.ts Embedding service
Components 25
API and identity
- Auth API
src/auth/auth.controller.ts - Login service
src/auth/auth.service.ts - JWT strategy
src/auth/jwt-verifier.strategy.ts
File storage
- File system API
src/file-system/file-system.controller.ts - S3 integration
src/s3-module/s3-service.service.ts - Prisma service
src/prisma/prisma.service.ts
Document processing
- Kafka bridge
src/bridge/kafka/kafka.service.ts - S3 event trigger
src/lambdas/s3_event_trigger.ts - File creation consumer
src/bridge/kafka/kafka-create/kafka-create-file.ts - Zip extraction
src/bridge/kafka/kafka-extract-zip/kafka-extract-zip.service.ts - Delete consumer
src/bridge/kafka/kafka-delete-consumer/kafka-delete-consumer.service.ts - Indexing service
src/bridge/kafka/kafka-create/kafka-index-file-service.service.ts - PDF parser
src/bridge/pdf-parser/pdf-parser.service.ts - S3 event DLQ processor
src/lambdas/s3_event_dlq_processor.ts
Search and chat
- Embedding service
src/bridge/embedding/embedding.service.ts - OpenSearch service
src/bridge/open-search/open-search.service.ts - AI retrieval
src/bridge/ai-retrieval/ai-retrieval.service.ts
Groups and cleanup
- Group API
src/user-groups/group-controller.controller.ts - Trash scheduler
src/bridge/delete-trash-scheduler/delete-trash-scheduler.service.ts
Other components
- User
- Google OAuth
- S3 object store
- PostgreSQL metadata
- OpenSearch indexes
- Chat and embedding APIs
Connections 35
- User → Auth API: signs in
- Auth API → Google OAuth: authenticates with
- Auth API → Login service: handles login
- Login service → Prisma service: reads/writes users
- Auth API → JWT strategy: guards requests
- User → File system API: manages files
- File system API → Prisma service: reads/writes metadata
- File system API → S3 integration: requests signed URLs
- S3 integration → S3 object store: reads/writes objects
- Prisma service → PostgreSQL metadata: persists metadata
- S3 object store → S3 event trigger: emits object events
- S3 event trigger → Kafka bridge: publishes work
- Kafka bridge → File creation consumer: dispatches events
- Kafka bridge → Zip extraction: dispatches events
- Kafka bridge → Delete consumer: dispatches events
- File creation consumer → Prisma service: records metadata
- Zip extraction → S3 integration: extracts objects
- File creation consumer → Indexing service: requests indexing
- Indexing service → PDF parser: extracts PDF text
- PDF parser → S3 integration: reads object stream
- Indexing service → Embedding service: generates vectors
- Indexing service → OpenSearch service: indexes content
- Embedding service → Chat and embedding APIs: requests embeddings
- OpenSearch service → OpenSearch indexes: indexes/searches
- User → AI retrieval: searches and chats
- AI retrieval → Embedding service: embeds query
- AI retrieval → OpenSearch service: retrieves chunks
- AI retrieval → Prisma service: reads metadata
- AI retrieval → Embedding service: submits conversation
- Embedding service → Chat and embedding APIs: requests chat response
- User → Group API: manages groups
- Group API → Prisma service: reads/writes membership
- Trash scheduler → Prisma service: cleans expired trash
- Trash scheduler → S3 integration: deletes objects
- S3 event DLQ processor → Kafka bridge: processes failed events