tdenkov123/garchive

Purpose and Entry Point

GArchive is a Go gRPC service for file metadata and upload lifecycle management. File bytes are stored in S3-compatible MinIO; PostgreSQL holds metadata and multipart state; Redis accelerates reads; Kafka receives domain events. The process starts at cmd/server/main.go, which composes dependencies through internal/app/app.go.

Public API Contract

The gRPC contract defines single-upload, multipart-upload, read/list, download URL, and delete operations. Generated server/client bindings are under api/gen/file/v1/file_grpc.pb.go; the source-of-truth schema is api/proto/file/v1/file.proto. Request handling enters through internal/grpc/handler.go.

Application and Service Layer

internal/service/file_service.go implements file lifecycle operations: create metadata, issue presigned upload/download URLs, confirm checksums, read/list records, and soft-delete files. internal/service/multipart_service.go owns multipart state transitions, part URL issuance/reporting, completion, and abort. internal/service/adapters.go defines the boundary-facing dependencies consumed by services, keeping business workflows independent of concrete infrastructure.

Request Control Plane

gRPC middleware applies cross-cutting controls before handlers: JWT identity extraction/authentication, rate limiting, validation, logging/metrics/error handling. Relevant paths are internal/grpc/middleware/interceptors.go, internal/grpc/middleware/ratelimit.go, and internal/auth/jwt.go. Authenticated ownership and request context are propagated through internal/auth/context.go; audit records are emitted via internal/audit/audit.go.

Persistence, Cache, and Object Storage

PostgreSQL is the authoritative metadata store, with schema migrations for files and multipart uploads in internal/repository/postgres/migrations/001_init.sql and internal/repository/postgres/migrations/002_multipart.sql. Repository implementations live in internal/repository/postgres/file.go. Redis caching is isolated behind internal/cache/redis/cache.go. MinIO access, including presigned URL and object lifecycle operations, is encapsulated in internal/storage/minio/client.go.

Events and Lifecycle Flow

For a single upload, the service persists a pending file record, returns a presigned PUT URL, then validates a supplied SHA-256 checksum during confirmation before marking the record ready. Multipart uploads persist upload/part progress until completion or abort. Ready and deletion lifecycle changes publish Kafka domain events through internal/service/events.go and internal/events/kafka/publisher.go. Object operations cross the MinIO boundary; metadata transitions cross PostgreSQL; cache invalidation/population accompanies read/write paths.

Runtime and Deployment Boundaries

The service runs as a containerized Go process (Dockerfile) and exposes gRPC on port 50051, Prometheus metrics on 9090, and gRPC health. Local orchestration is defined in docker-compose.yml; production composition is in docker-compose.prod.yml. Core external runtime services are PostgreSQL, Redis, MinIO, and Kafka; optional profiles add Keycloak for OIDC demonstration and Jaeger observability. TLS server credentials are handled by internal/tlsutil/credentials.go, while configuration is centralized in internal/config/config.go.