How scribd/objinsync works
Project type and purpose
• ObjInSync is a Go-based command-line tool plus daemon that continuously and incrementally synchronizes a directory from a remote object store (e.g., AWS S3 or Minio) to a local file system.
• It exposes health and Prometheus metrics endpoints when running in long-running (“daemon”) mode and integrates with Sentry for error reporting.
File structure
• Top‐level:
• main.go: CLI entrypoint using Cobra/Viper-style command parsing (pull command, flags, daemon vs once).
• go.mod / go.sum: Go module definitions.
• Dockerfile, .dockerignore, Makefile, .github/workflows/cicd.yml, goreleaser.yml: build, CI/CD, containerization, and release automation.
• pkg/sync: core sync logic—dir.go and pull.go contain the pull algorithms; internal tests verify correctness.
README insights
• Commands: objinsync pull with flags for interval, once, exclude patterns, custom endpoints, SSL, status/metrics ports.
• Behavior: single-run (“once”) vs continuous polling (default 5s interval).
• Monitoring: HTTP server on default port 8087 serving /health and /metrics.
• Integrations: AWS SDK (S3), Prometheus client, Sentry SDK.