How googlecloudplatform/gcs-analytics-core works
GCS Analytics Core is a Java library that sits between analytics frameworks and Google Cloud Storage access layers. Its principal read workflow opens an object through the file-system API, then provides a seekable stream with vectored reads, adaptive access behavior, and optional footer or small-object prefetching. The client layer supplies GCS operations and shared caching; configuration, authentication, and telemetry support those runtime paths. The README documents the high-level flow and capabilities; unsampled implementation details are represented only where the README or source index supports them.
Main entry points: GcsFileSystemImpl.java GCS File System · GoogleCloudStorageInputStream.java Input Stream · AnalyticsCacheManager.java Cache Manager · GcsClientImpl.java GCS Client
Components 19
Framework Integration
- File-System Adapters
- Core Options
core/src/main/java/com/google/cloud/gcs/analyticscore/core/GcsAnalyticsCoreOptions.java
Optimized Read Path
- Input Stream
core/src/main/java/com/google/cloud/gcs/analyticscore/core/GoogleCloudStorageInputStream.java - Smart Read Channel
core/src/main/java/com/google/cloud/gcs/analyticscore/core/channel/SmartReadChannel.java - Vectored Reads
client/src/main/java/com/google/cloud/gcs/analyticscore/client/VectoredSeekableByteChannel.java - Adaptive Reads
client/src/main/java/com/google/cloud/gcs/analyticscore/client/AdaptiveReadStrategy.java - Footer Prefetch
core/src/main/java/com/google/cloud/gcs/analyticscore/core/optimizer/GcsFooterOptimizer.java - Parquet Footer Parser
core/src/main/java/com/google/cloud/gcs/analyticscore/core/prefetch/ParquetFooterParser.java - Small-Object Prefetch
core/src/main/java/com/google/cloud/gcs/analyticscore/core/optimizer/SmallObjectOptimizer.java
GCS Client
- GCS File System
client/src/main/java/com/google/cloud/gcs/analyticscore/client/GcsFileSystemImpl.java - Output Stream
core/src/main/java/com/google/cloud/gcs/analyticscore/core/GoogleCloudStorageOutputStream.java - GCS Client
client/src/main/java/com/google/cloud/gcs/analyticscore/client/GcsClientImpl.java - Authentication
client/src/main/java/com/google/cloud/gcs/analyticscore/client/auth/GcsAuthOptions.java
Shared Services
- Cache Manager
client/src/main/java/com/google/cloud/gcs/analyticscore/client/AnalyticsCacheManager.java - Cache Backends
common/src/main/java/com/google/cloud/gcs/analyticscore/common/cache/AnalyticsCacheCaffeineImpl.java - Telemetry
common/src/main/java/com/google/cloud/gcs/analyticscore/common/telemetry/Telemetry.java
Other components
- Analytics Engines
- GCS Java SDK
- Google Cloud Storage
Connections 22
- Analytics Engines → File-System Adapters: uses
README.md - File-System Adapters → GCS File System: opens reads
README.md - Core Options → GCS File System: configures
README.md - GCS File System → Input Stream: opens stream
README.md - GCS File System → Cache Manager: holds
README.md - Input Stream → Smart Read Channel: uses
core/src/main/java/com/google/cloud/gcs/analyticscore/core/GoogleCloudStorageInputStream.java - Input Stream → Vectored Reads: uses
core/src/main/java/com/google/cloud/gcs/analyticscore/core/GoogleCloudStorageInputStream.java - Input Stream → Footer Prefetch: invokes
core/src/main/java/com/google/cloud/gcs/analyticscore/core/GoogleCloudStorageInputStream.java - Input Stream → Small-Object Prefetch: invokes
core/src/main/java/com/google/cloud/gcs/analyticscore/core/GoogleCloudStorageInputStream.java - Footer Prefetch → Parquet Footer Parser: parses footer
- Footer Prefetch → Cache Manager: uses cache
README.md - Small-Object Prefetch → Cache Manager: uses cache
README.md - Vectored Reads → Cache Manager: reads cache
README.md - Input Stream → Adaptive Reads: adapts reads
- GCS File System → GCS Client: delegates operations
client/src/main/java/com/google/cloud/gcs/analyticscore/client/GcsFileSystemImpl.java - Output Stream → GCS File System: writes through
core/src/main/java/com/google/cloud/gcs/analyticscore/core/GoogleCloudStorageOutputStream.java - GCS File System → GCS Java SDK: bypasses writes
README.md - GCS Client → GCS Java SDK: issues requests
client/src/main/java/com/google/cloud/gcs/analyticscore/client/GcsClientImpl.java - Authentication → GCS Client: configures transport
client/src/main/java/com/google/cloud/gcs/analyticscore/client/GcsClientImpl.java - GCS Java SDK → Google Cloud Storage: accesses
README.md - GCS Client → Telemetry: records operations
client/src/main/java/com/google/cloud/gcs/analyticscore/client/GcsClientImpl.java - Cache Manager → Cache Backends: uses backend
client/src/main/java/com/google/cloud/gcs/analyticscore/client/AnalyticsCacheManager.java