epicgames/lore

Purpose and executables

Lore is a Rust, centralized version-control system for large binary and source assets. It models repository state as content-addressed Merkle trees plus an immutable revision chain, with lightweight mutable branch references.

Primary entry points:

• CLI executable: lore-client/src/cli/main.rs

• Long-running server executable: lore-server/src/bin/loreserver/main.rs

• Embeddable core/FFI library: lore/src/lib.rs, lore-capi/lore.h

Public API and client boundary

The lore crate is the high-level library surface used by native consumers and language bindings. It exposes repository, revision, branch, file, storage, lock, and service operations, delegating either to local revision logic or remote services.

• lore/src/interface.rs

• lore/src/repository.rs

• lore/src/remote/connection.rs

The CLI parses commands and configuration, invokes this library, and presents progress/logging; it is a client frontend rather than the revision engine.

• lore-client/src/cli/commands.rs

• lore-client/src/cli/client_main.rs

Revision-domain engine

lore-revision contains the domain model and orchestration for repositories, revisions, branches, staging, merges, dependencies, working filesystem realization, and synchronization. This is the key boundary between user-facing version-control operations and storage primitives.

• lore-revision/src/lore.rs

• lore-revision/src/revision.rs

• lore-revision/src/repository.rs

Typical write flow is workspace changes → stage/file state → revision state/Merkle objects → immutable content storage → mutable branch-head update. Read/checkout flows resolve branch/revision state, hydrate required content, then realize it in the filesystem.

• lore-revision/src/stage.rs

• lore-revision/src/store/immutable.rs

• lore-revision/src/fs/realize.rs

Content-addressed storage layer

lore-storage implements local immutable and mutable stores, chunking, compression, fragment handling, pack storage, and maintenance. Immutable objects support deduplication and integrity by hash; mutable data holds changeable references/metadata such as branch state.

• lore-storage/src/fragment_engine.rs

• lore-storage/src/immutable_store.rs

• lore-storage/src/mutable_store.rs

The revision engine composes local, remote, and replica-backed stores, separating repository semantics from durability and replication mechanisms.

• lore-revision/src/store/composite.rs

• lore-revision/src/store/remote.rs

• lore-revision/src/store/packstore.rs

Server protocols and request handling

Lore Server hosts multiple access paths: versioned gRPC APIs for repository/revision/storage operations, HTTP endpoints for health and presigned content access, and QUIC storage/replication services. Protobuf contracts are the compatibility boundary for gRPC clients and services.

• lore-proto/proto/lore/revision/v1/revision.proto

• lore-server/src/grpc/server.rs

• lore-server/src/http/server.rs

gRPC handlers authenticate/authorize requests, resolve repository and execution context, then invoke revision/store services. Forwarded-request modules support routing operations to other service instances.

• lore-server/src/grpc/handlers/mod.rs

• lore-server/src/grpc/forwarded_requests.rs

• lore-server/src/execution_state.rs

Security, clustering, and extensibility

JWT/JWK authentication and repository authorization sit at server ingress; credentials are also managed client-side. Locking, notifications, hooks, and plugin registries are server-side cross-cutting integrations.

• lore-server/src/auth/jwt.rs

• lore-server/src/authnz/repository_authorizer.rs

• lore-credential/src/token_store.rs

Server topology and replicated-store abstractions support multi-instance operation. AWS plugins provide S3-backed immutable storage and DynamoDB-backed mutable/lock storage; HashiCorp support provides Consul peer discovery and Nomad telemetry.

• lore-server/src/store/replicated_store.rs

• lore-aws/src/store/immutable_store.rs

• lore-hashicorp/src/consul/service_peer_discovery.rs

Runtime and infrastructure

The system is Rust-based with asynchronous networking/runtime support, custom I/O backends, TLS, tracing, metrics, and correlation propagation.

• lore-base/src/runtime.rs

• lore-io/src/lib.rs

• lore-telemetry/src/metrics/mod.rs

Deployment artifacts include a containerized server and an AWS Terraform example provisioning network, compute, IAM, storage, and TLS-related infrastructure.

• lore-server/Dockerfile

• contrib/aws/main.tf

• contrib/aws/storage.tf