How google/graph-mining works
Project type and purpose
• This is an open-source, shared-memory, highly-parallel C++ library for graph mining and clustering.
• It provides in-memory graph representations, a scheduler for parallel execution, and a suite of graph algorithms (clustering, connected components, shortest paths, triangle counting, etc.).
File structure
• Top-level:
• BUILD.* & WORKSPACE.bazel – Bazel build configuration
• .bazelrc, .gitignore, LICENSE, CONTRIBUTING.md, docs/ – repository metadata and contribution guidelines
• examples/ – quickstart/demo code
• utils/ – common utilities (containers, math, status, timers, Bazel macros)
• sketching/ – point-cloud encoding module
• in_memory/ – core graph abstractions + algorithm packages
• in_memory/ substructure:
• parallel/ – scheduler, parallel utilities
• clustering/ – dozens of subpackages (affinity, hac, correlation, parline, etc.), each with its own BUILD, .cc/.h, .proto, tests
• connected_components, generation, shortest_paths, tree_partitioner, triangle_counting – other algorithm suites
• status_macros.h – error/status handling
Extract README insights
• Mission: “most scalable library for graph algorithms”
• Focus today on clustering, with pointers to research papers
• Build & run via Bazel; Protobuf used for “.proto” config in each algorithm
1. Build & Tooling Layer
• Bazel (BUILD files, WORKSPACE)
• Protobuf compiler (for .proto → .cc/.h)
2. Utility Layer
• utils/ (containers, math, status, timer, build_defs.bzl)
3. Core Graph & Parallel Engine
• in_memory/graph.*, in_memory/parallel/ (scheduler, parallel_graph_utils)
• in_memory/status_macros.h
4. Algorithm Modules
• Clustering packages (affinity, hac, correlation, parline, etc.)
• Other algorithms (connected_components, shortest_paths, triangle_counting, generation, tree_partitioner)
5. Sketching Module
• sketching/point_cloud
6. Examples & Demos
• examples/quickstart.cc
• Build & Tooling → compiles Utilities, Core, Algorithms, Sketching, Examples
• Utilities → Core Graph & Parallel Engine (utility functions imported here)
• Core Graph & Parallel Engine → Algorithm Modules (graph data structures + parallel scheduler consumed by algorithms)
• Algorithm Modules → Examples (examples link against selected algorithms)
• Algorithm Modules (.proto) ↔ Protobuf compiler (code-gen step)
c. Architectural Patterns & Principles
• Layered architecture: clear separation of concerns (Utilities → Core → Algorithms → Consumers)
• Modular plugin style: each clustering algorithm is its own Bazel target, with well-defined interfaces (.proto definitions)
• High cohesion & low coupling: algorithms depend only on core abstractions and utils
d. Key Technologies & Frameworks
• C++17 (source files: .cc/.h)
• Bazel build system
• Protocol Buffers (.proto → code-gen)
• Eigen, farmhash (via utils/eigen.BUILD, utils/farmhash.BUILD)