How polaris-hub/polaris works
Project type and purpose
• This is a Python client library (“polaris-lib”) for AI-based drug-discovery benchmarking.
• It provides both a CLI (polaris/cli.py) and a Python API to:
• Download datasets & benchmarks from Polaris Hub
• Load data into standard objects
• Evaluate custom methods against benchmarks
• Submit results back to the Hub
High-level architecture from the file structure
• Top-level modules:
• cli: parses commands & invokes core API
• loader: abstracts how data is pulled locally vs. from Hub
• dataset: adapter/factory pattern to build dataset objects, converters (PDB, SDF, Zarr)
• benchmark & competition: define tasks, splits, metadata
• evaluate: metrics, results aggregation, submission helpers
• hub: HTTP/OAuth client, external client interface, storage interface
• mixins/utils: shared helpers (checksum, formatting, constants)
• Tests validate each module; docs/hosted assets reflect API surface
Key insights from README
• Core workflows:
a. po.load_benchmark(...) → backend calls to hub.client → fetch metadata & raw data
b. benchmark.get_train_test_split() → uses benchmark internals to create dataset splits
c. Iteration over dataset → uses loader, dataset objects, converters to produce Python primitives (arrays, pandas, etc.)
d. benchmark.evaluate(predictions) → invokes evaluate module, metric classes (strategy pattern)
e. results.upload_to_hub(...) → back through hub.external_client with OAuth