How oracle-samples/oci-data-science-ai-samples works
Project Type and Purpose
– This is a large monorepo of code examples, tutorials, and demos for Oracle Cloud Infrastructure Data Science and AI services.
– It’s not a single running application but a “pattern library” illustrating how to use OCI Data Science: notebooks, jobs, pipelines, model catalog, feature store, distributed training, LLM integration, AI services, container deployments, and ML Applications.
File Structure
– Top‐level folders correspond to OCI Data Science service capabilities:
• notebook_examples, conda_environment_notebooks, labs
• LLM (large language models)
• model_catalog_examples, model_deploy_examples
• jobs, distributed_training, pipelines
• feature_store, ml-applications
• ai_services (vision, language, anomaly_detection, document_understanding)
• model-deployment containers (Triton, llama2, misc)
– CI/CD: .github/workflows and GitHub Actions under “actions/”
– Config, license, readmes at root
README
– It calls out each capability as a “Topic” mapping to a folder.
– Emphasizes ADS SDK, OCI service APIs, and how examples drive: notebooks → jobs → pipelines → deployments → feature store, model catalog, ML Apps.
Main Components
a. Users
– Data Scientists / Developers using JupyterLab, CLI, or ADS SDK
b. Local Example Code
– Notebooks, Python/Java/.NET samples, Terraform manifests, container definitions
c. ADS SDK Layer
– Accelerated Data Science Python library that abstracts OCI service APIs
d. OCI Data Science Services
– Notebook Session Service
– Jobs Service (batch, distributed training)
– Pipelines Service
– Model Catalog Service (models & artifacts storage)
– Feature Store Service
e. OCI ML Applications Service
f. OCI AI Services
– Vision, Language, Document Understanding, Anomaly Detection
g. Container Deployment Targets
– Custom inference containers (Triton, llama2, vLLM, TensorRT, Mistral, RAG)
h. OCI Core Infrastructure
– Compute (VMs, Containers), Object Storage, Networking, IAM, Container Registry
i. CI/CD
– GitHub Actions for README generation, tests, publishing examples
Relationships and Data Flows
– User → Local Examples (edit/run)
– Local Examples (notebook or script) call ADS SDK or CLI → OCI Data Science APIs
– Notebook Service / Jobs Service / Pipelines Service orchestrate compute on OCI Compute
– Jobs and Pipelines read/write data from Object Storage and push features to Feature Store
– Training Jobs publish models to Model Catalog
– Model Deployment examples launch containers via OCI ML Applications or custom containers
– AI Services invoked directly by examples for vision, language tasks
– Terraform and OCI SDK scripts under ml-applications and feature_store manage infrastructure provisioning
– CI/CD Actions validate samples, generate docs, enforce code quality
Architectural Patterns and Principles
– Modular folder‐per‐capability structure (separation of concerns)
– “Infrastructure as Code” for provisioning (Terraform modules, Helm charts)
– Example‐centric: each folder is a self‐contained demonstration
– Reuse of ADS SDK as a single interface to all services
Be Detailed, Not Overly Complex
– Capture every major folder‐to‐service mapping as a component.
– Represent both control flow (API calls) and data flow (model, data, artifacts).
– Emphasize how tutorials and samples map to real OCI services and features.
– Avoid unnecessary internal sub‐folder detail—focus on service boundaries and integration points.