How raga-ai-hub/ragaai-catalyst works
RagaAI Catalyst is a Python client for managing LLM projects and their datasets, evaluations, prompts, and safeguards, alongside data-generation and tracing workflows. Users authenticate through the client, then call management APIs; tracing records application activity and uploads it for analysis, while red-teaming generates and evaluates adversarial cases. The README documents the major capabilities, and the source index supports several internal tracing and red-teaming relationships. Some branches are represented by their documented entry modules because their implementation was not sampled; uncertain runtime links are omitted.
Main entry points: tracer.py Trace management · red_teaming.py Red-team workflow · base.py Trace lifecycle · synthetic_data_generation.py Synthetic data
Components 25
Project management
- Catalyst client
ragaai_catalyst/ragaai_catalyst.py - Project operations
ragaai_catalyst/ragaai_catalyst.py
Data and evaluation
- Dataset management
ragaai_catalyst/dataset.py - Metric evaluation
ragaai_catalyst/evaluation.py - Experiment management
ragaai_catalyst/experiment.py
Tracing
- Trace management
ragaai_catalyst/tracers/tracer.py - Framework callbacks
ragaai_catalyst/tracers/langchain_callback.py - Agentic tracers
ragaai_catalyst/tracers/agentic_tracing/tracers/llm_tracer.py - Trace data model
ragaai_catalyst/tracers/agentic_tracing/data/data_structure.py - Trace lifecycle
ragaai_catalyst/tracers/agentic_tracing/tracers/base.py - Agentic upload
ragaai_catalyst/tracers/agentic_tracing/upload/trace_uploader.py - Trace export
ragaai_catalyst/tracers/exporters/raga_exporter.py
Safety and prompts
- Guardrail management
ragaai_catalyst/guardrails_manager.py - Guard execution
ragaai_catalyst/guard_executor.py - Prompt management
ragaai_catalyst/prompt_manager.py - Red-team workflow
ragaai_catalyst/redteaming/red_teaming.py - Scenario generation
ragaai_catalyst/redteaming/data_generator/scenario_generator.py - Adversarial evaluation
ragaai_catalyst/redteaming/evaluator.py - LLM generation
ragaai_catalyst/redteaming/llm_generator.py
Data generation
- Synthetic data
ragaai_catalyst/synthetic_data_generation.py - Proxy calls
ragaai_catalyst/proxy_call.py - Completion API
ragaai_catalyst/internal_api_completion.py
Other components
- Catalyst user
- Catalyst platform
- LLM provider
Connections 27
- Catalyst user → Catalyst client: initializes
README.md - Catalyst client → Catalyst platform: authenticates
ragaai_catalyst/ragaai_catalyst.py - Catalyst user → Project operations: manages projects
README.md - Catalyst user → Dataset management: manages datasets
README.md - Catalyst user → Metric evaluation: runs evaluations
README.md - Metric evaluation → Dataset management: evaluates dataset
README.md - Catalyst user → Trace management: records traces
README.md - Trace management → Framework callbacks: uses callbacks
ragaai_catalyst/tracers/tracer.py - Trace management → Agentic tracers: initializes
ragaai_catalyst/tracers/tracer.py - Agentic tracers → Trace lifecycle: extends
ragaai_catalyst/tracers/agentic_tracing/tracers/llm_tracer.py - Trace lifecycle → Trace data model: constructs
ragaai_catalyst/tracers/agentic_tracing/tracers/base.py - Trace lifecycle → Agentic upload: submits uploads
ragaai_catalyst/tracers/agentic_tracing/tracers/base.py - Trace management → Trace export: configures exporter
ragaai_catalyst/tracers/tracer.py - Trace export → Catalyst platform: uploads traces
ragaai_catalyst/tracers/exporters/raga_exporter.py - Catalyst user → Prompt management: manages prompts
README.md - Catalyst user → Guardrail management: manages safeguards
README.md - Catalyst user → Guard execution: executes safeguards
README.md - Catalyst user → Red-team workflow: tests applications
README.md - Red-team workflow → Scenario generation: generates scenarios
ragaai_catalyst/redteaming/red_teaming.py - Red-team workflow → Adversarial evaluation: evaluates cases
ragaai_catalyst/redteaming/red_teaming.py - Red-team workflow → LLM generation: uses generation
ragaai_catalyst/redteaming/red_teaming.py - Scenario generation → LLM generation: generates with
ragaai_catalyst/redteaming/data_generator/scenario_generator.py - Catalyst user → Synthetic data: generates data
README.md - Synthetic data → Proxy calls: uses proxy
ragaai_catalyst/synthetic_data_generation.py - Synthetic data → Completion API: calls completion
ragaai_catalyst/synthetic_data_generation.py - LLM generation → LLM provider: requests generation
ragaai_catalyst/redteaming/llm_generator.py - Guardrail management → Catalyst platform: sends requests
ragaai_catalyst/guardrails_manager.py