How ifixai-ai/ifixai works
iFixAi is a tool for independently auditing AI agents against organizational and operational expectations, combining adversarial testing with assurance checks. The main workflow configures a target agent and judge, loads a fixture, runs selected inspections, evaluates evidence, scores results, and produces reports and a scorecard. CLI, API, and agent-plugin usage are documented entry routes. The README documents the product flow and capabilities; source excerpts and index references support several internal links, but many inspection implementations and report paths were not sampled, so those connections are documented rather than inferred from names.
Main entry points: runner.py Run Orchestrator · api.py Python API · main.py CLI Commands · bootstrap.py Plugin Bootstrap
Components 20
User Entry
- CLI Commands
ifixai/cli/main.py - Plugin Bootstrap
plugin/hooks/bootstrap.py - Python API
ifixai/api.py
Audit Execution
- Fixture Loading
ifixai/core/fixture_loader.py - Provider Resolver
ifixai/providers/resolver.py - Provider Adapters
ifixai/providers/base.py - Concurrency Control
ifixai/core/concurrency.py - Run Orchestrator
ifixai/core/runner.py - Inspection Registry
ifixai/harness/registry.py - Inspection Suite
ifixai/inspections - Inspection Harness
ifixai/harness/base.py
Evaluation
- Evaluation Pipeline
ifixai/evaluation/pipeline.py - Analytic Judge
ifixai/evaluation/analytic_judge.py - Judge Provider
ifixai/judge/evaluator.py - Score Engine
ifixai/scoring/engine.py
Results
- Scorecard Reports
ifixai/reporting/scorecard.py - Run Artifacts
ifixai/reporting/artifact.py - Run Logging
ifixai/observability/logging.py
Other components
- Auditor
- Agent Operator
Connections 23
- Auditor → CLI Commands: runs commands
README.md - CLI Commands → Fixture Loading: loads setup
README.md - CLI Commands → Python API: starts audit
README.md - Agent Operator → Plugin Bootstrap: requests audit
README.md - Plugin Bootstrap → Fixture Loading: builds fixture
README.md - Plugin Bootstrap → Python API: runs diagnostic
README.md - Python API → Fixture Loading: loads fixture
ifixai/api.py - Python API → Provider Resolver: resolves provider
ifixai/api.py - Python API → Run Orchestrator: calls run
ifixai/api.py - Provider Resolver → Provider Adapters: selects adapter
ifixai/providers/resolver.py - Run Orchestrator → Inspection Registry: selects inspections
ifixai/core/runner.py - Run Orchestrator → Concurrency Control: coordinates work
ifixai/core/runner.py - Inspection Registry → Inspection Suite: registers tests
ifixai/harness/registry.py - Run Orchestrator → Inspection Harness: executes tests
ifixai/core/runner.py - Inspection Harness → Provider Adapters: sends prompts
ifixai/harness/base.py - Inspection Harness → Evaluation Pipeline: evaluates responses
ifixai/harness/base.py - Evaluation Pipeline → Analytic Judge: requests judgments
ifixai/evaluation/pipeline.py - Analytic Judge → Judge Provider: grades evidence
ifixai/evaluation/analytic_judge.py - Run Orchestrator → Score Engine: computes scores
ifixai/core/runner.py - Run Orchestrator → Scorecard Reports: builds scorecard
ifixai/core/runner.py - Auditor → Scorecard Reports: reviews results
README.md - Run Orchestrator → Run Artifacts: writes reports
README.md - Run Orchestrator → Run Logging: records events
ifixai/core/runner.py