# codebyfernanda/desafio_maspacessivel architecture

> Architecture diagram of the GitHub repository codebyfernanda/desafio_maspacessivel, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/codebyfernanda/desafio_maspacessivel
- Repository: https://github.com/codebyfernanda/desafio_maspacessivel

## Overview

MASP Acessível is an evaluation campaign for a MASP guide agent, not the guide-agent runtime itself. Its main workflow runs golden and red-team cases against baseline or corrected Qwen configurations, records answers and RAG evidence, exports AgentCore spans, and evaluates the captured results through two independent fronts: AgentCore evaluators and DeepEval with a local DeepSeek-R1 judge. The repository also preserves comparison outputs and final reporting artifacts. Wiring is based on the README and sampled source; unsampled evaluation modules are represented only where the documented workflow supports them.

Main entry points: [`avaliacoes/campanha.py`](https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/avaliacoes/campanha.py) (Campaign runner), [`avaliacoes/avaliacao.py`](https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/avaliacoes/avaliacao.py) (Saved-answer evaluation), [`avaliacoes/src/agent_core/executar.py`](https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/avaliacoes/src/agent_core/executar.py) (AWS span evaluation), [`avaliacoes/src/agent_core/exportar_spans.py`](https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/avaliacoes/src/agent_core/exportar_spans.py) (Span exporter)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_campaign["Campaign inputs"]
  node_datasets["Golden and attacks"]
  node_configs["Baseline and final"]
end

subgraph group_capture["Capture and traces"]
  node_campaign_runner["Campaign runner<br/>[campanha.py]"]
  node_agentcore{{"Qwen on AgentCore"}}
  node_rag_evidence["RAG evidence parser<br/>[evidencias_rag.py]"]
  node_captures[("Captured responses")]
  node_cloudwatch[("CloudWatch Logs")]
  node_span_export["Span exporter<br/>[exportar_spans.py]"]
  node_spans["Exported spans"]
end

subgraph group_front_a["AgentCore evaluation"]
  node_agent_eval["AWS span evaluation<br/>[executar.py]"]
  node_custom_lambda["Domain rule evaluator"]
end

subgraph group_front_b["DeepEval evaluation"]
  node_deepeval_runner["Saved-answer evaluation<br/>[avaliacao.py]"]
  node_case_builder["Case construction<br/>[casos.py]"]
  node_faithfulness["Source fidelity metric<br/>[fidelidade.py]"]
  node_templates["Evaluation prompts<br/>[templates.py]"]
  node_local_judge{{"DeepSeek-R1 judge"}}
  node_ollama_adapter["Ollama judge adapter<br/>[juiz.py]"]
end

subgraph group_outputs["Results and reporting"]
  node_front_a_results[("Front A results")]
  node_front_b_results[("Front B results")]
  node_reports["Comparison and report<br/>[RELATORIO.md]"]
end

node_analyst(("Evaluator"))

node_analyst -->|"starts run"| node_campaign_runner
node_campaign_runner -->|"loads cases"| node_datasets
node_campaign_runner -->|"selects version"| node_configs
node_campaign_runner -->|"invokes harness"| node_agentcore
node_agentcore -->|"streams replies"| node_campaign_runner
node_campaign_runner -->|"analyzes events"| node_rag_evidence
node_campaign_runner -->|"records answers"| node_captures
node_span_export -->|"queries spans"| node_cloudwatch
node_span_export -->|"reads session IDs"| node_captures
node_span_export -->|"writes spans"| node_spans
node_campaign_runner -->|"starts export"| node_span_export
node_campaign_runner -->|"starts evaluation"| node_agent_eval
node_agent_eval -->|"reads spans"| node_spans
node_agent_eval -->|"invokes evaluator"| node_custom_lambda
node_agent_eval -->|"writes scores"| node_front_a_results
node_analyst -->|"starts evaluation"| node_deepeval_runner
node_deepeval_runner -->|"reads captured cases"| node_captures
node_deepeval_runner -->|"builds test cases"| node_case_builder
node_deepeval_runner -->|"runs fidelity metric"| node_faithfulness
node_deepeval_runner -->|"uses prompts"| node_templates
node_deepeval_runner -->|"requests judgments"| node_ollama_adapter
node_faithfulness -->|"judges claims"| node_ollama_adapter
node_ollama_adapter -->|"sends prompts"| node_local_judge
node_deepeval_runner -->|"writes scores"| node_front_b_results
node_front_a_results -->|"informs findings"| node_reports
node_front_b_results -->|"informs findings"| node_reports
node_reports -->|"presents results"| node_analyst

click node_datasets "https://github.com/codebyfernanda/desafio_maspacessivel/tree/main/avaliacoes/dados"
click node_configs "https://github.com/codebyfernanda/desafio_maspacessivel/tree/main/avaliacoes/config"
click node_campaign_runner "https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/avaliacoes/campanha.py"
click node_rag_evidence "https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/avaliacoes/evidencias_rag.py"
click node_captures "https://github.com/codebyfernanda/desafio_maspacessivel/tree/main/avaliacoes/resultados/frente_a"
click node_span_export "https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/avaliacoes/src/agent_core/exportar_spans.py"
click node_spans "https://github.com/codebyfernanda/desafio_maspacessivel/tree/main/avaliacoes/resultados/frente_a"
click node_agent_eval "https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/avaliacoes/src/agent_core/executar.py"
click node_custom_lambda "https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/avaliacoes/src/agent_core/lambda_avaliador.py"
click node_front_a_results "https://github.com/codebyfernanda/desafio_maspacessivel/tree/main/avaliacoes/resultados/frente_a"
click node_deepeval_runner "https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/avaliacoes/avaliacao.py"
click node_case_builder "https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/avaliacoes/src/deepeval/casos.py"
click node_faithfulness "https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/avaliacoes/src/deepeval/fidelidade.py"
click node_templates "https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/avaliacoes/src/deepeval/templates.py"
click node_ollama_adapter "https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/avaliacoes/src/deepeval/juiz.py"
click node_front_b_results "https://github.com/codebyfernanda/desafio_maspacessivel/tree/main/avaliacoes/resultados/frente_b"
click node_reports "https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/entrega_resumida/RELATORIO.md"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_datasets,node_configs toneBlue
class node_campaign_runner,node_agentcore,node_rag_evidence,node_captures,node_cloudwatch,node_span_export,node_spans toneAmber
class node_agent_eval,node_custom_lambda toneMint
class node_deepeval_runner,node_case_builder,node_faithfulness,node_templates,node_local_judge,node_ollama_adapter toneRose
class node_front_a_results,node_front_b_results,node_reports,node_analyst toneIndigo
```

## Components

### Campaign inputs

- [Golden and attacks](https://github.com/codebyfernanda/desafio_maspacessivel/tree/main/avaliacoes/dados): `avaliacoes/dados`
- [Baseline and final](https://github.com/codebyfernanda/desafio_maspacessivel/tree/main/avaliacoes/config): `avaliacoes/config`

### Capture and traces

- [Campaign runner](https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/avaliacoes/campanha.py): `avaliacoes/campanha.py`
- **Qwen on AgentCore**
- [RAG evidence parser](https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/avaliacoes/evidencias_rag.py): `avaliacoes/evidencias_rag.py`
- [Captured responses](https://github.com/codebyfernanda/desafio_maspacessivel/tree/main/avaliacoes/resultados/frente_a): `avaliacoes/resultados/frente_a`
- **CloudWatch Logs**
- [Span exporter](https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/avaliacoes/src/agent_core/exportar_spans.py): `avaliacoes/src/agent_core/exportar_spans.py`
- [Exported spans](https://github.com/codebyfernanda/desafio_maspacessivel/tree/main/avaliacoes/resultados/frente_a): `avaliacoes/resultados/frente_a`

### AgentCore evaluation

- [AWS span evaluation](https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/avaliacoes/src/agent_core/executar.py): `avaliacoes/src/agent_core/executar.py`
- [Domain rule evaluator](https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/avaliacoes/src/agent_core/lambda_avaliador.py): `avaliacoes/src/agent_core/lambda_avaliador.py`

### DeepEval evaluation

- [Saved-answer evaluation](https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/avaliacoes/avaliacao.py): `avaliacoes/avaliacao.py`
- [Case construction](https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/avaliacoes/src/deepeval/casos.py): `avaliacoes/src/deepeval/casos.py`
- [Source fidelity metric](https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/avaliacoes/src/deepeval/fidelidade.py): `avaliacoes/src/deepeval/fidelidade.py`
- [Evaluation prompts](https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/avaliacoes/src/deepeval/templates.py): `avaliacoes/src/deepeval/templates.py`
- **DeepSeek-R1 judge**
- [Ollama judge adapter](https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/avaliacoes/src/deepeval/juiz.py): `avaliacoes/src/deepeval/juiz.py`

### Results and reporting

- [Front A results](https://github.com/codebyfernanda/desafio_maspacessivel/tree/main/avaliacoes/resultados/frente_a): `avaliacoes/resultados/frente_a`
- [Front B results](https://github.com/codebyfernanda/desafio_maspacessivel/tree/main/avaliacoes/resultados/frente_b): `avaliacoes/resultados/frente_b`
- [Comparison and report](https://github.com/codebyfernanda/desafio_maspacessivel/blob/main/entrega_resumida/RELATORIO.md): `entrega_resumida/RELATORIO.md`

### Other components

- **Evaluator**

## Connections

- Evaluator → Campaign runner: starts run
- Campaign runner → Golden and attacks: loads cases
- Campaign runner → Baseline and final: selects version
- Campaign runner → Qwen on AgentCore: invokes harness
- Qwen on AgentCore → Campaign runner: streams replies
- Campaign runner → RAG evidence parser: analyzes events
- Campaign runner → Captured responses: records answers
- Span exporter → CloudWatch Logs: queries spans
- Span exporter → Captured responses: reads session IDs
- Span exporter → Exported spans: writes spans
- Campaign runner → Span exporter: starts export
- Campaign runner → AWS span evaluation: starts evaluation
- AWS span evaluation → Exported spans: reads spans
- AWS span evaluation → Domain rule evaluator: invokes evaluator
- AWS span evaluation → Front A results: writes scores
- Evaluator → Saved-answer evaluation: starts evaluation
- Saved-answer evaluation → Captured responses: reads captured cases
- Saved-answer evaluation → Case construction: builds test cases
- Saved-answer evaluation → Source fidelity metric: runs fidelity metric
- Saved-answer evaluation → Evaluation prompts: uses prompts
- Saved-answer evaluation → Ollama judge adapter: requests judgments
- Source fidelity metric → Ollama judge adapter: judges claims
- Ollama judge adapter → DeepSeek-R1 judge: sends prompts
- Saved-answer evaluation → Front B results: writes scores
- Front A results → Comparison and report: informs findings
- Front B results → Comparison and report: informs findings
- Comparison and report → Evaluator: presents results