# danielbmancini/cansa-vision architecture

> Architecture diagram of the GitHub repository danielbmancini/cansa-vision, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/danielbmancini/cansa-vision
- Repository: https://github.com/danielbmancini/cansa-vision

## Overview

1. **What this repository does**
   - `Cansa-Vision` is a document-processing pipeline for handwritten time cards (“cartões-ponto”).
   - It turns scanned images or photos of point cards into structured time entries and then writes those entries into an Excel template.

2. **Overall architecture**
   - The repo is a small, script-driven pipeline rather than an application server.
   - The flow is: **input image/document → OCR extraction → text normalization/interpretation → spreadsheet population**.
   - There are two AI services in the chain:
     - **Azure Computer Vision** for OCR.
     - **Vertex AI / gcloud** for higher-level interpretation of the raw OCR output.

3. **Main orchestration layer**
   - `processar.sh` is the primary document-processing entrypoint for a single file.
   - `integrar.sh` is the end-to-end shortcut that takes a document plus user parameters and produces the filled spreadsheet from the template.
   - These scripts are the top-level boundary between the user and the rest of the pipeline.

4. **OCR ingestion boundary**
   - The OCR stage uses Azure Computer Vision credentials from environment variables:
     - `AI_SERVICE_KEY`
     - `AI_SERVICE_ENDPOINT`
   - `body.json` appears to be the request payload template used when calling the Azure OCR endpoint.
   - `curl` is required, so the OCR step is implemented as an HTTP API call from shell rather than through a dedicated SDK.

5. **Text post-processing / normalization**
   - The repo includes Unix text-processing tools and scripts such as `organizar.awk`, plus `sed`, `jq`, and shell utilities.
   - This layer reshapes the raw OCR output into a more regular intermediate form.
   - Artifacts like `output.txt`, `fin4pontos.txt`, and `outvertex.txt` suggest staged outputs between OCR, AI interpretation, and final structured data.

6. **Semantic interpretation with Vertex AI**
   - `vertex.py` is the Python component that talks to Vertex AI.
   - The README indicates it uses `vertexai.init(...)`, so it depends on **Google Cloud application-default authentication** via `gcloud auth application-default login`.
   - This stage reduces assumptions about the document structure and helps recover time entries from noisy OCR text.

7. **Spreadsheet generation layer**
   - `excel.py` is the Python component that writes the processed point data into Excel.
   - The template-based workflow centers on `modelo.xlsx`, with a real employee workbook such as `funcionario.xlsx` as the target/output example.
   - The script takes a start day, workbook name, sheet name, and employee name, so the Excel stage is parameterized rather than hardcoded.

8. **Data contract between stages**
   - The pipeline is designed around a text intermediate representation rather than passing binary data between components.
   - The practical handoff format is likely the processed point list in `fin4pontos.txt`, which is then consumed by `excel.py`.
   - The README’s example `sed` cleanup step before Excel insertion indicates the final stage expects a strict text format.

9. **Domain assumptions and constraints**
   - The repo encodes business rules for timesheet interpretation, such as:
     - times are valid and relevant to the table,
     - non-time numbers are rare,
     - entry/exit ordering follows shift logic,
     - documents are scanned/straightened images,
     - absences and special days are marked.
   - These assumptions are important because they shape how OCR errors and ambiguous handwriting are resolved.

10. **Inputs, outputs, and sample assets**
   - `pontos/` holds example source documents.
   - `modelo.xlsx` is the spreadsheet template.
   - `funcionario.xlsx` is an example filled workbook or target output.
   - `prompt/` likely contains the prompt text used for AI interpretation.
   - `output.txt`, `outvertex.txt`, and `fin4pontos.txt` are trace/debug or intermediate outputs, useful for diagnosing each pipeline stage.

11. **Runtime and environment boundaries**
   - Python is used for the AI/spreadsheet stages, while shell/Unix tools handle orchestration and text manipulation.
   - The repo assumes a Linux-like environment; Windows users need extra tooling (`awk`, `sed`, `jq`).
   - Required cloud setup spans two vendors: Azure for OCR and Google Cloud for Vertex AI.

12. **How to draw the architecture**
   - A clean diagram would show:
     - **User / source document**
     - **Shell orchestration layer** (`processar.sh`, `integrar.sh`)
     - **Azure OCR service**
     - **Text normalization layer** (`awk`, `sed`, `jq`, intermediate text files)
     - **Vertex AI interpretation**
     - **Excel writer** (`excel.py`)
     - **Final spreadsheet output**
   - The key architectural idea is that this repo is a **multi-stage document understanding pipeline**, not a monolithic app.

Main entry points: [`excel.py`](https://github.com/danielbmancini/cansa-vision/blob/main/excel.py) (excel.py), [`processar.sh`](https://github.com/danielbmancini/cansa-vision/blob/main/processar.sh) (processar.sh), [`vertex.py`](https://github.com/danielbmancini/cansa-vision/blob/main/vertex.py) (vertex.py), [`integrar.sh`](https://github.com/danielbmancini/cansa-vision/blob/main/integrar.sh) (integrar.sh)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_entry["Entry & orchestration"]
  node_n_processar["processar.sh<br/>shell entrypoint<br/>[processar.sh]"]
  node_n_integrar["integrar.sh<br/>shell pipeline<br/>[integrar.sh]"]
end

subgraph group_group_ocr["OCR boundary"]
  node_n_bodyjson["body.json<br/>ocr payload<br/>[body.json]"]
  node_n_outputtxt["output.txt<br/>ocr output<br/>[output.txt]"]
end

subgraph group_group_transform["Text transform"]
  node_n_organizar["organizar.awk<br/>text filter<br/>[organizar.awk]"]
  node_n_texttools["sed / jq<br/>unix transforms"]
  node_n_fin4["fin4pontos.txt<br/>intermediate text<br/>[fin4pontos.txt]"]
end

subgraph group_group_vertex["Semantic interpretation"]
  node_n_outvertex["outvertex.txt<br/>vertex output<br/>[outvertex.txt]"]
  node_n_vertexpy["vertex.py<br/>vertex client<br/>[vertex.py]"]
end

subgraph group_group_excel["Spreadsheet output"]
  node_n_excelpy["excel.py<br/>excel writer<br/>[excel.py]"]
  node_n_modelo["modelo.xlsx<br/>template workbook<br/>[modelo.xlsx]"]
  node_n_funcionario["funcionario.xlsx<br/>filled workbook<br/>[funcionario.xlsx]"]
end

subgraph group_group_assets["Assets & samples"]
  node_n_pontos[("pontos/<br/>source docs")]
  node_n_prompt["prompt/<br/>prompt assets"]
end

node_n_azure[("Azure CV<br/>ocr service")]
node_n_vertexai[("Vertex AI<br/>llm service")]

node_n_pontos -->|"input file"| node_n_processar
node_n_integrar -->|"invokes"| node_n_processar
node_n_processar -->|"builds request"| node_n_bodyjson
node_n_bodyjson -->|"HTTP OCR"| node_n_azure
node_n_azure -->|"raw text"| node_n_outputtxt
node_n_outputtxt -->|"normalize"| node_n_organizar
node_n_organizar -->|"refine"| node_n_texttools
node_n_texttools -->|"stage data"| node_n_fin4
node_n_fin4 -->|"interpret"| node_n_vertexpy
node_n_prompt -->|"guidance"| node_n_vertexpy
node_n_vertexpy -->|"API call"| node_n_vertexai
node_n_vertexai -->|"structured text"| node_n_outvertex
node_n_outvertex -->|"consume"| node_n_excelpy
node_n_modelo -->|"template"| node_n_excelpy
node_n_excelpy -->|"write workbook"| node_n_funcionario
node_n_integrar -->|"orchestrates"| node_n_excelpy

click node_n_processar "https://github.com/danielbmancini/cansa-vision/blob/main/processar.sh"
click node_n_integrar "https://github.com/danielbmancini/cansa-vision/blob/main/integrar.sh"
click node_n_bodyjson "https://github.com/danielbmancini/cansa-vision/blob/main/body.json"
click node_n_outputtxt "https://github.com/danielbmancini/cansa-vision/blob/main/output.txt"
click node_n_organizar "https://github.com/danielbmancini/cansa-vision/blob/main/organizar.awk"
click node_n_fin4 "https://github.com/danielbmancini/cansa-vision/blob/main/fin4pontos.txt"
click node_n_outvertex "https://github.com/danielbmancini/cansa-vision/blob/main/outvertex.txt"
click node_n_vertexpy "https://github.com/danielbmancini/cansa-vision/blob/main/vertex.py"
click node_n_excelpy "https://github.com/danielbmancini/cansa-vision/blob/main/excel.py"
click node_n_modelo "https://github.com/danielbmancini/cansa-vision/blob/main/modelo.xlsx"
click node_n_funcionario "https://github.com/danielbmancini/cansa-vision/blob/main/funcionario.xlsx"
click node_n_pontos "https://github.com/danielbmancini/cansa-vision/tree/main/pontos"
click node_n_prompt "https://github.com/danielbmancini/cansa-vision/tree/main/prompt"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n_processar,node_n_integrar toneBlue
class node_n_bodyjson,node_n_outputtxt toneAmber
class node_n_organizar,node_n_texttools,node_n_fin4 toneMint
class node_n_outvertex,node_n_vertexpy toneRose
class node_n_excelpy,node_n_modelo,node_n_funcionario toneIndigo
class node_n_pontos,node_n_prompt toneTeal
class node_n_azure,node_n_vertexai toneNeutral
```

## Components

### Entry & orchestration

Shell entrypoints that drive the document-processing pipeline.

- [processar.sh](https://github.com/danielbmancini/cansa-vision/blob/main/processar.sh): shell entrypoint · Primary single-file processing script. · `processar.sh`
- [integrar.sh](https://github.com/danielbmancini/cansa-vision/blob/main/integrar.sh): shell pipeline · End-to-end shortcut from document to filled spreadsheet. · `integrar.sh`

### OCR boundary

Azure OCR request/response handling and raw extraction artifacts.

- [body.json](https://github.com/danielbmancini/cansa-vision/blob/main/body.json): ocr payload · Request body template for the Azure OCR call. · `body.json`
- [output.txt](https://github.com/danielbmancini/cansa-vision/blob/main/output.txt): ocr output · Raw OCR stage artifact. · `output.txt`

### Text transform

Shell and text-processing stages that regularize OCR output.

- [organizar.awk](https://github.com/danielbmancini/cansa-vision/blob/main/organizar.awk): text filter · AWK normalization step for OCR text. · `organizar.awk`
- **sed / jq**: unix transforms · Shell text-processing utilities used to reshape intermediate data.
- [fin4pontos.txt](https://github.com/danielbmancini/cansa-vision/blob/main/fin4pontos.txt): intermediate text · Structured point list handed toward spreadsheet population. · `fin4pontos.txt`

### Semantic interpretation

Vertex AI-based recovery of structured time-card entries.

- [outvertex.txt](https://github.com/danielbmancini/cansa-vision/blob/main/outvertex.txt): vertex output · Interpreted text artifact produced by the AI step. · `outvertex.txt`
- [vertex.py](https://github.com/danielbmancini/cansa-vision/blob/main/vertex.py): vertex client · Python component that calls Vertex AI. · `vertex.py`

### Spreadsheet output

Excel template filling and final workbook generation.

- [excel.py](https://github.com/danielbmancini/cansa-vision/blob/main/excel.py): excel writer · Python component that fills the workbook template. · `excel.py`
- [modelo.xlsx](https://github.com/danielbmancini/cansa-vision/blob/main/modelo.xlsx): template workbook · Base Excel template for time-card population. · `modelo.xlsx`
- [funcionario.xlsx](https://github.com/danielbmancini/cansa-vision/blob/main/funcionario.xlsx): filled workbook · Example target/output workbook. · `funcionario.xlsx`

### Assets & samples

Input documents, prompts, templates, and example outputs.

- [pontos/](https://github.com/danielbmancini/cansa-vision/tree/main/pontos): source docs · Example scanned input documents. · `pontos`
- [prompt/](https://github.com/danielbmancini/cansa-vision/tree/main/prompt): prompt assets · Prompt text used for AI interpretation. · `prompt`

### Other components

- **Azure CV**: ocr service · External OCR service boundary used via HTTP.
- **Vertex AI**: llm service · Google Cloud semantic interpretation boundary.

## Connections

- pontos/ → processar.sh: input file · User-selected scan or photo enters the processing script.
- integrar.sh → processar.sh: invokes · End-to-end wrapper delegates to the main processing flow.
- processar.sh → body.json: builds request · Prepares the OCR payload for the Azure request.
- body.json → Azure CV: HTTP OCR · Shell script posts the payload to the OCR service.
- Azure CV → output.txt: raw text · OCR response is captured as a text artifact.
- output.txt → organizar.awk: normalize · AWK reshapes the OCR output into a cleaner intermediate form.
- organizar.awk → sed / jq: refine · Shell text tools perform cleanup and formatting.
- sed / jq → fin4pontos.txt: stage data · Intermediate structured text is produced for downstream use.
- fin4pontos.txt → vertex.py: interpret · Python reads processed text for semantic recovery.
- prompt/ → vertex.py: guidance · Prompt assets shape the interpretation step.
- vertex.py → Vertex AI: API call · Vertex client sends the document understanding request.
- Vertex AI → outvertex.txt: structured text · AI interpretation is stored as a text artifact.
- outvertex.txt → excel.py: consume · Spreadsheet writer consumes the interpreted points.
- modelo.xlsx → excel.py: template · Workbook template is used as the base for population.
- excel.py → funcionario.xlsx: write workbook · Final Excel file is generated from the template and processed data.
- integrar.sh → excel.py: orchestrates · Integrated flow reaches the workbook-writing stage.