How kashish1530/evaluation_project works
This is an evaluation-driven HR policy question-answering system: a user question is checked for prohibited requests and policy exceptions, otherwise matched to a mock policy document and returned with safeguards. The v5 pipeline adds PII redaction and audit logging around the earlier retrieval pipeline; retrieval uses local TF-IDF/cosine ranking and stemming. Exception requests are filed for human review rather than answered automatically. The tree and sampled source support these main paths, but do not show how the CLI connects to the pipeline or how human reviewers process approvals.
Main entry points: pipeline_v5.py Answer pipeline v5 · pipeline_v3.py Retrieval pipeline v3 · retrieval_v3.py TF-IDF retrieval · approval.py Exception approval
Components 13
Answer pipeline
- Answer pipeline v5
pipeline_v5.py - Retrieval pipeline v3
pipeline_v3.py
Policy retrieval
- TF-IDF retrieval
retrieval_v3.py - Token stemming
stemmer.py
Safety and governance
- Refusal router
router.py - Exception approval
approval.py - PII redaction
redaction.py - Interaction audit
audit_log.py
Policy and records
- Mock HR policies
mock_hr_docs.py - Pending approvals
pending_approvals.jsonl - Audit records
audit_log.jsonl
Other components
- Questioner
- Human reviewer
Connections 14
- Questioner → Answer pipeline v5: asks
- Answer pipeline v5 → Refusal router: checks refusal
- Answer pipeline v5 → Exception approval: checks exception
- Answer pipeline v5 → Retrieval pipeline v3: runs normal path
- Answer pipeline v5 → PII redaction: redacts answer
- Answer pipeline v5 → Interaction audit: logs interaction
- Retrieval pipeline v3 → Refusal router: checks refusal
- Retrieval pipeline v3 → TF-IDF retrieval: retrieves policy
- TF-IDF retrieval → Token stemming: stems tokens
- TF-IDF retrieval → Mock HR policies: ranks documents
- Exception approval → Pending approvals: files request
- Human reviewer → Pending approvals: reviews requests
- Interaction audit → Audit records: appends records
- Answer pipeline v5 → Questioner: returns response