How geek007git/hogyoku-oss works
Hogyoku is a self-hosted, multimodal RAG workspace: users upload private documents, ask questions in managed threads, and receive cited answers whose claims are independently checked before presentation. The documented lifecycle and sampled routes show upload-to-background-ingestion and ask-to-retrieval-to-verification. The graph also preserves the documented browser workspace and offline Python guardrails. Several capabilities, including parsing/OCR internals and answer verification, are documented but not excerpted; their subsystem paths are therefore exact enclosing directories, and uncertain implementation links are omitted.
Main entry points: ingestion.ts Document ingestion · threads.ts Thread routes · app.ts Fastify API · ai.ts Model and embeddings
Components 26
Workspace and API
- Browser workspace
public/app.js - Fastify API
src/app.ts - Accounts and sessions
src/http/auth.ts - Document routes
src/http/documents.ts - Thread routes
src/http/threads.ts - Web server
src/server.ts
Document ingestion
- Ingestion queue
src/lib/queue.ts - Ingestion worker
src/worker.ts - Document ingestion
src/services/ingestion.ts - Extraction and OCR
src/services - Page-aware chunking
src/lib/chunking.ts
Conversational RAG
- Query rewriting
src/services/conversation.ts - Hybrid retrieval
src/services/retrieval.ts - Grounded verification
src/services/answers.ts
Persistence and providers
- Model and embeddings
src/services/ai.ts - PostgreSQL and pgvector
src/db/client.ts - Private object storage
src/lib/storage.ts - Redis
- Google Gemini
Offline guardrails
- Guardrail utilities
python/hogyoku_guardrails/common.py - Citation audit
python/hogyoku_guardrails/citation_audit.py - Chunk audit
python/hogyoku_guardrails/chunk_audit.py - RAG lint
python/hogyoku_guardrails/rag_lint.py - Security scanning
python/hogyoku_guardrails/security_scan.py - Security report
python/hogyoku_guardrails/security_report.py
Other components
- Workspace user
Connections 33
- Workspace user → Browser workspace: uses
README.md - Browser workspace → Fastify API: calls API
- Web server → Fastify API: starts
src/server.ts - Fastify API → Accounts and sessions: registers routes
src/app.ts - Fastify API → Document routes: registers routes
src/app.ts - Fastify API → Thread routes: registers routes
src/app.ts - Document routes → Private object storage: uploads files
src/http/documents.ts - Document routes → Ingestion queue: enqueues jobs
src/http/documents.ts - Web server → Private object storage: ensures bucket
src/server.ts - Ingestion worker → Ingestion queue: consumes jobs
src/worker.ts - Ingestion worker → Document ingestion: invokes
src/worker.ts - Document ingestion → PostgreSQL and pgvector: reads and writes
src/services/ingestion.ts - Document ingestion → Private object storage: downloads object
src/services/ingestion.ts - Document ingestion → Extraction and OCR: extracts pages
src/services/ingestion.ts - Document ingestion → Page-aware chunking: chunks pages
src/services/ingestion.ts - Document ingestion → Model and embeddings: embeds chunks
src/services/ingestion.ts - Thread routes → Query rewriting: rewrites query
src/http/threads.ts - Thread routes → Hybrid retrieval: retrieves evidence
src/http/threads.ts - Thread routes → Grounded verification: requests answer
src/http/threads.ts - Thread routes → PostgreSQL and pgvector: reads and saves turns
src/http/threads.ts - Hybrid retrieval → Model and embeddings: embeds query
src/services/retrieval.ts - Hybrid retrieval → PostgreSQL and pgvector: searches indexed chunks
src/services/retrieval.ts - Grounded verification → Hybrid retrieval: uses evidence
src/services/answers.ts - Grounded verification → Query rewriting: uses history
src/services/answers.ts - Grounded verification → Model and embeddings: generates and checks
src/services/answers.ts - Model and embeddings → Google Gemini: calls when configured
src/services/ai.ts - Model and embeddings → Google Gemini: embeds remotely
src/services/ai.ts - Citation audit → Guardrail utilities: uses helpers
python/hogyoku_guardrails/citation_audit.py - Chunk audit → Guardrail utilities: uses helpers
python/hogyoku_guardrails/chunk_audit.py - RAG lint → Guardrail utilities: uses helpers
python/hogyoku_guardrails/rag_lint.py - Security scanning → Guardrail utilities: uses helpers
python/hogyoku_guardrails/security_scan.py - Security report → Guardrail utilities: uses helpers
python/hogyoku_guardrails/security_report.py - Ingestion queue → Redis: stores jobs