How kiwi0719/jev-edge works
jev-edge is typed-judgment admission control for LLM traffic: gateway integrations inspect requests before they reach protected services, applying cheap rules before optional model judgment. The principal flow is request decoding and L1 filtering, cached or provider-backed L2 scoring, then policy enforcement or annotated pass-through; ambiguous cases feed a non-blocking L3 path. The repository also provides JavaScript runtime integrations, multiple judge providers, operational state/configuration endpoints, and gateway protocol adapters. README documents these capabilities; sampled source confirms parts of the JS runtime and adapter behavior, while several Lua and adapter links rely on documented contracts.
Main entry points: init.lua Core evaluator · edge.lua Authz endpoint · runtime.ts JS shared runtime · judge.lua L2 judge prompts
Components 28
Gateway runtime
- OpenResty access
adapters/openresty/lib/resty/jev/edge.lua - Authz endpoint
adapters/openresty/lib/resty/jev/edge.lua - Body decoding
adapters/openresty/lib/resty/jev/body.lua - JS shared runtime
adapters/js/src/runtime.ts - Verdict response
adapters/openresty/lib/resty/jev/edge.lua
Judgment pipeline
- L1 rules
core/rules.lua - Core evaluator
core/init.lua - Text normalization
core/normalize.lua - L2 judge prompts
core/judge.lua - Policy verdict
core/policy.lua - Prompt templates
core/templates - Async L3 path
adapters/openresty/lib/resty/jev/async.lua - JS core evaluator
adapters/js/src/core/index.ts
State and operations
- Verdict cache
adapters/openresty/lib/resty/jev/cache.lua - Subject reputation
core/subject.lua - Admin configuration
adapters/openresty/lib/resty/jev/config.lua - JS runtime stores
adapters/js/src/cf/stores.ts
Runtime integrations
- JS host adapters
adapters/js/src/frameworks.ts - Judge providers
adapters/openresty/lib/resty/jev/providers - Laya judge
adapters/laya-server/laya_server.py - Gateway adapters
adapters - Envoy gRPC shim
adapters/envoy/grpc-shim/main.go - HAProxy SPOE agent
adapters/haproxy/spoa/main.go - LiteLLM guardrail
adapters/litellm/jev_edge_guardrail.py - JS judge providers
adapters/js/src/providers/index.ts
Other components
- Incoming client
- TypeSafe Jev
- Protected backend
Connections 31
- Incoming client → OpenResty access: sends request
- OpenResty access → Body decoding: decodes body
- OpenResty access → Core evaluator: evaluates request
- Core evaluator → L1 rules: checks rules
- Core evaluator → Text normalization: normalizes text
- Core evaluator → Verdict cache: reads and writes
- Core evaluator → L2 judge prompts: builds judgment
- L2 judge prompts → Prompt templates: uses templates
- L2 judge prompts → Judge providers: requests scores
- Judge providers → TypeSafe Jev: calls API
- Judge providers → Laya judge: calls protocol
- Core evaluator → Policy verdict: applies policy
- Policy verdict → Verdict response: produces verdict
- Verdict response → Protected backend: passes with headers
- Verdict response → Incoming client: blocks request
- Core evaluator → Async L3 path: defers ambiguous case
- Async L3 path → Subject reputation: updates reputation
- Admin configuration → OpenResty access: updates live config
- Incoming client → JS host adapters: sends request
- JS host adapters → JS shared runtime: adapts request
- JS shared runtime → JS core evaluator: evaluates request
- JS shared runtime → JS runtime stores: selects stores
- JS core evaluator → JS judge providers: requests scores
- JS judge providers → TypeSafe Jev: calls API
- JS shared runtime → Protected backend: passes or blocks
- Incoming client → Gateway adapters: enters gateway
- Gateway adapters → Authz endpoint: forwards authz check
- Envoy gRPC shim → Authz endpoint: forwards HTTP check
- HAProxy SPOE agent → Authz endpoint: requests verdict
- LiteLLM guardrail → Authz endpoint: requests verdict
- Authz endpoint → Core evaluator: evaluates request