# sudobhavik/layarustparser architecture

> Architecture diagram of the GitHub repository sudobhavik/layarustparser, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sudobhavik/layarustparser
- Repository: https://github.com/sudobhavik/layarustparser

## Overview

ULPF receives firewall/syslog lines, classifies and parses them into lossless OCSF NetworkActivity events, then batches and persists evidence in Parquet with an RFC 6962 Merkle ledger for verification. Its CLI also exposes onboarding, evaluation, and serving workflows; the AI crate supplies the tiered parser and template-learning components. The README documents these capabilities, while sampled source confirms core parsing, socket ingestion, and parts of persistence and CLI behavior; some subsystem wiring is therefore left unspecified.

Main entry points: [`crates/ulpf-cli/src/main.rs`](https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-cli/src/main.rs) (ULPF CLI), [`crates/ulpf-cli/src/serve/handlers.rs`](https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-cli/src/serve/handlers.rs) (Serve Handlers), [`crates/ulpf-ai/src/pipeline.rs`](https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-ai/src/pipeline.rs) (Three-Tier Pipeline), [`crates/ulpf-core/src/parser/mod.rs`](https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-core/src/parser/mod.rs) (Universal Parser)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Ingest and Parse"]
  node_socket["Socket Ingest<br/>[socket.rs]"]
  node_universal["Universal Parser<br/>[mod.rs]"]
  node_classifier["Format Classifier<br/>[classifier.rs]"]
  node_extractors["Vendor Extractors<br/>[mod.rs]"]
  node_ocsf["OCSF Schema<br/>[ocsf.rs]"]
end

subgraph group_ai["AI Pipeline"]
  node_tiers["Three-Tier Pipeline<br/>[pipeline.rs]"]
  node_drain["Drain Miner<br/>[drain.rs]"]
  node_laya["Laya Triage<br/>[laya.rs]"]
  node_onboarder["Parser Onboarding<br/>[onboarder.rs]"]
  node_evaluation["Engine Evaluation<br/>[evaluator.rs]"]
  node_duel["Engine Duel<br/>[duel.rs]"]
end

subgraph group_integrity["Evidence Integrity"]
  node_batcher["Block Batcher<br/>[batcher.rs]"]
  node_merkle["Merkle Ledger<br/>[merkle.rs]"]
  node_storage[("Parquet Storage<br/>[storage.rs]")]
  node_tamper["Tamper Checks<br/>[tamper.rs]"]
end

subgraph group_operations["CLI and Operations"]
  node_cli["ULPF CLI<br/>[main.rs]"]
  node_serve["Serve Handlers<br/>[handlers.rs]"]
  node_state["Service State<br/>[state.rs]"]
end

node_syslog(("Syslog Sources"))
node_user(("Operator"))
node_parquetfiles[("Parquet Files")]
node_ledgerfile["Ledger File"]

node_syslog -->|"sends logs"| node_socket
node_cli -->|"starts ingest"| node_socket
node_socket -->|"parses batches"| node_universal
node_universal -->|"classifies format"| node_classifier
node_universal -->|"dispatches parse"| node_extractors
node_extractors -->|"builds events"| node_ocsf
node_cli -->|"processes lines"| node_tiers
node_tiers -->|"parses known logs"| node_universal
node_tiers -->|"mines templates"| node_drain
node_tiers -->|"triages samples"| node_laya
node_cli -->|"batches events"| node_batcher
node_batcher -->|"roots blocks"| node_merkle
node_batcher -->|"writes blocks"| node_storage
node_merkle -->|"records roots"| node_ledgerfile
node_storage -->|"writes Parquet"| node_parquetfiles
node_cli -->|"verifies roots"| node_merkle
node_cli -->|"runs verification"| node_tamper
node_user -->|"runs commands"| node_cli
node_user -->|"uses service"| node_serve
node_serve -->|"reads state"| node_state
node_serve -->|"checks evidence"| node_tamper
node_serve -->|"validates samples"| node_onboarder
node_serve -->|"reads records"| node_storage
node_cli -->|"onboards formats"| node_onboarder
node_cli -->|"evaluates engines"| node_evaluation
node_evaluation -->|"scores pipeline"| node_tiers
node_cli -->|"compares engines"| node_duel

click node_cli "https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-cli/src/main.rs"
click node_socket "https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-core/src/ingest/socket.rs"
click node_universal "https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-core/src/parser/mod.rs"
click node_classifier "https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-core/src/parser/classifier.rs"
click node_extractors "https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-core/src/parser/extractors/mod.rs"
click node_ocsf "https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-core/src/schema/ocsf.rs"
click node_tiers "https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-ai/src/pipeline.rs"
click node_drain "https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-ai/src/drain.rs"
click node_laya "https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-ai/src/laya.rs"
click node_onboarder "https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-ai/src/onboarder.rs"
click node_batcher "https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-integrity/src/batcher.rs"
click node_merkle "https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-integrity/src/merkle.rs"
click node_storage "https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-integrity/src/storage.rs"
click node_tamper "https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-integrity/src/tamper.rs"
click node_evaluation "https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-ai/src/evaluator.rs"
click node_duel "https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-ai/src/duel.rs"
click node_serve "https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-cli/src/serve/handlers.rs"
click node_state "https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-cli/src/serve/state.rs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_socket,node_universal,node_classifier,node_extractors,node_ocsf toneBlue
class node_tiers,node_drain,node_laya,node_onboarder,node_evaluation,node_duel,node_parquetfiles toneAmber
class node_batcher,node_merkle,node_storage,node_tamper toneMint
class node_cli,node_serve,node_state toneRose
class node_syslog,node_user,node_ledgerfile toneIndigo
```

## Components

### Ingest and Parse

- [Socket Ingest](https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-core/src/ingest/socket.rs): `crates/ulpf-core/src/ingest/socket.rs`
- [Universal Parser](https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-core/src/parser/mod.rs): `crates/ulpf-core/src/parser/mod.rs`
- [Format Classifier](https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-core/src/parser/classifier.rs): `crates/ulpf-core/src/parser/classifier.rs`
- [Vendor Extractors](https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-core/src/parser/extractors/mod.rs): `crates/ulpf-core/src/parser/extractors/mod.rs`
- [OCSF Schema](https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-core/src/schema/ocsf.rs): `crates/ulpf-core/src/schema/ocsf.rs`

### AI Pipeline

- [Three-Tier Pipeline](https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-ai/src/pipeline.rs): `crates/ulpf-ai/src/pipeline.rs`
- [Drain Miner](https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-ai/src/drain.rs): `crates/ulpf-ai/src/drain.rs`
- [Laya Triage](https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-ai/src/laya.rs): `crates/ulpf-ai/src/laya.rs`
- [Parser Onboarding](https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-ai/src/onboarder.rs): `crates/ulpf-ai/src/onboarder.rs`
- [Engine Evaluation](https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-ai/src/evaluator.rs): `crates/ulpf-ai/src/evaluator.rs`
- [Engine Duel](https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-ai/src/duel.rs): `crates/ulpf-ai/src/duel.rs`

### Evidence Integrity

- [Block Batcher](https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-integrity/src/batcher.rs): `crates/ulpf-integrity/src/batcher.rs`
- [Merkle Ledger](https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-integrity/src/merkle.rs): `crates/ulpf-integrity/src/merkle.rs`
- [Parquet Storage](https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-integrity/src/storage.rs): `crates/ulpf-integrity/src/storage.rs`
- [Tamper Checks](https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-integrity/src/tamper.rs): `crates/ulpf-integrity/src/tamper.rs`

### CLI and Operations

- [ULPF CLI](https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-cli/src/main.rs): `crates/ulpf-cli/src/main.rs`
- [Serve Handlers](https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-cli/src/serve/handlers.rs): `crates/ulpf-cli/src/serve/handlers.rs`
- [Service State](https://github.com/sudobhavik/layarustparser/blob/master/crates/ulpf-cli/src/serve/state.rs): `crates/ulpf-cli/src/serve/state.rs`

### Other components

- **Syslog Sources**
- **Operator**
- **Parquet Files**
- **Ledger File**

## Connections

- Syslog Sources → Socket Ingest: sends logs
- ULPF CLI → Socket Ingest: starts ingest
- Socket Ingest → Universal Parser: parses batches
- Universal Parser → Format Classifier: classifies format
- Universal Parser → Vendor Extractors: dispatches parse
- Vendor Extractors → OCSF Schema: builds events
- ULPF CLI → Three-Tier Pipeline: processes lines
- Three-Tier Pipeline → Universal Parser: parses known logs
- Three-Tier Pipeline → Drain Miner: mines templates
- Three-Tier Pipeline → Laya Triage: triages samples
- ULPF CLI → Block Batcher: batches events
- Block Batcher → Merkle Ledger: roots blocks
- Block Batcher → Parquet Storage: writes blocks
- Merkle Ledger → Ledger File: records roots
- Parquet Storage → Parquet Files: writes Parquet
- ULPF CLI → Merkle Ledger: verifies roots
- ULPF CLI → Tamper Checks: runs verification
- Operator → ULPF CLI: runs commands
- Operator → Serve Handlers: uses service
- Serve Handlers → Service State: reads state
- Serve Handlers → Tamper Checks: checks evidence
- Serve Handlers → Parser Onboarding: validates samples
- Serve Handlers → Parquet Storage: reads records
- ULPF CLI → Parser Onboarding: onboards formats
- ULPF CLI → Engine Evaluation: evaluates engines
- Engine Evaluation → Three-Tier Pipeline: scores pipeline
- ULPF CLI → Engine Duel: compares engines