How apoorvgarg-creator/go-logql works
1. Repository purpose and boundary
- This is a pure Go LogQL query-construction library, not a Loki client or server. The root package only builds LogQL strings; networking, execution, and auth are deliberately out of scope.
- The architectural boundary is clear: the library produces query text, while callers supply their own HTTP/client layer if they want to send queries to Grafana Loki.
2. Top-level package structure
- The core implementation lives in a small set of root-package files: logql.go, metric.go, expr.go, and label.go.
- That split maps directly to the three main query domains in LogQL:
- log queries
- metric/range queries
- binary expressions
- label.go acts as a shared utility layer for label names, values, quoting, and formatting concerns.
3. Immutable builder model
- The central design pattern is a fluent immutable builder.
- Every method returns a new value rather than mutating the existing one, which makes query objects safe to reuse across goroutines and across derived queries.
- This is the main internal data-flow pattern: base query object → chained transformations → final serialized string.
4. Log query subsystem
- NewLogQuery() creates the root log-query builder.
- logql.go represents the stream selector and pipeline stages used to form LogQL log queries:
- stream selectors such as Eq, Neq, Re, Nre
- line filters such as LineContains, LineMatch, etc.
- parsers such as JSON, Logfmt, Regexp, Pattern, Unpack
- label filters and pipeline formatting stages such as LineFormat, LabelFormatEntry, Drop, Keep, Decolorize
- extraction stage Unwrap
- The output of this subsystem is a complete LogQL log query string like {job="api"} |= "error" or {...} | json | unwrap latency_ms.
5. Metric query subsystem
- metric.go wraps a log query in a range vector / metric expression.
- It provides constructors for Loki range aggregations such as Rate, CountOverTime, BytesRate, SumOverTime, QuantileOverTime, and others.
- It also supports metric operators and presentation controls:
- aggregations: Sum, Avg, Min, Max, Count, Stddev, Stdvar
- selectors: By, Without
- modifiers: Offset
- ranking/sorting: TopK, BottomK, Sort, SortDesc
- Architecturally, this layer takes a log query as input and produces metric-form LogQL that can be graphed in Loki/Grafana.
6. Binary expression subsystem
- expr.go composes metric expressions using arithmetic, comparison, and logical operators.
- It supports Loki-style expression combination such as Add, Sub, Mul, Div, CmpGt, And, Or, Unless, plus boolean modifiers via Bool().
- This layer is what lets callers build formulas like error-rate percentages or threshold alerts by combining two metric queries.
7. Shared validation and rendering
- Validation is part of the build step rather than construction:
- at least one stream selector is required
- label names must be non-empty
- regexes must compile
- durations must be positive
- quantiles must be within [0, 1]
- topk/bottomk require k > 0
- Build() returns (string, error) for safe use in application code.
- String() is a convenience layer that panics on invalid input, so it is intended only for already-validated queries.
8. Data flow through the library
- The typical flow is:
1. create a base log query
2. chain selector/filter/parser stages
3. optionally wrap it in a metric/range query
4. optionally combine metrics with expressions
5. serialize to a LogQL string
- This one-way transformation is the core architectural pipeline of the package.
9. Testing strategy
- The repository contains focused unit tests beside each core subsystem: label_test.go, logql_test.go, metric_test.go, expr_test.go, plus example_test.go.
- That mirrors the code organization and suggests the important contract is string correctness and validation behavior rather than runtime integration.
10. Examples and integration boundary
- The examples/ directory is a separate Go module that demonstrates end-to-end use against a live Loki stack.
- It is not part of the library runtime; it is an integration sandbox showing how the generated queries are consumed externally.
11. Example infrastructure
- examples/docker-compose.yml brings up three external services:
- Loki as the log backend
- Grafana as the UI
- log-simulator as a log producer
- examples/grafana/provisioning/datasources/loki.yml auto-configures the Loki datasource in Grafana.
- examples/loki-config.yml provides the Loki server config.
- examples/log-simulator.sh continuously emits realistic JSON logs so the sample queries have live data.
12. How the example app fits
- examples/main.go demonstrates programmatic query construction and execution against local Loki.
- The example also highlights the key separation in this repo: the library builds queries, while the example app provides the missing client/execution layer.
13. External technology footprint
- The library itself depends only on the Go standard library.
- The architecture only becomes distributed in the examples/ sandbox, where Docker Compose, Loki, and Grafana form the runtime environment.
14. Diagramming takeaway
- For an architecture diagram, the core repo should be drawn as a single pure-Go query builder library with three internal subsystems:
- log query builder
- metric query builder
- expression builder
- Surround that with shared validation/formatting utilities and a separate examples/integration environment that connects generated LogQL strings to Loki/Grafana.
Main entry points: docker-compose.yml Stack setup · README.md API surface · metric.go Metric queries · logql.go Log queries
Components 10
Core library
Pure-Go LogQL query construction and rendering.
- API surface · Root package entry points for constructing and rendering LogQL queries.
README.md - Log queries · Builds LogQL stream selectors, line filters, parsers, and pipeline stages.
logql.go - Metric queries · Wraps log queries in range vectors and metric aggregations, selectors, offsets, and ranking.
metric.go - Binary expr · Combines metric expressions with arithmetic, comparison, and logical operators.
expr.go - Labels · Shared label-name, quoting, and formatting logic used across query builders.
label.go
Examples sandbox
Separate integration module that demonstrates generated queries against a live Loki stack.
- Example app · Programmatic query construction and execution against a local Loki instance.
examples/main.go - Stack setup · Runs the local Loki, Grafana, and log-simulator services for the example environment.
examples/docker-compose.yml - Loki · External log store and query backend used by the examples.
examples/loki-config.yml - Grafana · Visualization layer that reads from Loki via the provisioned datasource.
examples/grafana/provisioning/datasources/loki.yml - Log simulator · Shell-based generator that emits sample JSON logs for the demo stack.
examples/log-simulator.sh
Connections 14
- API surface → Log queries: creates · Root construction begins with log query builders.
- API surface → Metric queries: creates · Root construction also exposes metric query builders.
- API surface → Binary expr: creates · Root construction exposes expression composition.
- Log queries → Labels: uses · Stream selectors and pipeline stages rely on shared label formatting.
- Metric queries → Log queries: wraps · Metric queries take log queries as their input source.
- Metric queries → Binary expr: composes · Metric expressions can be combined into larger formulas.
- Binary expr → Metric queries: combines · Binary expressions operate on metric queries or metric subexpressions.
- Example app → API surface: uses · The example app builds queries with the library API.
- Example app → Stack setup: runs against · The example app targets the local integration stack.
- Stack setup → Loki: starts · Docker Compose brings up Loki as the backend.
- Stack setup → Grafana: starts · Docker Compose brings up Grafana as the UI.
- Stack setup → Log simulator: starts · Docker Compose brings up the sample log producer.
- Grafana → Loki: queries · Grafana reads logs and metrics from Loki through the configured datasource.
- Log simulator → Loki: feeds · Generated logs are ingested by Loki for the demo.