How flipkart-incubator/krystal works
Project Type and Purpose
• The Krystal Project is a multi-module framework aimed at super-charging the development and execution of both synchronous (using “Vajram” and “Krystex”) and asynchronous (using “Honeycomb”) business logic workflows.
• Its purpose is to provide a developer-friendly, high-performance, and observable runtime environment. It encapsulates developer SDKs, code generation (annotation processors), an execution engine for DAG-driven business logic (Krystex), an asynchronous orchestrator (Honeycomb), and visualization modules to aid in debugging and monitoring.
File Structure
• The file tree shows a collection of modules (each with its own build.gradle), such as:
– krystal-bom: The single Bill-of-Materials managing versioning for all Krystal artefacts.
– krystal-common: Contains common data models, utilities, and shared annotations.
– vajram (and its variants like vajram-codegen, vajram-java-sdk, and vajram-krystex): These modules provide the developer API for writing business logic (vajrams), a code generation plugin/annotation processor for automatic model creation, and translators (transpilers) to convert vajramDef models into runtime types the execution engine (krystex) understands.
– krystex: The synchronous execution engine that constructs a logical Directed Acyclic Graph (DAG) of dependency calls and executes them concurrently.
– honeycomb: An asynchronous orchestrator that handles non-blocking workflows.
– krystal-visualization: Provides tools for generating call graphs and visual debugging of execution flows.
– Extensions (e.g., vajram-guice): Adding dependency injection capabilities to support injection of facet values via Guice.
– Additional support modules such as lattice, mojo-publish-plugin, oss-java-deps-bom, and even language tooling modules (vajram-lang) contribute to the broader ecosystem.
• The presence of module-info.java files and multiple gradle configurations indicates use of Java’s modular system and a complex build pipeline.
Use of the README
• The README confirms the central components (Vajram, Krystex, Honeycomb) and the design goals (separation of concerns, minimal glue code, zero lateral dependency impact, observability, and optimal execution).
• Specific sections detail how the regeneration of developer models is integrated into the build process (codegen module plugging into the “compileJava” task) and how the different modules interact in both compile-time (annotation processing) and runtime phases.
Guidance
1. Developer API & Code Generation:
– VajramDef Java SDK (module: vajram-java-sdk) where developers write their business logic.
– Vajram-Guice extension (module: vajram-guice) that offers dependency injection support.
2. Core Runtime Engines:
– Krystex (execution engine) that takes the generated DAG of interdependent logic and executes synchronous workflows concurrently.
– Honeycomb (asynchronous orchestrator) which manages the asynchronous portions of the workflow.
3. Shared and Common Layers:
– Krystal Common (module: krystal-common) which supplies shared data models, annotations, utilities, and configuration.
– BOM (krystal-bom) to ensure consistent versioning across modules.
4. Support and Auxiliary Modules:
– Krystal Visualization (module: krystal-visualization) for generating DAG visualizations and call graphs.
– Additional modules like lattice, mojo-publish-plugin, and oss-java-deps-bom which support build-, publication-, and dependency-management concerns.
5. Language Infrastructure:
– Vajram-lang: Contains grammar definitions (ANTLR) for parsing or linting vajram files.
– Vajram-samples: Example usage demonstrating how the modules come together.
– Show how Krystal Common underpins both the developer SDK and runtime engines (shared utility line).
– Illustrate that Honeycomb acts as a parallel engine managing asynchronous workflows while the synchronous branch (Krystex) processes the generated DAG.
– Highlight integration connectors like the Guice extension feeding into the dependency injection required during execution.
– Include connectors between the production runtime and the Visualization module, which might consume runtime metrics or DAG data.
• Mention any additional build-time configurations or external technologies:
– Gradle tasks (like code generation tasks integrated with compileJava) and standardized configuration files (found under the config/ directory) should be noted as supporting infrastructure.
– Use labels to show that separate “modules” form independent artifacts but interact closely through common interfaces defined in krystal-common.