How roy-alpha/call-me-maybe works
1. What this repo is
- Call-Me-Maybe is a small, local function-calling pipeline for Python 3.10+.
- Its main goal is to map a natural-language prompt to:
- one function name from a provided catalog, and
- a typed argument payload for that function.
- The architecture is built around constrained decoding rather than free-form text generation.
2. Top-level execution path
- The runtime entry point is src/__main__.py, which routes into the CLI layer in src/core/cli.py.
- The CLI parses paths for:
- functions_definition.json
- input prompts/test cases
- output file location
- The CLI then delegates the actual work to the orchestration layer in src/core/engine.py.
3. Input and validation boundary
- src/core/validators.py is the schema gate for the repo.
- It validates the JSON structures for function definitions and prompt/test inputs before any model call happens.
- This boundary is important because the rest of the system assumes the function catalog, parameter types, and prompt format are structurally correct.
4. Core orchestration layer
- src/core/engine.py is the main coordinator.
- It likely performs the full sequence:
- load and validate inputs,
- load the model/tokenizer abstraction,
- build function-name constraints,
- run function selection,
- run parameter extraction,
- write the final structured result to disk.
- This file is the best place to anchor an architecture diagram’s “application logic” box.
5. Model boundary
- The llm_sdk/ package is the abstraction boundary for the language model runtime.
- Based on the README, the system is designed to run against local Hugging Face models and use the model’s tokenizer/logits directly.
- This package is where the repo separates “model access” from “decoding logic,” which keeps the core engine model-agnostic.
6. Function-selection subsystem
- The repository’s distinguishing feature is the function-name selector.
- Function names from functions_definition.json are tokenized and inserted into a Trie in src/core/Trie.py.
- At generation time, the model is not allowed to emit arbitrary next tokens: only tokens that continue a valid function-name path in the Trie are permitted.
7. Constrained decoding flow
- The system builds a function-selection prompt from prompts/func_prompt.txt.
- That prompt includes the user request and the available function list.
- The engine then performs greedy decoding over model logits, but only among tokens allowed by the current Trie node.
- Once a terminal Trie node is reached, the decoded token sequence resolves to the selected function name.
8. Argument extraction subsystem
- After function selection, the engine creates a second prompt describing the chosen function and its parameter schema.
- Argument generation is type-aware:
- numbers get restricted token sets,
- booleans are constrained to true/false,
- strings are decoded until a delimiter is hit.
- This stage is separate from function selection because the constraints and stopping conditions differ by type.
9. Prompt assets and separation of concerns
- prompts/func_prompt.txt contains the reusable instruction text for function selection.
- Keeping prompts external to code makes the system easier to tune without changing orchestration logic.
- The repo cleanly separates:
- prompt templates,
- validation,
- constrained decoding,
- and I/O.
10. Data flow through the repository
- Input data lives under data/input/, including:
- functions_definition.json
- function_calling_tests.json
- The engine consumes those files, produces structured predictions, and writes output JSON to a target path such as data/output/function_calls.json.
- The output format is a compact record per prompt: original prompt, chosen function name, and parsed parameters.
11. Important architectural dependency
- The system depends on tokenization alignment between:
- the selected model’s tokenizer,
- the Trie token sequences,
- and the constrained argument decoders.
- This is a key architectural boundary: if the tokenizer changes, the constraint logic must still match the same token IDs.
12. Repository-level tooling
- Makefile, pyproject.toml, and uv.lock show the project is packaged for reproducible local execution.
- make install, make run, make debug, and make lint are the main developer workflows.
- This is an offline/local inference project rather than a service with networked runtime dependencies.
13. What an architecture diagram should emphasize
- Draw the system as a pipeline with three major zones:
- CLI / file I/O
- Core decoding engine
- Local model runtime
- Inside the engine, show two sequential phases:
- function-name selection via Trie-constrained decoding,
- argument extraction via type-constrained generation.
- Put validators.py as a gate before the engine, and prompts/ and data/input/ as external assets feeding it.
Main entry points: engine.py Engine · validators.py Validators · cli.py CLI · Trie.py Trie
Components 16
CLI / I/O
Command entry and file boundaries
Core engine
Validation, orchestration, and constrained decoding
- Validators · Checks function and prompt input structure
src/core/validators.py - Engine · Coordinates loading, decoding, and output
src/core/engine.py - Trie · Stores tokenized function-name paths
src/core/Trie.py - Func select · Chooses one function under token constraints
- Arg extract · Generates typed arguments for the selected function
- Constrained decode · Filters next tokens by allowed sets and stop rules
- Prompt build · Assembles selection and argument prompts
Local model
Tokenizer and logits boundary for local inference
- LLM SDK · Abstracts local model and tokenizer access
llm_sdk/__init__.py - Tokenizer · Shared tokenization contract for trie and decoders
- Logits · Provides next-token scores for greedy decoding
Assets
Prompt and input data files
- Function prompt · Reusable instruction text for function selection
prompts/func_prompt.txt - Function catalog · Function names and parameter schemas
data/input/functions_definition.json - Test prompts · Prompt cases used by the pipeline
data/input/function_calling_tests.json
Connections 21
- __main__ → CLI: start · CLI entrypoint hands off to command parsing
- CLI → Validators: validate · Checks input files and JSON shape before execution
- CLI → Engine: run · Passes parsed paths and options into the coordinator
- Function catalog → Validators: schema · Function catalog is validated before use
- Test prompts → Validators: schema · Prompt/test input is validated before use
- Validators → Engine: clean inputs · Validated structures feed the orchestration layer
- Engine → Prompt build: assemble · Builds prompts for selection and argument generation
- Function prompt → Prompt build: template · Provides reusable instruction text
- Function catalog → Trie: tokenize · Function names are converted into token paths
- Trie → Func select: allow tokens · Constrains which function-name tokens can be emitted
- Prompt build → Func select: selection prompt · Feeds the function-choice decoding stage
- Func select → Constrained decode: decode · Greedy decoding runs under trie constraints
- Constrained decode → LLM SDK: scores · Uses model logits for next-token choice
- LLM SDK → Tokenizer: token ids · Tokenizer alignment is required for constraints
- Tokenizer → Trie: align · Trie paths and token IDs must match
- Func select → Arg extract: selected function · Chosen function becomes the argument-generation target
- Prompt build → Arg extract: arg prompt · Builds a second prompt from the chosen function schema
- Arg extract → Constrained decode: type constraints · Applies number, boolean, and string stopping rules
- Constrained decode → Logits: score · Reads model scores during argument generation
- Engine → Result write: emit · Writes structured predictions to the target output
- Result write → __main__: done · Completion returns control to the top level