How adam-s-tech/synthid-text works
Project type and purpose
• SynthID Text is a researcher-focused Python library/reference implementation for watermarking and detecting watermarks in large-language-model text.
• It is not a standalone service but a pip-installable library with example Jupyter notebooks, integrating with Hugging Face Transformers models (Gemma & GPT-2) and providing two detectors (Weighted-Mean and Bayesian).
File structure
Top-level:
• .github/workflows/ci.yaml – CI for linting/tests
• data/ – released human evaluation JSONL
• notebooks/ – two Colab/Jupyter notebooks demonstrating model integration & testing
• src/synthid_text/ – core library modules
– detector_mean.py, detector_bayesian.py – scoring functions
– synthid_mixin.py – mix-in to inject watermarking into Transformers models
– logits_processing.py – compute G values, masks, EOS logic
– hashing_function.py, g_value_expectations.py – lower-level support
– torch_testing.py, *_test.py – unit tests
• pyproject.toml – build/config
README for additional insights
• Describes two workflows: (a) watermark‐injection via SynthID mix-in subclass of Gemma/GPT-2; (b) detection via Weighted-Mean (no train) or Bayesian (requires JAX training).
• Explains use of external services: Hugging Face model hub, TensorFlow Datasets for human prompts, Colab for notebooks.
1. User Interface layer (Notebook or local Python script)
2. Library layer (synthid_text package)
• Watermark Mixin module
• Hashing & G-value module
• Logits Processing module
• Detector modules (Mean, Bayesian)
• Detector Training (Bayesian optimizer)
3. External ML Models (Hugging Face Transformers: Gemma & GPT-2)
4. Data layer
• Input prompts (user or TFDS)
• Human evaluation data (data/human_eval.jsonl)
5. External services
• Hugging Face model hub (for from_pretrained)
• PyPI (for pip install)
• Colab runtime / Jupyter server
1. Notebook → Library: calls mix-in’s .from_pretrained(), .generate(), .detect() APIs
2. Library → Transformers Model: mix-in intercepts forward/generate to insert watermarking (calls hashing_function + logits_processing)
3. Transformers Model → Library: returns logits → LogitsProcessing computes G-values & masks → Detector modules consume these to score
4. Detector Training: uses G-values + mask + labels (from data loader) → JAX optimizer → produces Bayesian detector parameters
5. Data flow: TFDS → prompts → library → models → watermarked text → detection → evaluation
6. CI Pipeline (ci.yaml) → unit tests → ensures correctness
c. Architectural patterns & design principles
• Modular Python package: clear separation of concerns (watermarking vs. detection vs. training)
• Extension via mix-in pattern to minimally modify HF model classes
• Functional separation: hashing → g-value computation → scoring → training
• Dependency injection: configuration TypedDict for watermark parameters
• Test-driven: each module has corresponding tests
d. Key technologies, frameworks, libraries
• Python 3, pyproject.toml (Poetry/PEP 621)
• PyTorch (model inference & mix-in)
• Hugging Face Transformers (Gemma, GPT-2 classes)
• JAX (& jax.numpy) for Bayesian detector training
• TensorFlow Datasets for prompt retrieval
• Jupyter/Colab for notebooks
• pytest for tests
Detailed capture of essential elements
• Modules in layers:
Presentation (Notebook) → Library API → Model Extension → Processing & Detection → Data & Training
• Don’t overthink: break into as many logical components as possible
• Test/CI pipeline as a side branch verifying each module