How cverluise/patcit works
Project type and purpose
• patCit is a data-processing/ETL pipeline and library (not a client/server web app). It ingests patent and front-page citation corpora, applies NLP models (classification, extraction, enrichment), versions data/models via DVC, and exposes the results on BigQuery.
File structure
• Top-level directories:
• data, models: versioned via DVC for raw/enriched datasets and trained spaCy models
• patcit: the core Python package subdivided into subpackages:
– bq: BigQuery ingestion/export
– brew: brewing data recipes
– data: raw data download/processing
– grobid: calls to GROBID for parsing citations
– model: spaCy pipeline configuration (classification, fine-tuning, evaluation)
– serialize: convert pipeline outputs into JSON/CSV, validate via JSON Schemas
– utils: shared utilities and CLI entrypoints
• schema: JSON Schema definitions for serialized outputs
• notebook: example Colab notebooks for exploration
• docs: MkDocs-based user/developer documentation
• Configuration files: pyproject.toml, setup.py, Makefile, .pre-commit-config.yaml, mkdocs.yml, scripts for TLS preparation
• Deployment: no web server; uses Google Cloud (BigQuery) and external services (GROBID, biblio-glutton, Google APIs)
Extract architecture hints from README
• Data sources: DOCDB (front-page), USPTO full-text patents via Google Patents OCR
• NLP stack: spaCy, GROBID, biblio-glutton, Google Match API
• Storage: BigQuery tables for front-page & in-text citations
• Versioning: DVC handles large data and models
• Outputs: BigQuery, Zenodo, downloadable artifacts