# intel-spark/corenlp architecture

> Architecture diagram of the GitHub repository intel-spark/corenlp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/intel-spark/corenlp
- Repository: https://github.com/intel-spark/corenlp

## Overview

1. **What this repository is**
   Stanford CoreNLP is a large Java NLP platform organized as a library plus command-line tools, servers, demos, and web apps. The top-level architecture is not a single app, but a set of reusable NLP components wired together through a common annotation pipeline.

2. **Core data model and control flow**
   The central abstraction is the `Annotation` / `CoreMap` object in `src/edu/stanford/nlp/pipeline` and `src/edu/stanford/nlp/util` / `src/edu/stanford/nlp/ling`. Raw text is converted into an `Annotation`, then annotators add structured outputs step by step: tokens, sentences, POS tags, lemmas, NER, parse trees, dependencies, coreference chains, sentiment, time expressions, and extracted relations.

3. **Pipeline orchestration layer**
   `StanfordCoreNLP`, `AnnotationPipeline`, `AnnotatorPool`, and the `Annotator` implementations define the main execution boundary. This layer is responsible for ordering annotators, enforcing requirements, and passing shared document state between modules. It is the primary integration point for users of the library and for server/web entrypoints.

4. **Tokenization, sentence splitting, and text normalization**
   The `process` package and several pipeline annotators handle the early text normalization stages: tokenization (`PTBTokenizer`, language-specific tokenizers), sentence segmentation, XML stripping, truecasing, stemming/morphology, and word-shape normalization. This is where unstructured text becomes a token sequence that downstream components can consume.

5. **Sequence labeling and classification subsystem**
   The `sequences`, `classify`, `ie/crf`, `tagger/maxent`, and `regexp` packages implement the statistical sequence models used for POS tagging, NER, and related labeling tasks. These packages contain both model abstractions and training/inference code, with supporting optimization in `optimization`, `maxent`, and `math`. The repository also includes example datasets and property files for model configuration.

6. **Parsing subsystem**
   Syntactic parsing is split across `parser`, `parser/lexparser`, `parser/shiftreduce`, `parser/nndep`, and `parser/dvparser`. This includes the legacy lexicalized parser, a shift-reduce parser, a neural dependency parser, and evaluation tooling. Parser outputs feed later annotators such as dependency conversion, coreference, and OpenIE.

7. **Coreference subsystem**
   Coreference is represented by multiple generations of code: `dcoref`, `hcoref`, and `scoref`. These packages implement mention detection, sieves/rules, clustering, scoring, and document readers. They depend heavily on earlier pipeline stages, especially tokenization, parsing, NER, and speaker/entity metadata.

8. **Higher-level semantic extraction**
   `naturalli` and the `pipeline` annotators for relation extraction and entity mentions sit above the basic linguistic layers. `OpenIE`/Natural Logic uses parse trees and dependencies to extract relation triples. `ie/machinereading`, `ie/kbp*`, and `patterns` support rule-based and statistical extraction workflows for entities, relations, and pattern mining.

9. **Language-specific processing**
   The `international` and `trees/international` subtrees contain language-specific tokenizers, tree normalizers, head finders, grammars, and pipeline adaptations for Arabic, Chinese, French, German, Hebrew, and Spanish. These are not just data files; they define alternate processing pipelines and treebank conventions that branch from the English defaults.

10. **Tree, dependency, and graph infrastructure**
   `trees`, `semgraph`, `graph`, `fsm`, and `tregex` provide the shared structural representation layer. Constituency trees, typed dependencies, semantic graphs, and graph algorithms are used across parsing, OpenIE, coreference, and tree transformation tools. `tregex`/`tsurgeon` are especially important as pattern/query and tree-editing utilities used throughout the ecosystem.

11. **Temporal and numeric normalization**
   `time` and `ie/qe` implement specialized extraction for dates, times, holidays, and quantifiable entities. These modules behave like focused annotators built on top of tokenization and sentence boundaries, and their outputs become additional annotations on the same document model.

12. **Serving, demos, and UI entrypoints**
   The repo includes multiple deployment surfaces: `pipeline/webapp`, `parser/webapp`, `ie/ner/webapp`, `naturalli/demo`, and `time/suservlet`, plus CLI scripts under `scripts` and docs under `doc`. `StanfordCoreNLPServer`, `StanfordCoreNLPClient`, servlet classes, and demo frontends show that the same core pipeline can be embedded in an HTTP service or run interactively.

13. **Training, configuration, and artifacts**
   `scripts`, `data`, and many `.properties`, `.txt`, `.prop`, and `.conf` files define model training recipes, language configs, and test corpora. The codebase is meant to be paired with external model jars, and the repo also carries generated parser/lexer sources, implying build-time generation from JFlex/JavaCC but source-controlled outputs for convenience.

14. **Build and dependency boundaries**
   The project is primarily Java, built with Ant and also supported by Gradle files. External libraries materially shape the architecture: servlet APIs and Tomcat for web endpoints, Lucene for indexing/pattern search, Protobuf for serialized annotations, Joda-Time/Jollyday for temporal reasoning, EJML for linear algebra, and logging via Log4j/SLF4J/Redwood.

15. **Testing and regression coverage**
   `test`, `itest`, and `test/regression` provide a broad test matrix across nearly every subsystem: tokenization, parsing, NER, coref, semantic graphs, patterns, time, and utilities. The test structure mirrors the production package layout, which is useful for drawing subsystem boundaries in an architecture diagram.

16. **Architectural summary**
   The dominant pattern is a shared-document pipeline with many pluggable annotators layered on common linguistic representations. Most subsystems either produce annotations into that shared document or consume them for higher-level inference. For a diagram, the clearest spine is: input text → normalization/tokenization → sentence/sequence labeling → syntax/dependencies → semantic extraction/coreference/time/sentiment → structured outputs, with language-specific branches and server/web/CLI entrypoints wrapping the same core pipeline.

Main entry points: [`src/edu/stanford/nlp/pipeline/TokenizerAnnotator.java`](https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/pipeline/TokenizerAnnotator.java) (Annotators), [`src/edu/stanford/nlp/pipeline/StanfordCoreNLPServer.java`](https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/pipeline/StanfordCoreNLPServer.java) (Entry points), [`src/edu/stanford/nlp/pipeline/StanfordCoreNLP.java`](https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/pipeline/StanfordCoreNLP.java) (Pipeline core), [`src/edu/stanford/nlp/parser/lexparser/LexicalizedParser.java`](https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/parser/lexparser/LexicalizedParser.java) (Parsing)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_core["Core pipeline"]
  node_annotation["Annotation model<br/>doc state<br/>[Annotation.java]"]
  node_pipeline["Pipeline core<br/>orchestration"]
  node_annotators["Annotators<br/>stage modules"]
  node_serialization["Output formats<br/>serialization"]
end

subgraph group_nlp["NLP stages"]
  node_process["Text processing<br/>preprocess<br/>[PTBTokenizer.java]"]
  node_sequence["Sequence models<br/>labeling<br/>[SequenceModel.java]"]
  node_classify["Classifiers<br/>classification<br/>[Classifier.java]"]
  node_parser["Parsing<br/>syntax"]
  node_coref["Coreference<br/>resolution"]
  node_semantic["Semantics<br/>extraction<br/>[OpenIE.java]"]
  node_time["Time/number<br/>normalization<br/>[TimeAnnotator.java]"]
  node_patterns["Pattern mining<br/>patterning<br/>[Pattern.java]"]
end

subgraph group_infra["Infrastructure"]
  node_trees["Tree infra<br/>representation<br/>[Tree.java]"]
  node_graph["Graph infra<br/>graphs<br/>[Graph.java]"]
  node_semgraph["Semantic graphs<br/>deps graph<br/>[SemanticGraph.java]"]
  node_build_data["Build/data<br/>build assets<br/>[build.xml]"]
end

subgraph group_entry["Entry points"]
  node_entrypoints["Entry points<br/>cli/web/ui"]
end

subgraph group_lang["Language branches"]
  node_lang_process["Lang processing<br/>language-specific"]
  node_lang_trees["Lang trees<br/>language-specific"]
end

node_process -->|"creates"| node_annotation
node_annotation -->|"flows through"| node_pipeline
node_pipeline -->|"runs"| node_annotators
node_annotators -->|"uses"| node_process
node_annotators -->|"uses"| node_sequence
node_annotators -->|"uses"| node_parser
node_annotators -->|"uses"| node_coref
node_annotators -->|"uses"| node_semantic
node_annotators -->|"uses"| node_time
node_pipeline -->|"emits"| node_serialization
node_parser -->|"produces"| node_trees
node_parser -->|"converts to"| node_semgraph
node_trees -->|"feeds"| node_semgraph
node_semgraph -->|"supports"| node_semantic
node_sequence -.->|"shares math"| node_classify
node_classify -.->|"trained with"| node_build_data
node_patterns -->|"feeds"| node_semantic
node_patterns -->|"wraps"| node_annotators
node_lang_process -->|"specializes"| node_process
node_lang_trees -->|"specializes"| node_trees
node_entrypoints -->|"hosts"| node_pipeline
node_entrypoints -->|"hosts"| node_parser
node_entrypoints -->|"hosts"| node_time
node_entrypoints -.->|"depends on"| node_build_data

click node_annotation "https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/pipeline/Annotation.java"
click node_pipeline "https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/pipeline/StanfordCoreNLP.java"
click node_annotators "https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/pipeline/TokenizerAnnotator.java"
click node_serialization "https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/pipeline/ProtobufAnnotationSerializer.java"
click node_process "https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/process/PTBTokenizer.java"
click node_sequence "https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/sequences/SequenceModel.java"
click node_classify "https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/classify/Classifier.java"
click node_parser "https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/parser/lexparser/LexicalizedParser.java"
click node_trees "https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/trees/Tree.java"
click node_graph "https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/graph/Graph.java"
click node_semgraph "https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/semgraph/SemanticGraph.java"
click node_coref "https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/dcoref/CoreferenceSystem.java"
click node_semantic "https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/naturalli/OpenIE.java"
click node_time "https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/time/TimeAnnotator.java"
click node_patterns "https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/patterns/Pattern.java"
click node_lang_process "https://github.com/intel-spark/corenlp/tree/master/src/edu/stanford/nlp/international"
click node_lang_trees "https://github.com/intel-spark/corenlp/tree/master/src/edu/stanford/nlp/trees/international"
click node_entrypoints "https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/pipeline/StanfordCoreNLPServer.java"
click node_build_data "https://github.com/intel-spark/corenlp/blob/master/build.xml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_annotation,node_pipeline,node_annotators,node_serialization toneBlue
class node_process,node_sequence,node_classify,node_parser,node_coref,node_semantic,node_time,node_patterns toneAmber
class node_trees,node_graph,node_semgraph,node_build_data toneMint
class node_entrypoints toneRose
class node_lang_process,node_lang_trees toneIndigo
```

## Components

### Core pipeline

Shared document model and annotator orchestration.

- [Annotation model](https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/pipeline/Annotation.java): doc state · Shared document and per-token/per-sentence annotation containers used throughout the system. · `src/edu/stanford/nlp/pipeline/Annotation.java`
- [Pipeline core](https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/pipeline/StanfordCoreNLP.java): orchestration · Annotator pipeline, pool, factories, and top-level CoreNLP driver that order and run stages. · `src/edu/stanford/nlp/pipeline/StanfordCoreNLP.java`
- [Annotators](https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/pipeline/TokenizerAnnotator.java): stage modules · Main built-in annotators for tokenization, sentence split, POS, parse, NER, regex, sentiment, coref, and relations. · `src/edu/stanford/nlp/pipeline/TokenizerAnnotator.java`
- [Output formats](https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/pipeline/ProtobufAnnotationSerializer.java): serialization · Annotation serializers and outputters for JSON, XML, CoNLL, CoNLLU, protobuf, and text. · `src/edu/stanford/nlp/pipeline/ProtobufAnnotationSerializer.java`

### NLP stages

Text processing, tagging, parsing, semantics, and coreference.

- [Text processing](https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/process/PTBTokenizer.java): preprocess · Tokenization, sentence splitting, XML stripping, truecasing, morphology, stemming, and word shape normalization. · `src/edu/stanford/nlp/process/PTBTokenizer.java`
- [Sequence models](https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/sequences/SequenceModel.java): labeling · CRF, maxent, and sequence decoding infrastructure used by POS tagging, NER, and similar tasks. · `src/edu/stanford/nlp/sequences/SequenceModel.java`
- [Classifiers](https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/classify/Classifier.java): classification · General-purpose classifiers, datasets, objective functions, and linear/logistic model training code. · `src/edu/stanford/nlp/classify/Classifier.java`
- [Parsing](https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/parser/lexparser/LexicalizedParser.java): syntax · Lexicalized, shift-reduce, neural dependency, and evaluation tooling for constituency and dependency parsing. · `src/edu/stanford/nlp/parser/lexparser/LexicalizedParser.java`
- [Coreference](https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/dcoref/CoreferenceSystem.java): resolution · Deterministic, hybrid, and statistical coreference systems with mention detection, sieves, clustering, and scoring. · `src/edu/stanford/nlp/dcoref/CoreferenceSystem.java`
- [Semantics](https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/naturalli/OpenIE.java): extraction · OpenIE, natural logic, clause splitting, relation triples, and higher-level semantic inference. · `src/edu/stanford/nlp/naturalli/OpenIE.java`
- [Time/number](https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/time/TimeAnnotator.java): normalization · Temporal expression extraction, holiday reasoning, and quantifiable entity normalization. · `src/edu/stanford/nlp/time/TimeAnnotator.java`
- [Pattern mining](https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/patterns/Pattern.java): patterning · Surface, dependency, and tokensregex-based pattern application and scoring workflows. · `src/edu/stanford/nlp/patterns/Pattern.java`

### Infrastructure

Shared representations, utilities, and learning/tooling support.

- [Tree infra](https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/trees/Tree.java): representation · Tree, dependency, grammar, head-finding, and tree transformation infrastructure shared by parsing and downstream analysis. · `src/edu/stanford/nlp/trees/Tree.java`
- [Graph infra](https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/graph/Graph.java): graphs · Generic graph, shortest-path, and connected-components utilities used by semantic and dependency structures. · `src/edu/stanford/nlp/graph/Graph.java`
- [Semantic graphs](https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/semgraph/SemanticGraph.java): deps graph · SemanticGraph, Semgrex, and Ssurgeon tools for dependency-style graph analysis and rewriting. · `src/edu/stanford/nlp/semgraph/SemanticGraph.java`
- [Build/data](https://github.com/intel-spark/corenlp/blob/master/build.xml): build assets · Ant/Gradle build files, scripts, model properties, corpora, and bundled data artifacts. · `build.xml`

### Entry points

Servers, demos, and UI surfaces.

- [Entry points](https://github.com/intel-spark/corenlp/blob/master/src/edu/stanford/nlp/pipeline/StanfordCoreNLPServer.java): cli/web/ui · CLI tools, servers, servlets, demos, and webapp entry surfaces built on the shared pipeline. · `src/edu/stanford/nlp/pipeline/StanfordCoreNLPServer.java`

### Language branches

Language-specific tokenization and tree/pipeline adaptations.

- [Lang processing](https://github.com/intel-spark/corenlp/tree/master/src/edu/stanford/nlp/international): language-specific · Language-specific tokenizers, segmenters, and pipeline adaptations for Arabic, Chinese, French, and Spanish. · `src/edu/stanford/nlp/international`
- [Lang trees](https://github.com/intel-spark/corenlp/tree/master/src/edu/stanford/nlp/trees/international): language-specific · Language-specific treebank language packs, head finders, and normalizers for non-English tree conventions. · `src/edu/stanford/nlp/trees/international`

## Connections

- Text processing → Annotation model: creates · Raw text is turned into annotations and token sequences.
- Annotation model → Pipeline core: flows through · The shared document state is passed between annotators.
- Pipeline core → Annotators: runs · StanfordCoreNLP assembles and executes annotator stages.
- Annotators → Text processing: uses · Early annotators depend on tokenization and sentence splitting.
- Annotators → Sequence models: uses · Tagging and NER rely on sequence labeling components.
- Annotators → Parsing: uses · Parse and dependency annotators invoke parser subsystems.
- Annotators → Coreference: uses · Coreference annotators consume earlier linguistic annotations.
- Annotators → Semantics: uses · OpenIE, relation extraction, and natural logic build on syntax and dependencies.
- Annotators → Time/number: uses · Time and quantity annotators add specialized document annotations.
- Pipeline core → Output formats: emits · Pipeline results can be written through multiple output formats.
- Parsing → Tree infra: produces · Parsing centers on tree and dependency representations.
- Parsing → Semantic graphs: converts to · Dependency and semantic outputs are exposed as graph structures.
- Tree infra → Semantic graphs: feeds · Tree and dependency structures underpin semantic graph construction.
- Semantic graphs → Semantics: supports · Semgrex and dependency graphs are used by OpenIE and related extraction.
- Sequence models → Classifiers: shares math · Labeling models reuse classifier and optimization infrastructure.
- Classifiers → Build/data: trained with · Model recipes and corpora are stored with the repository.
- Pattern mining → Semantics: feeds · Pattern-based extraction contributes to higher-level semantic workflows.
- Pattern mining → Annotators: wraps · TokensRegex and regex NER are exposed as pipeline annotators.
- Lang processing → Text processing: specializes · Language-specific preprocessing extends the generic text pipeline.
- Lang trees → Tree infra: specializes · Language-specific tree conventions extend shared tree infrastructure.
- Entry points → Pipeline core: hosts · Servers and demos embed the same core pipeline for interactive use.
- Entry points → Parsing: hosts · Parser-specific web and UI entry points wrap parsing components.
- Entry points → Time/number: hosts · SUTime servlet and demo surfaces expose temporal extraction.
- Entry points → Build/data: depends on · CLI and web launches are configured via scripts and properties.
