How astrochialinko/watsonqasystem works
Project type and purpose
• This is an open-source, experimental QA / IR (information retrieval) tool that replicates part of IBM Watson’s Jeopardy QA pipeline, focusing on indexing Wikipedia pages and retrieving the most likely page for a given trivia clue.
File structure
• Top-level artifacts:
• pom.xml: Maven build, dependencies (Lucene, OpenNLP, OpenAI API)
• src/main/java/edu/arizona/cs: core Java code
– BuildIndex.java: builds Lucene indexes from Wikipedia dump
– *Analyzer.java (LemmaAnalyzer, WikipediaAnalyzer): custom Lucene Analyzers
– QueryEngine.java: issues queries against Lucene index, optional GPT re-ranking
– MainWatson.java: orchestrator / CLI entry point
– ResultClass.java: holds retrieval results
• src/main/resources/questions.txt: 100 Jeopardy questions
• index folders (not shown in file_tree but in README): four separate Lucene indexes (std, lemma, stem, wiki)
• src/test/java/...: JUnit tests for each retrieval configuration
• analysis/analysis_plot.py & outputs/: Python script for plotting performance
README for additional insights
• Describes five retrieval modes (standard tokenizer, lemmatization, stemming, Wikipedia tokenizer, ChatGPT rerank)
• Performance metrics (P@1, MRR)
• External dependencies: Lucene, OpenNLP dictionary, OpenAI API for ChatGPT reranking
a. Main components
1. Data Ingestion & Indexing
• Wikipedia Dump → BuildIndex → Lucene Index (std/lemma/stem/wiki)
2. Query Processing
• questions.txt or interactive input → QueryEngine
• Lucene Analyzers (Standard, LemmaAnalyzer, PorterStem, WikiAnalyzer)
3. Retrieval & Ranking
• Lucene Searcher applies chosen Similarity (BM25, Boolean, etc.)
• Results → ResultClass
4. Optional Reranking
• Top-K results → OpenAI ChatGPT API → reranked list
5. Evaluation & Analysis
• TestWaston*.java → performance metrics
• analysis_plot.py → plots MRR/P@1 from outputs/
b. Relationships & data flows
c. Architectural patterns / design principles
• Modular “Analyzer” strategy: interchangeable tokenization pipelines via Lucene’s Analyzer API
• Pluggable retrieval modes: tests select one similarity/analysis chain
• Separation of concerns: indexing vs. querying vs. evaluation vs. analysis
d. Key technologies / libraries
• Apache Lucene (indexing, search, Analyzer framework, Similarity measures)
• Apache OpenNLP for lemmatization (en-lemmatizer.dict, LemmaAnalyzer)
• OpenAI API (ChatGPT) for reranking in TestWastonStemChat
• JUnit / Maven Surefire for automated testing
• Python / matplotlib (analysis_plot.py) for performance visualization