How daintiness-group/pythia works
Pythia is a Java/Spark library for producing automated statistical profiles and reports from registered datasets. The documented workflow is to create a profiler, register a dataset and schema, optionally add labeling and dominance settings, choose analyses, compute the profile, and generate reports. The graph preserves the documented analysis capabilities and the principal data products. Relationships within unsampled analysis implementations are omitted; documented workflow links cite the README.
Main entry points: DatasetProfiler.java Dataset Profiler · DescriptiveStatisticsCalculator.java Descriptive Statistics · HistogramManager.java Histograms · report Report Generators
Components 20
Profiler API
- Dataset Profiler
src/main/java/gr/uoi/cs/pythia/engine/DatasetProfiler.java - App Controller
src/main/java/gr/uoi/cs/pythia/Appcontroller/AppController.java - Dataset Readers
src/main/java/gr/uoi/cs/pythia/reader
Profile Analyses
- Type Detection
src/main/java/gr/uoi/cs/pythia/datatypeIdentifier/ScoreCalculatorManager.java - Descriptive Statistics
src/main/java/gr/uoi/cs/pythia/descriptivestatistics/DescriptiveStatisticsCalculator.java - Cardinalities
src/main/java/gr/uoi/cs/pythia/cardinalities - Histograms
src/main/java/gr/uoi/cs/pythia/histogram/HistogramManager.java - Correlations
src/main/java/gr/uoi/cs/pythia/correlations - Decision Trees
src/main/java/gr/uoi/cs/pythia/decisiontree/DecisionTreeManager.java - Clustering
src/main/java/gr/uoi/cs/pythia/clustering - Outlier Detection
src/main/java/gr/uoi/cs/pythia/outliers - Regression
src/main/java/gr/uoi/cs/pythia/regression
Patterns and Highlights
- Column Labeling
src/main/java/gr/uoi/cs/pythia/labeling/RuleSet.java - Dominance Patterns
src/main/java/gr/uoi/cs/pythia/patterns/PatternManager.java - Story Highlights
src/main/java/gr/uoi/cs/pythia/highlights/HighlightsManagerV01.java
Results and Reports
- Dataset Profile
src/main/java/gr/uoi/cs/pythia/model/DatasetProfile.java - Column Results
src/main/java/gr/uoi/cs/pythia/model/Column.java - Report Generators
src/main/java/gr/uoi/cs/pythia/report - Dataset Writers
src/main/java/gr/uoi/cs/pythia/writer
Other components
- Library User
Connections 21
- Library User → Dataset Profiler: creates
- Library User → Dataset Profiler: registers dataset
- Dataset Profiler → Dataset Readers: loads dataset
src/main/java/gr/uoi/cs/pythia/engine/DatasetProfiler.java - Library User → Dataset Profiler: sets labeling
- Dataset Profiler → Column Labeling: computes labels
src/main/java/gr/uoi/cs/pythia/engine/DatasetProfiler.java - Library User → Dataset Profiler: sets dominance
- Library User → Dataset Profiler: selects analyses
- Dataset Profiler → Descriptive Statistics: computes statistics
- Dataset Profiler → Histograms: computes histograms
src/main/java/gr/uoi/cs/pythia/engine/DatasetProfiler.java - Dataset Profiler → Correlations: computes correlations
src/main/java/gr/uoi/cs/pythia/engine/DatasetProfiler.java - Dataset Profiler → Decision Trees: computes trees
src/main/java/gr/uoi/cs/pythia/engine/DatasetProfiler.java - Dataset Profiler → Clustering: runs clustering
src/main/java/gr/uoi/cs/pythia/engine/DatasetProfiler.java - Dataset Profiler → Outlier Detection: runs detection
src/main/java/gr/uoi/cs/pythia/engine/DatasetProfiler.java - Dataset Profiler → Regression: runs regression
src/main/java/gr/uoi/cs/pythia/engine/DatasetProfiler.java - Dataset Profiler → Dominance Patterns: identifies patterns
- Dataset Profiler → Dataset Profile: computes profile
src/main/java/gr/uoi/cs/pythia/engine/DatasetProfiler.java - Descriptive Statistics → Column Results: profiles columns
src/main/java/gr/uoi/cs/pythia/descriptivestatistics/DescriptiveStatisticsCalculator.java - Histograms → Column Results: sets histograms
src/main/java/gr/uoi/cs/pythia/histogram/HistogramManager.java - Story Highlights → Dataset Profile: stores highlights
src/main/java/gr/uoi/cs/pythia/highlights/HighlightsManagerV01.java - Dataset Profiler → Report Generators: generates reports
src/main/java/gr/uoi/cs/pythia/engine/DatasetProfiler.java - Report Generators → Library User: returns findings