How ethicalml/xai works
Project type and purpose
• This is a Python-based open-source library/toolkit (“XAI”) for eXplainable AI: a set of modules to analyze datasets, evaluate models, and monitor production metrics.
• It is *not* a full-stack web app, but rather a library consumed by data scientists (e.g. via Jupyter notebooks) and documented via Sphinx/MkDocs.
Main logical layers/components
a. User/Client Layer
– Jupyter Notebook or Python script (“Examples” folder)
– CLI / direct Python import (pip install xai)
b. Application Layer (XAI Core Package)
– xai.data: data loaders (e.g. load_census)
– xai.preprocessing: balancing, grouping, train‐test split
– xai.visualisation: imbalance_plot, correlations, confusion_matrix_plot, roc_plot, smile_imbalance
– xai.metrics: feature_importance, metrics_plot
c. Infrastructure / Dependencies
– External libraries: pandas, numpy, scikit-learn, TensorFlow/Keras, matplotlib, seaborn
– Data sources: built-in CSV (xai/data/census.csv) or user-supplied datasets
d. Documentation Pipeline
– Sphinx (docs_src/ + conf.py) → HTML docs (docs/)
– MkDocs (mkdocs.yml) for GitHub Pages
e. Packaging & Distribution
– setup.py / meta.yaml / env.yml / requirements.txt → PyPI distribution
Relationships & interactions
• The user (data scientist) writes notebook code that imports xai.* modules → invokes data loaders and analysis functions → generates DataFrame outputs and calls visualization routines → matplotlib / seaborn render charts inline.
• XAI libraries call on external dependencies for numeric computing (NumPy), data manipulation (Pandas), ML evaluation (scikit-learn), deep‐learning (TensorFlow/Keras).
• The built-in data file (census.csv) is bundled in the package under xai/data; user data can also be passed in.
• Documentation build tools read the source docs (docs_src/) and generate static assets (docs/) that get deployed to GitHub Pages.
• Packaging scripts (setup.py, MANIFEST.in, meta.yaml) read source code to package and publish to PyPI.
Architectural patterns or design principles
• Modular library structure: each concern (data loading, sampling, visualization, metrics) lives in its own module.
• Dependency Injection: user supplies model evaluation callback (e.g., get_avg) to feature_importance.
• Separation of concerns: visualization and data processing are decoupled—functions return DataFrames or metrics objects which are then passed to plot routines.
• Pluggable backends: supports both pure-Python computing (scikit-learn) and deep learning (TensorFlow).
Elements
• Notebook → xai.data → Data Source
• Notebook → xai.preprocessing → Notebook
• Notebook → xai.visualisation → matplotlib/seaborn → renders plots
• xai.metrics → scikit-learn / TensorFlow → returns metrics → xai.visualisation
• docs_src/ → Sphinx/MkDocs → docs/ → GitHub Pages
• setup.py + source code → PyPI