How hshashank06/gaidp-blizzard works
Project type and purpose
• This is an AI-driven data-profiling and regulatory compliance pipeline, implemented as a collection of Jupyter notebooks and scripts.
• It extracts structured data from PDF reports, builds semantic search indexes, dynamically generates compliance rules, and assigns risk scores via anomaly detection and LLM-based reasoning. A Gradio UI front-end provides real-time interaction.
File structure
• code/data/synthetic-data-generation: pipeline to produce training/test data (GPT-4 prompts, embedding similarity).
• code/models: three submodules
• cross-encoder (reranking)
• embedder-fine-tuning (custom sentence-transformer)
• unsupervised-learning (Isolation Forest)
• code/src: core pipelines
• automated-rules-generation (LangChain + LangGraph workflows)
• risk-analysis (risk scoring notebooks)
• code/test: sample regulatory PDFs
• artifacts/arch: design docs (initial + final)
• README.md: end-to-end workflow and tech stack
README for additional insights
• Five stages: PDF parsing, vector embeddings & retrieval (Pinecone/FAISS), reranking with cross-encoder, dynamic rule generation (KeyBERT + LangChain/Graph), customer risk profiling (Isolation Forest + LLM scoring), plus external data (SerpAPI) and synthetic data.
• Gradio UI sits atop for user queries.