How yuguerten/plot_keywords works
Project type and purpose
• This is a standalone data‐processing and visualization tool (“Keywords Visualization Tool”), not a full‐stack web app. Its sole responsibility is to take a free‐form idea description, extract keywords, embed them in vector space, reduce dimensionality, and render an interactive 2D plot (HTML).
File structure
• Top level:
– README.md → project description and pipeline steps
– plot_embbeding.ipynb → Jupyter notebook orchestrating the pipeline and demo
– keywords_2d_plot.html → final interactive output
• No frontend/backend folders; everything lives in one code pipeline.
README for additional insights
• Pipeline stages are clearly enumerated:
1. Input description
2. Keyword extraction (Gemini 1.5 Flash)
3. Sentence embeddings (SentenceTransformer)
4. 2D reduction (PCA via scikit‐learn)
5. Plotly‐based plotting
• External service: Google Generative AI (Gemini)
• Core libraries: langchain, sentence-transformers, scikit-learn, plotly, numpy, matplotlib
Components & interactions
a. Main components
• Input Module (CLI or Notebook front‐end)
• KeywordExtraction Service (external API call to Google Gemini via google-generativeai, orchestrated by LangChain)
• Embedding Module (SentenceTransformer model running locally)
• DimensionalityReduction Module (PCA via scikit‐learn)
• Visualization Module (Plotly to generate HTML)
• Data artifacts (temporary in-memory vectors; final HTML file)
b. Relationships & dataflow
• Input Module → (user idea) → KeywordExtraction Service
• KeywordExtraction Service → (keywords list) → Embedding Module
• Embedding Module → (high‐dim vectors) → DimensionalityReduction Module
• DimensionalityReduction Module → (2D coords) → Visualization Module
• Visualization Module → (keywords_2d_plot.html) → User browser
c. Architectural patterns & principles
• Pipeline / linear dataflow pattern
• Separation of concerns: each stage is isolated (extraction, embed, reduce, viz)
d. Key technologies/frameworks
• LangChain for orchestration of LLM calls
• google-generativeai for Gemini 1.5 Flash API
• sentence-transformers for embeddings
• scikit-learn PCA for reduction
• Plotly for interactive plotting
• NumPy/Matplotlib for numerical ops and optional static plots