How ak1484/git_doc works
Project Type and Purpose
• This is an open-source, command-line documentation generator tool.
• It automatically downloads or reads a code repository, parses its files, uses OpenAI (via LangChain) to generate per‐file documentation, stores the results in a FAISS vector database, and produces (or updates) a final Markdown document.
High-Level Architecture Components
a. CLI / Entry Point (main.py)
• Parses user arguments (local path vs. GitHub URL, mode, output directory, API key).
• Coordinates the overall workflow.
b. Repository Acquisition
• download_repo.py – clones a GitHub repo when needed.
• (Or direct file system access for a local path.)
c. Repository Parser (repo_parser.py)
• Walks the directory tree.
• Extracts file metadata and contents.
d. Documentation Generator (doc_generator.py & final_doc_generator.py)
• Uses OpenAI + LangChain to produce doc snippets per file.
• final_doc_generator.py merges snippets into a cohesive Markdown file.
e. Vector Store (vector_store.py & doc_storage.py)
• Wraps FAISS for indexing and retrieving documentation embeddings.
• Supports lookup/update of doc vectors.
f. Updater (updater.py)
• Detects changed files since last run.
• Re-generates docs only for those files.
g. Output (example_usage.py is sample client code; main.py drives actual runs)
• Writes final Markdown doc tree to disk.
• Persists FAISS database files under output_dir.
Data Flow and Interactions
1. User invokes main.py → argument parser
2. main imports/executes download_repo or skips to repo_parser
3. repo_parser emits a file list + contents
4. For each file:
– doc_generator calls OpenAI API via LangChain → returns doc snippet
– vector_store adds/updates FAISS embedding
5. Once all snippets are processed:
– final_doc_generator collates snippets into one .md
– updater (in “update” mode) diffs file list, re-calls doc_generator & vector_store on changed files only
6. Output directory populated with:
– FAISS index files
– final Markdown documentation
Architectural Patterns & Design Principles
• Pipeline pattern: clear separation of “Acquire → Parse → Generate → Store → Collate.”
• Single Responsibility per module (each .py focuses on one stage).
• Pluggable storage (vector_store abstraction over FAISS).
• Incremental update logic in updater.py (avoid full reprocessing).
• External integrations as well‐defined services (OpenAI API, GitHub).
Technologies & Frameworks
• Python 3.x
• OpenAI API + LangChain library
• FAISS (vector similarity search)
• GitPython or subprocess git calls (download_repo.py)
• Markdown output