How eriiiire/openclaw-lark-knowledge-agent works
This repository showcases two content-archiving tools and a local personal-knowledge assistant. The archive workflows start with a user-supplied WeChat or Dedao course URL, crawl and extract content, then organize and save it. The assistant’s documented workflow normalizes and classifies documents, chunks and indexes them, then provides hybrid retrieval and grounded answers through agent, UI, API, MCP, and OpenClaw integrations. Some PKAI pipeline stages are unsampled; their links are shown only where the source index or README supports them.
Main entry points: pipeline_ops.py Pipeline operations · src/main.py Crawler CLI · src/main.py Course crawler CLI · agent_runtime.py Agent runtime
Components 34
WeChat archive
- Crawler CLI
projects/Celueshi/src/main.py - WeChat browser
projects/Celueshi/src/wechat_client.py - Directory parser
projects/Celueshi/src/directory_parser.py - Article extractor
projects/Celueshi/src/article_extractor.py - Article grouping
projects/Celueshi/src/grouping.py - Archive records
projects/Celueshi/src/models.py - Markdown archive
projects/Celueshi/src/writer.py
Dedao archive
- Course crawler CLI
projects/Jingyingrike/src/main.py - Dedao browser
projects/Jingyingrike/src/dedao_client.py - Course catalog
projects/Jingyingrike/src/catalog.py - Lesson extractor
projects/Jingyingrike/src/extractor.py - Topic grouping
projects/Jingyingrike/src/grouping.py - Markdown archive
projects/Jingyingrike/src/writer.py
Knowledge pipeline
- Document ingestion
projects/PersonalKnowledgeAI/src/ingest.py - Document normalization
projects/PersonalKnowledgeAI/src/normalize_docs.py - Classification and review
projects/PersonalKnowledgeAI/src/classify_docs.py - Document chunking
projects/PersonalKnowledgeAI/src/chunking.py - Pipeline operations
projects/PersonalKnowledgeAI/src/pipeline_ops.py - Index builder
projects/PersonalKnowledgeAI/src/build_indexes.py - Embedding backends
projects/PersonalKnowledgeAI/src/embedding_backends.py - Vector stores
projects/PersonalKnowledgeAI/src/vector_stores.py - Chunk and index files
projects/PersonalKnowledgeAI/src/pipeline_ops.py
Assistant access
- Streamlit UI
projects/PersonalKnowledgeAI/src/streamlit_app.py - FastAPI service
projects/PersonalKnowledgeAI/src/api_server.py - MCP service
projects/PersonalKnowledgeAI/src/mcp_server.py - OpenClaw RAG skill
projects/PersonalKnowledgeAI/integrations/openclaw/skills/pkai-rag/scripts/pkai_rag_query.py - Agent runtime
projects/PersonalKnowledgeAI/src/agent_runtime.py - Knowledge agent
projects/PersonalKnowledgeAI/src/agent.py - Hybrid retrieval
projects/PersonalKnowledgeAI/src/retrieval_pipeline.py - Search engine
projects/PersonalKnowledgeAI/src/retrieve.py - Grounded answer generation
projects/PersonalKnowledgeAI/src/rag_answer.py
Other components
- Archive user
- Course archivist
- Knowledge user
Connections 39
- Archive user → Crawler CLI: starts crawl
README.md - Crawler CLI → WeChat browser: opens pages
projects/Celueshi/src/main.py - Crawler CLI → Directory parser: loads directory
projects/Celueshi/src/main.py - Crawler CLI → Article extractor: extracts articles
projects/Celueshi/src/main.py - Crawler CLI → Article grouping: groups records
projects/Celueshi/src/main.py - Crawler CLI → Markdown archive: writes archive
projects/Celueshi/src/main.py - Directory parser → Archive records: uses records
projects/Celueshi/src/directory_parser.py - Article grouping → Archive records: groups records
projects/Celueshi/src/grouping.py - Markdown archive → Archive records: serializes records
projects/Celueshi/src/writer.py - Course archivist → Course crawler CLI: starts crawl
README.md - Course crawler CLI → Dedao browser: opens lessons
projects/Jingyingrike/src/main.py - Course crawler CLI → Course catalog: loads catalog
projects/Jingyingrike/src/main.py - Course crawler CLI → Lesson extractor: extracts lessons
projects/Jingyingrike/src/main.py - Course crawler CLI → Topic grouping: groups content
projects/Jingyingrike/src/main.py - Course crawler CLI → Markdown archive: writes archive
projects/Jingyingrike/src/main.py - Course catalog → Topic grouping: uses grouping
projects/Jingyingrike/src/catalog.py - Knowledge user → Streamlit UI: asks questions
README.md - Knowledge user → FastAPI service: sends requests
README.md - Knowledge user → MCP service: uses tools
README.md - Knowledge user → OpenClaw RAG skill: queries skill
README.md - Document ingestion → Document normalization: normalizes sources
- Pipeline operations → Document normalization: runs normalization
projects/PersonalKnowledgeAI/src/pipeline_ops.py - Pipeline operations → Classification and review: runs classification
projects/PersonalKnowledgeAI/src/pipeline_ops.py - Pipeline operations → Document chunking: runs chunking
projects/PersonalKnowledgeAI/src/pipeline_ops.py - Pipeline operations → Index builder: builds indexes
projects/PersonalKnowledgeAI/src/pipeline_ops.py - Index builder → Embedding backends: creates embeddings
projects/PersonalKnowledgeAI/src/build_indexes.py - Pipeline operations → Vector stores: syncs vectors
projects/PersonalKnowledgeAI/src/pipeline_ops.py - Pipeline operations → Chunk and index files: writes pipeline data
projects/PersonalKnowledgeAI/src/pipeline_ops.py - Streamlit UI → Agent runtime: serves agent workflow
- FastAPI service → Agent runtime: calls runtime
projects/PersonalKnowledgeAI/src/api_server.py - MCP service → Agent runtime: calls runtime
projects/PersonalKnowledgeAI/src/mcp_server.py - OpenClaw RAG skill → Agent runtime: queries RAG
- Agent runtime → Knowledge agent: dispatches requests
projects/PersonalKnowledgeAI/src/agent_runtime.py - Agent runtime → Hybrid retrieval: dispatches search
projects/PersonalKnowledgeAI/src/agent_runtime.py - Knowledge agent → Hybrid retrieval: requests evidence
projects/PersonalKnowledgeAI/src/agent.py - Knowledge agent → Grounded answer generation: generates response
projects/PersonalKnowledgeAI/src/agent.py - Hybrid retrieval → Search engine: runs hybrid search
projects/PersonalKnowledgeAI/src/retrieval_pipeline.py - Hybrid retrieval → Chunk and index files: reads chunks
projects/PersonalKnowledgeAI/src/retrieval_pipeline.py - Search engine → Embedding backends: embeds query
projects/PersonalKnowledgeAI/src/retrieve.py