jbiaojerry/ebook-treasure-chest

1. What this repo is

- This is a static ebook catalog and search site, not a traditional application server.

- The published product lives in docs/ and is intended for GitHub Pages hosting, with all search and navigation happening client-side.

2. Primary content layer

- The large md/ directory is the canonical content store.

- Each Markdown file represents a category/tag page such as 文学.md, 历史.md, Python.md, etc., and contains curated ebook download links or entries for that topic.

- The README’s “热门分类” section is effectively a human-readable summary of these category pages.

3. Generated site output

- docs/index.html is the entry page for the hosted site.

- docs/search.js powers the browser-side search experience.

- docs/books.json, docs/all-books.json, and docs/parse-stats.json are generated data files consumed by the static site.

- docs/.nojekyll indicates GitHub Pages is serving the folder as plain static content, without Jekyll processing.

4. Search architecture

- Search is fully client-side.

- The browser loads prebuilt JSON indexes from docs/, and search.js filters/ranks results in real time.

- The README emphasizes multi-keyword search, so the search index is designed around fast local matching rather than server queries.

5. Data transformation pipeline

- scripts/parse_md_to_json.py is the main bridge from Markdown content to structured JSON.

- scripts/generate_index.py likely turns the category corpus into the searchable homepage/index structure.

- docs/parse-stats.json suggests the generation step also computes content statistics for display or validation.

6. Content sync pipeline

- scripts/sync/ contains the ingestion/update machinery:

- sync_all_books.py for full corpus refreshes

- incremental_sync.py for partial updates

- find_max_book_id.py and max_book_id_found.txt for checkpointing incremental state

- parse_book_detail_enhanced.py for extracting richer book metadata

- backup_md.py for preserving existing Markdown before overwrites

- This is the boundary where external ebook source data is normalized into the repo’s markdown format.

7. Automation and build orchestration

- .github/workflows/full-sync.yml handles complete refresh runs.

- .github/workflows/incremental-sync.yml handles ongoing deltas.

- .github/workflows/generate-site.yml rebuilds/publishes the static site artifacts.

- The architecture is therefore GitHub Actions + generated static assets, with no always-on backend service.

8. Quality and validation

- scripts/sync/test_batch_sync.py indicates batch-sync behavior is tested before or during ingestion.

- The presence of backup and checkpoint files shows the repo is designed to recover from partial sync failures and avoid losing curated Markdown content.

9. README generation / topical curation

- scripts/sync/update_readme_hot_categories.py appears to keep the README’s “热门分类” list aligned with actual category popularity.

- This means the README is not just documentation; it is partly a generated/maintained presentation layer over the corpus.

10. End-to-end data flow

- External ebook sources → sync scripts → Markdown category files in md/ → parser/generator scripts → JSON + HTML assets in docs/ → GitHub Pages static site → browser-side search/navigation.

11. Important architectural boundary

- The repo cleanly separates:

- source content and curation in md/

- generation logic in scripts/

- published runtime in docs/

- That boundary is the key to drawing the architecture diagram correctly: the site is a generated artifact, not a live application with a server database.

12. Overall shape

- Think of it as a content pipeline + static site renderer + GitHub Pages deployment.

- The core user experience is browsing and searching a large, precomputed ebook index, while maintenance is handled by scheduled or manual GitHub Actions workflows.