dead8309/markitdown-ts

1. What this repo is

- markitdown-ts is a TypeScript conversion library, not an app.

- Its core job is to turn many input formats into a single Markdown-oriented output shape: { title, markdown, text_content }.

- The repository is organized as a library package plus a set of example mini-projects and a test suite.

2. Public entrypoint and orchestration

- src/index.ts exposes the library API, centered on the MarkItDown class defined in src/markitdown.ts.

- MarkItDown is the orchestration layer: it accepts a source, selects the right converter, and returns a normalized result or null/undefined when conversion cannot be performed.

- The two main input modes are convert(source) for path/URL/Response and convertBuffer(source, { file_extension }) for in-memory data.

3. Source ingestion boundary

- The library is built around multiple source types: local file paths, remote URLs, Response objects, and raw Buffers.

- That boundary matters architecturally because the same converter logic is reused across filesystem, network, and serverless/edge scenarios.

- The README explicitly positions this for environments without persistent disks, and fetch can be overridden for custom runtimes.

4. Converter subsystem

- src/converters/ contains the format-specific adapters, which are the main internal subsystem.

- Each file in that folder handles one content family: PDF, DOCX, XLSX, HTML, plain text, XML/RSS/Atom, IPYNB, image, audio/WAV/MP3, ZIP, YouTube, Wikipedia, and Bing SERP.

- The architecture is “dispatch to specialized converter,” not “one generic parser.”

5. Document and text-oriented conversions

- Document formats are handled by dedicated converters for pdf, docx, xlsx, ipynb, and text-based formats.

- The html converter likely depends on the custom Turndown layer in src/custom-turndown.ts to turn HTML into Markdown with project-specific rules.

- plain-text.ts and xml-rss-atom.ts cover simple textual sources, while bingserp.ts and wikipedia.ts are URL/document hybrids that use page structure rather than file extension alone.

6. Media and enrichment paths

- image.ts, mp3.ts, and wav.ts represent media-specific branches.

- Image conversion supports optional LLM-based description generation, which introduces an external model/client dependency into an otherwise local conversion pipeline.

- Audio conversion currently appears metadata-only in practice, with speech recognition explicitly not implemented yet in the README.

7. Network-dependent converters

- Some converters are inherently remote or URL-driven: YouTube transcript extraction, Wikipedia page conversion, and Bing SERP parsing.

- These paths depend on network access and/or page-specific structure, so they are separate from pure file parsing.

- The YouTube path has an explicit feature flag for transcript extraction plus a language option, making transcript retrieval an optional enrichment step.

8. Archive handling and recursion

- zip.ts is a special boundary because it recursively iterates over extracted contents instead of treating the archive as a single document.

- The cleanupExtracted option in the API shows that ZIP conversion has lifecycle concerns around temporary extraction artifacts.

- This is the main “composite input” path in the repo: one conversion request can fan out into many nested conversions.

9. Shared types and implementation support

- src/types.ts centralizes the shared option/result shapes used across converters.

- src/declarations.d.ts indicates the library needs local type declarations for some dependencies or runtime APIs.

- This shared layer helps keep the many converters consistent while still allowing format-specific options like styleMap, url, llmModel, and transcript settings.

10. Examples as executable documentation

- examples/ contains one self-contained package per supported scenario, each with its own src/index.ts, package.json, and TypeScript config.

- These are not just docs; they show the intended integration style for each converter family, including local files, URLs, and sample assets.

- The examples also reveal the repo’s supported inputs and serve as reference integration boundaries for consumers.

11. Testing and fixture strategy

- test/index.test.ts drives automated verification using representative files in test/__files.

- The fixture set mirrors the supported converter matrix: PDF, DOCX, XLSX, HTML, ZIP, notebooks, WAV, TXT, XML/RSS, SERP, Wikipedia, and more.

- This suggests the test architecture is conversion-result based, with format fixtures acting as the main dependency boundary.

12. Build, CI, and release flow

- The repo uses pnpm workspaces (pnpm-workspace.yaml) and TypeScript configuration at the root and per-example level.

- GitHub Actions (.github/workflows/ci.yml, release.yml) handle validation and publishing/release automation.

- build.config.ts and Renovate config indicate standard library packaging and dependency maintenance rather than application runtime infrastructure.

13. How to draw the main architecture

- The simplest diagram is: consumer input → MarkItDown orchestration → source normalization/fetching → format-specific converter → Markdown result.

- Add side branches for optional external services: LLM-backed image descriptions, YouTube transcript retrieval, and network-driven page converters.

- Add a composite branch for ZIP archives that loops back into the same converter pipeline for nested files.