How github/markup works
1. What this repo is
- github-markup is a Ruby gem that converts repository markup files into HTML.
- Its scope is intentionally narrow: it only performs the “raw markup → HTML” conversion step. Sanitization, syntax highlighting, emoji, task lists, and other GitHub.com rendering steps are outside this repo.
2. Primary entrypoint
- The public API is centered in lib/github/markup.rb and exposed through require 'github/markup'.
- The main usage pattern is: provide a filename plus content, and the library chooses the correct renderer based on the file extension.
3. Markup type registry
- lib/github/markups.rb acts as the registry of supported formats and their file extensions.
- This is the routing layer for the whole gem: it maps extensions like .md, .rst, .pod, .asciidoc, .textile, etc. to the right implementation.
4. Implementation abstraction
- lib/github/markup/implementation.rb defines the common contract for renderers.
- The architecture is pluggable: each markup format is represented as an implementation object rather than hardcoded branching scattered through the codebase.
5. Two implementation styles
- lib/github/markup/gem_implementation.rb covers markups rendered through Ruby gems or in-process libraries.
- lib/github/markup/command_implementation.rb covers markups rendered by shelling out to external commands or interpreters.
- This split is important for the architecture: some formats are native Ruby dependencies, while others depend on system tools such as Python, Perl, or standalone CLI utilities.
6. Format-specific adapters
- lib/github/markup/markdown.rb and lib/github/markup/rdoc.rb are format-specific wrappers for special cases that need custom behavior.
- These adapters sit above the generic implementation layer and handle format quirks without changing the registry or API shape.
7. Executable/rendering bridge
- bin/github-markup is the command-line entrypoint for converting files from the shell.
- The lib/github/commands/ directory contains helper command wrappers such as pod2html, pod62html, and rest2html, which are used to bridge to external format converters.
8. External rendering dependencies
- Supported markups depend on a mix of libraries and tools: commonmarker, RedCloth, rdoc, org-ruby, creole, wikicloth, docutils, asciidoctor, and Perl Pod::Simple.
- In architecture terms, the gem is a dispatcher that delegates rendering to these downstream engines rather than implementing the markup parsers itself.
9. Data flow through the library
- Input begins as a filename plus raw text.
- The filename determines the markup type.
- The selected implementation converts the text to HTML.
- The caller receives HTML and is expected to handle any downstream sanitization or display concerns.
10. Test strategy
- test/markups/ contains paired source and expected HTML fixtures for each supported syntax.
- test/markup_test.rb and test/coverage_test.rb validate the registry, dispatch behavior, and renderer coverage across all supported formats.
- The fixture-heavy layout suggests the repo’s main correctness concern is parity across many converters.
11. Development and automation
- script/bootstrap and script/cibuild support local dependency setup and CI execution.
- GitHub Actions in .github/workflows/ci.yml and .github/workflows/stale.yml handle automated testing and repository maintenance.
- Dependabot configuration indicates dependency updates are part of the operational model.
12. How to draw the architecture
- A good diagram would show:
- a public API / CLI boundary,
- a format registry,
- a shared implementation abstraction,
- multiple concrete renderer adapters,
- and a set of external parser/converter dependencies behind those adapters.
- The key architectural idea is dispatch-by-file-extension into heterogeneous rendering backends.
Main entry points: gem_implementation.rb Gem adapter · implementation.rb Impl contract · markups.rb Markup registry · markup_test.rb Markup tests
Components 23
Public entrypoints
User-facing Ruby and CLI boundaries for conversion.
- Library API · Main require path and public conversion API.
lib/github/markup.rb - CLI tool · Command-line interface for converting files from the shell.
bin/github-markup
Dispatch core
Registry and implementation abstraction that route markup to backends.
- Markup registry · Maps filename extensions to supported markup implementations.
lib/github/markups.rb - Impl contract · Common abstraction shared by all renderers.
lib/github/markup/implementation.rb
Renderer adapters
Format-specific wrappers and backend families used by the dispatcher.
- Gem adapter · Implementation path for Ruby gems and libraries.
lib/github/markup/gem_implementation.rb - Command adapter · Implementation path for shelling out to external tools.
lib/github/markup/command_implementation.rb - Markdown wrapper · Custom behavior for markdown-family rendering.
lib/github/markup/markdown.rb - RDoc wrapper · Custom behavior for RDoc-family rendering.
lib/github/markup/rdoc.rb - POD helpers · Helper wrappers for pod-to-html conversion.
lib/github/commands/pod2html - RST helper · Helper wrapper for reStructuredText conversion.
lib/github/commands/rest2html
External engines
Native gems, interpreters, and commands that perform actual markup conversion.
- CommonMarker · Markdown engine used for modern Markdown conversion.
- RedCloth · Textile rendering library.
- RDoc · RDoc parser and formatter used for documentation markup.
- Org/Creole · In-process support for Org mode and Creole-like formats.
- Wiki/MediaWiki · Wiki-style converters such as Wikicloth.
- Docutils · External reStructuredText converter.
- AsciiDoctor · AsciiDoc rendering engine.
- Pod::Simple · Perl POD converter used through command wrappers.
Verification
Fixture-driven tests that validate dispatch and renderer coverage.
- Markup tests · Registry and dispatch tests across supported formats.
test/markup_test.rb - Coverage tests · Checks renderer coverage against supported markups.
test/coverage_test.rb - Markup fixtures · Paired source and expected HTML fixtures for each syntax.
test/markups
Automation
Local scripts and CI/maintenance configuration.
- Build scripts · Bootstrap and CI helper scripts for local and automated runs.
script/bootstrap - CI config · GitHub Actions workflows and dependency automation.
.github/workflows/ci.yml
Connections 21
- Library API → Markup registry: dispatches by extension · The public API delegates filename-based routing to the registry.
- CLI tool → Library API: uses API · CLI entrypoint invokes the library conversion flow.
- Markup registry → Impl contract: selects impls · Registry entries resolve to implementation objects.
- Markdown wrapper → Gem adapter: specializes · Markdown-specific wrapper builds on the gem implementation path.
- RDoc wrapper → Impl contract: specializes · RDoc wrapper customizes behavior over the shared contract.
- Gem adapter → CommonMarker: delegates to · Gem-backed renderer uses CommonMarker for markdown.
- Gem adapter → RedCloth: delegates to · Gem-backed renderer uses RedCloth for textile.
- Gem adapter → RDoc: delegates to · Gem-backed renderer uses RDoc for documentation markup.
- Gem adapter → Org/Creole: delegates to · Gem-backed renderer uses org/creole libraries for alternate syntaxes.
- Gem adapter → Wiki/MediaWiki: delegates to · Gem-backed renderer uses wiki-style libraries for wiki markups.
- Gem adapter → AsciiDoctor: delegates to · Gem-backed renderer uses AsciiDoctor for AsciiDoc.
- Command adapter → Docutils: spawns · Command-backed renderer shells out to docutils for reStructuredText.
- POD helpers → Pod::Simple: bridges to · POD helper wraps Perl POD conversion tooling.
- RST helper → Docutils: bridges to · RST helper wraps external docutils-based conversion.
- Impl contract → Gem adapter: implemented by · One major renderer family is in-process Ruby libraries.
- Impl contract → Command adapter: implemented by · One major renderer family shells out to external commands.
- Markup tests → Markup registry: validates · Tests assert registry mappings and dispatch behavior.
- Markup tests → Markup fixtures: uses fixtures · Behavior is verified against paired source/HTML samples.
- Coverage tests → Markup registry: covers · Coverage tests ensure supported formats are exercised.
- Build scripts → CI config: feeds · Local scripts support CI and automation workflows.
- CI config → Markup tests: runs · CI executes the main test suite.