How hernehunter/sitemap works
This Go library parses sitemap documents from readers, files, or URLs and streams page entries as ParseResult values. Fetching can recursively follow sitemap indexes, with concurrency limits and separate filters for index and page URLs. Parsing uses the standard XML decoder by default or an opt-in custom lexer; selected fields, date cutoffs, Jalali dates, and regex filtering support the workflow. The graph reflects README capabilities and shown calls; unsampled modules are included where documented, but their internal wiring is not inferred.
Main entry points: sitemap.go Parse and Fetch API · parsefast.go Custom Lexer Parser · parser.go Parser Dispatch · parsesafe.go Standard XML Parser
Components 19
Streaming API
- Parse and Fetch API
sitemap.go - File Input
sitemap.go - Result Channel
sitemap.go
Fetch and Filter
- Recursive Fetch
sitemap.go - HTTP Fetch
fetcher.go - URL Filters
filter.go
XML Parsing
- Parser Dispatch
parser.go - Standard XML Parser
parsesafe.go - Custom Lexer Parser
parsefast.go - XML Tokenizer
lexer.go - Entity Decoding
entity.go
Fields and Dates
- Field Selection
fields.go - Entry Buffers
buffers.go - Date Parsing
time.go - Results and Errors
types.go - Error Definitions
errors.go
Other components
- Library Caller
- Result Consumer
- Sitemap Host
Connections 21
- Library Caller → Parse and Fetch API: calls
README.md - Parse and Fetch API → File Input: opens file
sitemap.go - Parse and Fetch API → Recursive Fetch: schedules fetch
sitemap.go - Parse and Fetch API → Result Channel: streams results
sitemap.go - Result Channel → Result Consumer: yields values
README.md - Recursive Fetch → HTTP Fetch: fetches URL
sitemap.go - Sitemap Host → HTTP Fetch: serves sitemap
README.md - Recursive Fetch → Parser Dispatch: parses reader
sitemap.go - Recursive Fetch → Recursive Fetch: recurses indexes
sitemap.go - Recursive Fetch → URL Filters: checks URLs
sitemap.go - Parser Dispatch → Standard XML Parser: defaults to
parser.go - Parser Dispatch → Custom Lexer Parser: dispatches option
parser.go - Standard XML Parser → Entry Buffers: fills buffers
parsesafe.go - Standard XML Parser → Field Selection: matches fields
parsesafe.go - Custom Lexer Parser → XML Tokenizer: reads tokens
parsefast.go - Custom Lexer Parser → Entry Buffers: fills buffers
parsefast.go - Custom Lexer Parser → Field Selection: matches fields
parsefast.go - XML Tokenizer → Entity Decoding: decodes entities
entity.go - Entry Buffers → Date Parsing: parses lastmod
buffers.go - Parse and Fetch API → Results and Errors: returns results
README.md - Parse and Fetch API → Error Definitions: reports errors
README.md