How blackhat955/high_performance_texts_search_engine works
Project Type and Purpose
• This is a single‐binary, command‐line C++ tool for text search with fuzzy matching.
• It reads an input text file, searches for exact or inexact (Levenshtein distance) matches of a given keyword in parallel using multiple threads, and prints unique matching lines.
File Structure Insights
• Top level:
• textSearch.cpp – entire implementation (parsing, search logic, thread management, Levenshtein).
• sample.txt – example input.
• README.md – detailed description, requirements, usage.
• .github/workflows/c-cpp.yml – CI build/test pipeline.
• No frontend/backend split; it’s a CLI utility. Concurrency and algorithmic modules are co-located in one source file.
Key Components
a. Command-Line Interface (CLI)
• Parses arguments (input filename, search keyword, max edit distance, thread count).
b. Input Loader
• Reads the file into memory or streams it line by line.
c. Search Dispatcher / Thread Manager
• Creates a thread pool or spawns N threads.
• Divides the input (e.g., by file chunk or line ranges).
d. Search Worker (per thread)
• For each assigned line:
• Performs exact match check.
• If no exact match or fuzzy mode: invokes Levenshtein Distance module.
• Determines if the line is within the allowed edit distance.
e. Levenshtein Module
• Computes edit distance between keyword and line content.
• Exposed as a function called by each Search Worker.
f. Result Collector (Thread‐Safe)
• Aggregates unique matching lines in a shared vector.
• Employs mutex or lock to ensure thread safety and prevent duplicates.
g. Output Formatter
• After all threads complete, iterates over the result vector and prints each line with its line number.
h. (Optional) CI/CD Pipeline
• .github/workflows/c-cpp.yml – build and test configuration.
Relationships and Data Flow
• User → CLI: supplies filename & parameters.
• CLI → Input Loader: hands off filename.
• Input Loader → Search Dispatcher: provides text buffer or line indices.
• Search Dispatcher → Thread Pool: issues work items.
• Thread Pool → Search Worker: each thread processes its chunk.
• Search Worker ↔ Levenshtein Module: calls distance function.
• Search Worker → Result Collector: pushes matches under lock.
• After join, Search Dispatcher → Output Formatter: prints results to stdout.
Architectural Patterns and Principles
• Concurrency Pattern: Thread Pool or Fork–Join.
• Single Responsibility: Split parsing, searching, matching, collecting, printing.
• Data Encapsulation: Levenshtein logic isolated as a pure function.
• Thread Safety: Mutex‐guarded shared resource.