How sh0g0-1758/scarlet works
Project & Architectural Overview
• Scarlet is a compiler built in C++, and its file hierarchy clearly separates the logical phases of compilation (lexical analysis, parsing, semantic analysis, code generation, and backend optimizations).
• The README confirms that the primary goal is to deepen our understanding of compiler construction while providing the platform for future machine learning projects.
Main Architectural Components
• Compiler Driver:
• The “driver” directory holds the central “driver.cc” that orchestrates the entire compilation process.
• This as the entry point that invokes each compiler phase.
• Frontend: Scanning & Parsing
• Lexer: Located under the “lexer” directory, responsible for reading source code and generating tokens.
• Parser: Found in the “parser” directory, it consumes tokens and constructs the Abstract Syntax Tree (AST).
• Token Module: Under “token” directory, which supports token data structures and definitions.
• AST: The “ast” directory holds implementations (ast.cc, ast.hh) representing syntax trees.
• Additional Submodules such as “binary_operations” and “unary_operations” help manage expression nodes and operator behaviors.
• Semantic Analysis:
• Inside the “parser” directory, the subdirectory “semantic_analysis” suggests that type checking and variable resolution are integrated.
• Backend / Code Generation:
• Code generation is implemented in the “codegen” directory. This component converts the AST to lower-level representations or direct assembly.
• That additional directories such as “codegen/scargen” and “codegen/scasmgen” indicate detailed sub-phases for pretty-printing, generating pseudo assembly, and fixing register issues.
• Supporting Modules & Tools:
• “cmd” may include command-line interface abstraction.
• “regex” is available for lexing pattern matching.
• “numloc.py” might be used to handle numeric literals or locations, which is a utility component.
• Test & Validation Infrastructure:
• There are extensive test directories (“tests”, “test_framework”, “stress”) that encapsulate unit tests, integration tests, and stress/memory tests.
• As secondary modules that interact with the core compiler pipeline for validation and continuous integration (the GitHub workflows reflect multiple OS/compiler configurations).
Data Flow & Relationships
• Within or immediately after Parser the Semantic Analysis phase, which refines the AST.
• Finally, illustrate the transformation into assembly or machine code along with a connection to the underlying linking (handled externally through system tools like GCC) as mentioned in the README.
• Compiler Driver as a controller connecting these phases.