How matteobenvenuti/rpgleparser works
Main Components
• Grammar Files:
• Located under src/main/antlr4, they include “RpgLexer.g4” and “RpgParser.g4”. These files form the basis of the language grammar and are used by ANTLR to generate the lexer and parser classes.
• Java Processing and Utility Classes:
• Under src/main/java, you have several packages (e.g., org/rpgleparser/tokens and org/rpgleparser/utils) that include classes for token transformation (PreprocessTokenSource, TransformTokenSource, WrappedTokenSource) and utilities (FixedWidthBufferedReader, FixedWidthInputStream, TreePrinterListener, TreeUtils). These classes enrich the basic parsing functionality with custom token handling and parse tree processing.
• Example and Listener Implementations:
• The examples package (e.g., examples/loggingListener/LoggingListener.java) demonstrates how the parser’s outputs might be used in a real scenario, showing integration of listeners for logging and debugging.
• Testing Suite:
• A comprehensive set of unit and integration tests reside in src/test, including various directories for checking different aspects such as opcodes, expressions, directives, declarations, and more. This robust testing structure verifies correct parsing and transformation of RPG source code.
Relationships and Data Flow
• Input Stage:
• The system starts with a source of RPG code (in either fixed- or free-format).
• Lexical Analysis:
• The ANTLR-generated lexer (from RpgLexer.g4) tokenizes the input.
• Syntactic Parsing:
• The parser (from RpgParser.g4) builds a parse tree from the token stream.
• Token Source Transformation:
• Custom token sources (PreprocessTokenSource, TransformTokenSource, WrappedTokenSource) process tokens further, possibly applying pre-processing or transformations specific to RPG.
• Parse Tree Processing and Utilities:
• Utility classes (TreePrinterListener and TreeUtils) and the listener implementations (e.g., LoggingListener) help in traversing or handling the parse tree. Additionally, fixed-width utilities (FixedWidthBufferedReader, FixedWidthInputStream) manage special input requirements.
• Testing:
• The test suite feeds sample RPG source files through this pipeline to ensure the reliability of all the above components.
Architectural Patterns and Design Principles
• Layered Architecture:
• The project cleanly separates the grammar (parsing) layer, processing (custom token sources and utilities) layer, and testing/validation layer.
• Modular Organization:
• Organized into Maven-standard directories (src/main and src/test), it allows easy integration and extension.
• Usage of ANTLR v4:
• Leveraging ANTLR’s capabilities, the separation of lexer, parser, and listener implementation adheres to well-established compilation and transformation patterns.