How geopython/pycql works
Project Type and Purpose
• This project is a Python library that implements a parser for the OGC CQL standard.
• The key purpose is to convert a CQL filter expression string into an AST, which is then recursively evaluated to generate filters for use in ORM systems.
• It offers integrations for Django (and also SQLAlchemy) to bridge the AST evaluation into actual query construction.
File Structure
• The top-level directories include configuration files (.github, debian), documentation (docs), library source files (pycql), and tests.
• In the pycql directory, you have modules such as:
• Core parser components (lexer.py, parser.py, ast.py, values.py, util.py).
• Integration support directories (pycql/integrations/django and pycql/integrations/sqlalchemy) which implement methods to convert the AST into ORM-specific filters.
• The tests directory includes test cases for Django integration and for lexer and parser, illustrating how the parser is exercised in different contexts.
Key Components
• Input / CQL Expression: Represents the raw filter string supplied to the parser.
• Lexer Component:
• Source: pycql/lexer.py (and the generated/auxiliary files lextab.py)
• Function: Tokenize the input string.
• Parser Component:
• Source: pycql/parser.py (along with the output file parser.out and supporting tables in parsetab.py)
• Function: Convert the stream of tokens into an AST.
• AST Representation:
• Source: pycql/ast.py and values.py
• Represents the hierarchical structure of conditions, comparisons, and operations (e.g., attribute comparisons, logical conditions, arithmetic expressions).
• Evaluator / Filter Builder:
• Contains the recursive evaluator code (illustrated in the README under the FilterEvaluator class), which uses the AST to build filters.
• Depending on usage, conversion methods like to_filter recursively evaluate AST nodes.
• Integration Adapters:
• Django Integration: Located under pycql/integrations/django. They include modules (evaluate.py, filters.py, parser.py) that convert AST nodes into Django Q objects or ORM filters.
• SQLAlchemy Integration: Located under pycql/integrations/sqlalchemy with similar responsibilities.
• External Tools and Frameworks:
• Although this is a library, it depends on external systems like Django ORM or SQLAlchemy for database filtering.
• CI/CD components (in .github/workflows) and packaging (debian) can be illustrated as supporting infrastructure.
Relationships and Interactions
• A user supplies a CQL filter string.
• The string flows into the Lexer, which tokenizes the input.
• Tokens are fed to the Parser, which constructs the AST.
• The AST is then processed by an Evaluator component.
• Depending on the target integration, the Evaluator uses:
• Django integration (or SQLAlchemy integration) to convert the AST into ORM-specific filtering objects.
• Finally, in an end-to-end scenario (as shown in tests), the filters are applied to execute ORM queries.
Architectural Patterns and Design Principles
• Separation of Concerns: Parsing, AST generation, and evaluation into filters are cleanly separated.
• Recursion: Note the recursive descent in AST evaluation (the evaluator calls itself on subnodes).
• Modularity: Integration modules (Django, SQLAlchemy) are isolated from the core parser so that extensions or alternative integrations are straightforward.
• Testability: The tests directory indicates a commitment to thorough testing via pytest and Django test projects.
• Extensibility: The integration adapters show design flexibility, where support for additional backends could be added.