Skip to content

Commit 00ea65e

Browse files
authored
Update grammar files and developer guide (opensearch-project#4301)
* Update grammar files and developer guide Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com>
1 parent 4b3b846 commit 00ea65e

3 files changed

Lines changed: 743 additions & 387 deletions

File tree

DEVELOPER_GUIDE.rst

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -173,6 +173,7 @@ Here are other files and sub-folders that you are likely to touch:
173173
- ``build.gradle``: Gradle build script.
174174
- ``docs``: documentation for developers and reference manual for users.
175175
- ``doc-test``: code that run .rst docs in ``docs`` folder by Python doctest library.
176+
- ``language-grammar``: centralized package for ANTLR grammar files. See `Language Grammar Package`_ for details.
176177

177178
Note that other related project code has already merged into this single repository together:
178179

@@ -441,3 +442,29 @@ with an appropriate label `backport <backport-branch-name>` is merged to main wi
441442
PR. For example, if a PR on main needs to be backported to `1.x` branch, add a label `backport 1.x` to the PR and make sure the
442443
backport workflow runs on the PR along with other checks. Once this PR is merged to main, the workflow will create a backport PR
443444
to the `1.x` branch.
445+
446+
Language Grammar Package
447+
========================
448+
449+
The ``language-grammar`` package serves as a centralized repository for all ANTLR grammar files used throughout the OpenSearch SQL project. This package contains the definitive versions of grammar files for:
450+
451+
- SQL parsing (``OpenSearchSQLParser.g4``, ``OpenSearchSQLLexer.g4``)
452+
- PPL parsing (``OpenSearchPPLParser.g4``, ``OpenSearchPPLLexer.g4``)
453+
- Legacy SQL parsing (``OpenSearchLegacySqlParser.g4``, ``OpenSearchLegacySqlLexer.g4``)
454+
- Spark SQL extensions (``SparkSqlBase.g4``, ``FlintSparkSqlExtensions.g4``, ``SqlBaseParser.g4``, ``SqlBaseLexer.g4``)
455+
456+
Purpose
457+
-------
458+
459+
The language-grammar package enables sharing of grammar files between the main SQL repository and the Spark repository, ensuring consistency and reducing duplication. Once updated, the package automatically triggers CI to upload the new version to Maven Central for consumption by other projects.
460+
461+
Updating Grammar Files
462+
----------------------
463+
464+
When grammar files are modified in their respective modules (``sql/``, ``ppl/``, ``legacy/``, ``async-query-core/``), they must be manually copied to the ``language-grammar/src/main/antlr4/`` directory.
465+
466+
**Workflow:**
467+
468+
1. Modify grammar files in their source locations (e.g., ``sql/src/main/antlr/``)
469+
2. Copy updated files to ``language-grammar/src/main/antlr4/``
470+
3. Commit changes to trigger automatic Maven publication via CI

0 commit comments

Comments
 (0)