[Feature] Support Latest Xgrammar Structural Tag#24708
Conversation
Signed-off-by: Hanchenli <lihanc2002@gmail.com>
There was a problem hiding this comment.
Code Review
This pull request adds support for a new structural tag format in XGrammar, including updates to the API protocol and the backend implementation. The changes introduce backward compatibility for the legacy format while enabling the new one. My review focuses on improving the type safety of the new API definition and reducing code duplication in the backend logic. I've identified two high-severity issues related to these points.
chaunceyjiang
left a comment
There was a problem hiding this comment.
Xgrammar has not officially released this feature yet. Should we wait for the official release before integrating this new feature?
yes, this should be held, at least after 0.10.2 |
|
This pull request has merge conflicts that must be resolved before it can be |
Signed-off-by: Hanchenli <lihanc2002@gmail.com>
|
I did some merging and XGrammar version has been bumped. Maybe we can merge this now after review? @aarnphm @chaunceyjiang |
|
Can you check the failure? has to do with the args rename recently. |
The tests seem to be passing for me in local. |
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com> Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Hanchenli <61769611+Hanchenli@users.noreply.github.com>
Signed-off-by: Hanchenli <lihanc2002@gmail.com>
|
|
||
| structural_tag_config = { | ||
| "type": "structural_tag", | ||
| "format": { |
There was a problem hiding this comment.
I suggest adding some usage examples in the examples directory.
|
Closing as the functionality has been added through #25515 |
Purpose
This PR is created to support the upcoming XGrammar Structural Tag feature. The latest tags may be in the format of:
{
"type": "structural_tag",
"format": {
"type":
"triggered_tags",
"tags": [{
"begin": "hello_flag",
"content": {
"type": "any_text"
},
"end": "hello"
}],
"triggers": ["hello"],
"stop_after_first":
False
}
}
This requires us to update the format checking for the API format and the construction of structural tag at XGrammar Backend.
Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.