Ygot diff tool#103
Merged
anand-kumar-subramanian merged 5 commits intosonic-net:masterfrom May 22, 2023
Merged
Conversation
Standard ygot.Diff API flattens both structs into {path, value} pair
list and compares them to resolve new, modified and deleted values.
This approach does not scale well for large ygot trees. Following are
the go benchmarks for ygot.Diff API with openconfig-acl ygot trees.
Size "MxN" stands for ygot tree with M ACLs and N rules in each ACL.
+-------------------+---------------+----------------+
| Test case | size=10x10 | size=10x100 |
+-------------------+---------------+----------------+
| No changes | 9024089697ns | 854023452896ns |
| One attr changed | 9285911808ns | 857292949475ns |
| All attrs changed | 9333680743ns | 855581319202ns |
| Create all attrs | 80830018ns | 858752644ns |
| Delete all attrs | 83082154ns | 755762682ns |
+-------------------+---------------+----------------+
Implemented a custom ygot diff tool to compare the ygot struct fields
inline. Uses reflection to traverse and compare structs.
Function signature:
func Diff(s1, s2 ygot.GoStruct, opt DiffOptions) (DiffResults, error)
type DiffOptions struct {
RecordAll bool // include all s2 attrs, even if not modified
}
type DiffResults struct {
Update []*gnmi.Update
Delete []*gnmi.Path
}
Diff function will be used for generating notification messages for
SMPLE subscriptions -- by comparing ygot objects from the previous
iteration with the current objects. By default, Diff returns delete
paths and update values for new/modified fields only. Suitable for
SAMPLE subscription processing when suppress_redundant=true. The
suppress_redundant=false case requires all set attributes of the ygot
struct as update values (even if they are not modified). This can be
achieved by passing RecordAll=true in the DiffOptions.
Go benchmark for the new Diff API with openconfig-acl model:
+-------------------+------------+-------------+---------------+
| Test case | size=10x10 | size=10x100 | size=100x1000 |
+-------------------+---------------+--------------------------+
| No changes | 1784778ns | 15460520ns | 1757874254ns |
| One attr changed | 1547512ns | 16506496ns | 1786245108ns |
| All attrs changed | 3562498ns | 34306626ns | 3846524858ns |
| Create all attrs | 7101530ns | 74663888ns | 8245685970ns |
| Delete all attrs | 120232ns | 125293ns | 453306ns |
+-------------------+------------+-------------+---------------+
Signed-off-by: Sachin Holla <sachin.holla@broadcom.com>
Contributor
Author
|
/azp run |
|
Commenter does not have sufficient privileges for PR 103 in repo sonic-net/sonic-gnmi |
sallylsy
reviewed
May 18, 2023
sallylsy
approved these changes
May 19, 2023
anand-kumar-subramanian
approved these changes
May 22, 2023
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why I did it
Subscription enhancements proposed in sonic-net/SONiC#1287 requires gNMI server to create a SAMPLE notification message by comparing YGOT objects from two successive iterations. We observed that the standard
ygot.DiffAPI takes more time and memory when the YGOT objects have multiple child objects. Hence developed an in-house diff APIHow I did it
Performance issues of
ygot.DiffAPI are mainly due to it flattening the YGOT objects into list of {leaf_path, leaf_value} pairs and then comparing those lists. The in-houseDiffAPI avoids this flattening step. It traverses both YGOT object attributes together, compares the values inline and records the changes if any. Following table captures the time taken (in nanoseconds) byygot.Diffand the in-house diff APIs for diffing two openconfig-acl objects. Size "MxN" stands for ygot object with M ACLs and N rules in each ACL. All values are in nanoseconds.How to verify it
Tested the functionality through gotests that handcraft two ygot objects, invoke the new Diff function and verify the results.
The gNMI server code to use this API is not ready as of now.
Which release branch to backport (provide reason below if selected)
Description for the changelog
Link to config_db schema for YANG module changes
A picture of a cute animal (not mandatory but encouraged)