Skip to content

[vtadmin] Add infrastructure for generating authz tests for vtadmin#10397

Merged
ajm188 merged 3 commits intovitessio:mainfrom
planetscale:andrew/vtadmin/authztests
Jun 2, 2022
Merged

[vtadmin] Add infrastructure for generating authz tests for vtadmin#10397
ajm188 merged 3 commits intovitessio:mainfrom
planetscale:andrew/vtadmin/authztests

Conversation

@ajm188
Copy link
Copy Markdown
Contributor

@ajm188 ajm188 commented May 31, 2022 •
edited
Loading

Description

The lack of verifying authz checks are where they should be is one of the
most glaring issues in vtadmin (in my opinion; it's also my "fault" things
are this way). At the same time, writing all the code by hand to verify
every single endpoint would be a giant pain (which is the main reason
things are this way). So, let's codegen all the bits we don't care about!
The bonus here is that the config.json now can serve as authoritative on
what permissions are required for what endpoints.

The goal here is to have the config primarily specify the rules needed for
each endpoint, with as minimal "overhead" (currently specifying test cases
and mock data) as possible.

I want to separate the introduction of this setup from its complete
adoption, so I will submit a follow-up change that adds the rest of the
endpoint tests.

Outstanding questions

  • What's the right spot for me to inject a make target to run go generate ./go/vt/vtadmin/... and go fmt ./go/vt/vtadmin ?

Related Issue(s)

Checklist

  • "Backport me!" label has been added if this change should be backported
  • Tests were added or are not required
  • Documentation was added or is not required

Deployment Notes

@ajm188 ajm188 added this to the v14.0 milestone May 31, 2022
@ajm188 ajm188 requested a review from deepthi May 31, 2022 13:50
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented May 31, 2022 •
edited by frouioui
Loading

Review Checklist

Hello reviewers! 👋 Please follow this checklist when reviewing this Pull Request.

General

  • Ensure that the Pull Request has the correct release notes label. release notes none should only be used for PRs that are so trivial that they need not be included.
  • If a new flag is being introduced, review whether it is really needed. The flag names should be clear and intuitive (as far as possible), and the flag's help should be descriptive.

Bug fixes

  • There should be at least one unit or end-to-end test.
  • The Pull Request description should either include a link to an issue that describes the bug OR an actual description of the bug and how to reproduce, along with a description of the fix.

Non-trivial changes

  • There should be some code comments as to why things are implemented the way they are.

New/Existing features

  • Should be documented, either by modifying the existing documentation or creating new documentation.
  • New features should have a link to a feature request issue or an RFC that documents the use cases, corner cases and test cases.

Backward compatibility

  • Protobuf changes should be wire-compatible.
  • Changes to _vt tables and RPCs need to be backward compatible.
  • vtctl command output order should be stable and awk-able.

deepthi
deepthi reviewed May 31, 2022
View reviewed changes
Copy link
Copy Markdown
Collaborator

@deepthi deepthi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great idea!

What's the right spot for me to inject a make target to run go generate ./go/vt/vtadmin/... and go fmt ./go/vt/vtadmin ?

The pattern we have followed is to create a separate (new) make target for code generation AND add a CI check to make sure generated files are up to date.

Andrew Mason added 3 commits May 31, 2022 15:29
Add infrastructure for generating authz tests for vtadmin
4c1cc86 
The lack of verifying authz checks are where they should be is one of the
most glaring issues in vtadmin (in my opinion; it's also my "fault" things
are this way). At the same time, writing all the code by hand to verify
every single endpoint would be a giant pain (which is the main reason
things are this way). So, let's codegen all the bits we don't care about!
The bonus here is that the config.json now can serve as authoritative on
what permissions are required for what endpoints.

The goal here is to have the config primarily specify the rules needed for
each endpoint, with as minimal "overhead" (currently specifying test cases
and mock data) as possible.

I want to separate the introduction of this setup from its complete
adoption, so I will submit a follow-up change that adds the rest of the
endpoint tests.

Signed-off-by: Andrew Mason <andrew@planetscale.com>
add missing license headers
657f944 
Signed-off-by: Andrew Mason <andrew@planetscale.com>
Add make target and CI check
3bc56d1 
Signed-off-by: Andrew Mason <andrew@planetscale.com>
@ajm188 ajm188 force-pushed the andrew/vtadmin/authztests branch from f5d03bb to 3bc56d1 Compare May 31, 2022 19:38
@ajm188
Copy link
Copy Markdown
Contributor Author

ajm188 commented May 31, 2022

The pattern we have followed is to create a separate (new) make target for code generation AND add a CI check to make sure generated files are up to date.

Done!!

@frouioui I will fully cop to admitting I just copied and modified the setup for make proto, can you give the CI bits a once-over just to double-check my work? It seems to work but I just want to be sure (especially) that I'm using your GitHub action correctly:

./tools/check_make_vtadmin_authz_testgen.sh
go/vt/vtadmin/api_authz_test.go
ERROR: Regenerated vtadmin_test files do not match the current version.
List of files containing differences:
0a1
>  M go/vt/vtadmin/api_authz_test.go

@ajm188 ajm188 requested a review from frouioui June 1, 2022 13:39
frouioui
frouioui approved these changes Jun 2, 2022
View reviewed changes
Copy link
Copy Markdown
Member

@frouioui frouioui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks good to me, nothing to report on the CI part! :)

@frouioui
Copy link
Copy Markdown
Member

frouioui commented Jun 2, 2022

This is a great idea!

Plus one on that 😃

@frouioui
Copy link
Copy Markdown
Member

frouioui commented Jun 2, 2022 

./tools/check_make_vtadmin_authz_testgen.sh
go/vt/vtadmin/api_authz_test.go
ERROR: Regenerated vtadmin_test files do not match the current version.
List of files containing differences:
0a1
>  M go/vt/vtadmin/api_authz_test.go

@ajm188, the above is fine in the case where we see a difference in api_authz_test.go

@ajm188
Copy link
Copy Markdown
Contributor Author

ajm188 commented Jun 2, 2022 

./tools/check_make_vtadmin_authz_testgen.sh
go/vt/vtadmin/api_authz_test.go
ERROR: Regenerated vtadmin_test files do not match the current version.
List of files containing differences:
0a1
>  M go/vt/vtadmin/api_authz_test.go

@ajm188, the above is fine in the case where we see a difference in api_authz_test.go

oh, yeah 😅 i was just demonstrating (without, ya know, actually making that clear at all) that if i change the config but don't re-gen, the check fails (as desired)

@ajm188 ajm188 merged commit dbfb9a4 into vitessio:main Jun 2, 2022
@ajm188 ajm188 deleted the andrew/vtadmin/authztests branch June 2, 2022 10:19
This was referenced Jun 2, 2022
Merged
Merged
This was referenced Jun 4, 2022
Merged
Merged
Merged
This was referenced Jun 7, 2022
Merged
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@frouioui frouioui frouioui approved these changes

@doeg doeg Awaiting requested review from doeg

@notfelineit notfelineit Awaiting requested review from notfelineit

@rohit-nayak-ps rohit-nayak-ps Awaiting requested review from rohit-nayak-ps rohit-nayak-ps is a code owner

+1 more reviewer

@deepthi deepthi deepthi left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

Component: VTAdmin VTadmin interface Type: Documentation Type: Testing

Projects

None yet

Milestone

v14.0

Development

Successfully merging this pull request may close these issues.

3 participants

@ajm188 @frouioui @deepthi