Skip to content
This repository was archived by the owner on Sep 30, 2024. It is now read-only.

internal/database/sub_repo_permissions: modify store to be able to insert ip based permissions#63811

Merged
ggilmore merged 4 commits into
mainfrom
graphite-ggilmoreinternal_database_sub_repo_permissions_modify_store_to_be_able_to_insert_ip_based_permissions
Jul 18, 2024
Merged

internal/database/sub_repo_permissions: modify store to be able to insert ip based permissions#63811
ggilmore merged 4 commits into
mainfrom
graphite-ggilmoreinternal_database_sub_repo_permissions_modify_store_to_be_able_to_insert_ip_based_permissions

Conversation

@ggilmore

@ggilmore ggilmore commented Jul 12, 2024

Copy link
Copy Markdown
Contributor

Closes https://linear.app/sourcegraph/issue/SRC-459/
Closes

This PR adds support for saving and retreiving the IP addressess associated with each path rule in the sub_repo_permissions store.

It does this by:

Adding a new permissions type to the internal/authz package:

https://github.com/sourcegraph/sourcegraph/blob/1be7df6d79c96017b9a49c289038cf59cb4a8eb2/internal/authz/iface.go#L52-L96

Adding new *WithIPs versions of all the setter and getter methods

The new setter methods uses the above authz.SubRepoPermissionsWithIPs type that write to the appropriate ips column in the DB.

The new getter methods retrieve the ip addresses associated with each path entry. However, here there is an additional complication: It's possible for someone to call the *WithIPs getters when the ips column is still NULL (indicating that the perforce syncer hasn't been updated / ran in order to save the IP addresses from the protection table yet.

repo_id user_id version updated_at paths ips
1 1 1 2023-07-01 10:00:00 {"/depot/main/...", "/depot/dev/...", "-/depot/secret/..."} NULL
2 1 1 2023-07-01 11:00:00 {"/depot/public/...", "-/depot/private/..."} NULL

In order to address this, the getters each have a backfill boolean that allows the caller to choose the behavior that they want.

  • If backfill = true, the paths without IP entries will be returned with a * (wildcard) IP indicating that any client IP address is okay. (This is effectively the behavior we have today since we don't check IPs for sub_repo_permisisons). I imagine this can be used when callers don't care about enforcing IP-based permissions (such as when IP address enforcement is disabled in site configuration).

  • If backfill = false, if the IPs column is NULL - an error is returned instead of backfilling ("The IP addresses associated with this sub-repository-permissions entry have not been synced yet."). This allows for callers that care about IP address enforcement to know explicitly if the IP address information hasn't been updated yet - so we can't know whether or not the user is able to view the file (e.g when IP based enforcement is enabled).

Ensuring that the old setter methods set the IPs column to NULL:

self-explanatory, if someone uses the non *WithIP variants of the setters, we want to ensure that we zero out that column so that we don't leave stale / inconsistent information for those Path entries.


Overall, the design this adds the new IP address functionality without having to immediately update all the call sites in the codebase to force them to interpret all this information (which would make for a gargantuan PR). Eventually, we should be able to simply delete the old versions of the setters/getters once the IP address functioanlity has been threaded through everywhere.

Test plan

Extensive unit tests.

For each new setter and getter, I added unit tests that tested along all of the following dimenisons:

  • initial store state: empty database, database seeded with permissions with no IP information (paths column only), database seeded with permissions that have the IP information synced
  • insertion method: was the data for the test inserted with IP information (using the withIP variant of upsert, etc.), or was it inserted with the old legacy way with no ip information
  • retreieval method: was the data reterived with the legacy getters (that don't look at the IP information), with the new IP getters that either backfill (if the IP information for that paths entry hasn't been synced yet, it will return an * for that entry), or avoids backfilling (will return the information in the IPs column, or hard-error)?

Changelog

  • The sub_repository_permissions_ database store can now save and retrieve the IP addresses associated with each path rule.

@cla-bot cla-bot Bot added the cla-signed label Jul 12, 2024

ggilmore commented Jul 12, 2024

Copy link
Copy Markdown
Contributor Author

@github-actions github-actions Bot added team/product-platform team/source Tickets under the purview of Source - the one Source to graph it all labels Jul 12, 2024
@ggilmore ggilmore force-pushed the graphite-ggilmoreinternal_database_sub_repo_permissions_modify_store_to_be_able_to_insert_ip_based_permissions branch 6 times, most recently from 3ba461a to 4dbc13d Compare July 15, 2024 23:36
@ggilmore ggilmore force-pushed the graphite-ggilmoreinternal_database_sub_repo_permissions_modify_store_to_be_able_to_insert_ip_based_permissions branch from 4dbc13d to 1be7df6 Compare July 15, 2024 23:59
@ggilmore ggilmore requested a review from a team July 16, 2024 00:43
@ggilmore ggilmore marked this pull request as ready for review July 16, 2024 00:53

@eseliger eseliger left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for all the tests!

Eventually, we should be able to simply delete the old versions of the setters/getters once the IP address functioanlity has been threaded through everywhere.

once we do that, I think we should also consider renaming the methods again to drop the WithIPs suffix, since that is just the standard behavior.

Comment thread internal/database/sub_repo_perms_store.go Outdated
@ggilmore ggilmore changed the base branch from graphite-ggilmorefeature_db_sub_repo_perms_add_ips_column_to_sub_repo_perms to graphite-base/63811 July 18, 2024 19:54
@ggilmore ggilmore changed the base branch from graphite-base/63811 to main July 18, 2024 19:55
…permissions_modify_store_to_be_able_to_insert_ip_based_permissions
@ggilmore ggilmore merged commit 57de59c into main Jul 18, 2024

Copy link
Copy Markdown
Contributor Author

Merge activity

@ggilmore ggilmore deleted the graphite-ggilmoreinternal_database_sub_repo_permissions_modify_store_to_be_able_to_insert_ip_based_permissions branch July 18, 2024 21:05
ggilmore referenced this pull request Jul 30, 2024
…addresses (#64010)

The perforce permission syncer has been adapted to now read and save the HOST field from the perforce protections table, which contains the IP address(es) that the path rule that it applies to.  It uses the updated sub_repository_rules store methods introduced in https://github.com/sourcegraph/sourcegraph/pull/63811/.

### Notes

- There is some existing logic in the permissions syncer that attempts to re-insert the existing sub_repo_permissions if we encounter a temporary (timeout, etc.) error when syncing. However, there is an edge case: what do we do if the existing permissions don't have an IP address associated with them yet (they were inserted before the updated permission syncer ran)? For simplicity, in this PR I leaned towawrds correctness - I fail the operation outright (I'd rather temporarirly lock someone out rather than accidentally leak information). I implemented a more robust straetgy for this in https://github.com/sourcegraph/sourcegraph/pull/64086. 


## Test plan

- The existing unit tests have been adapted to use the new authz.SubRepoPermissionsWithIP structs (I use wildcard IP addresses). 


- The big new test to pay attention to is TestScanIPPermissions (and the associated `sample-protects-ip.txt` file). 

## Changelog

- The perforce permissions syncer has been updated to save the IP address associated with each sub_repository_permissions rule.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

cla-signed team/product-platform team/source Tickets under the purview of Source - the one Source to graph it all

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants