Proposal: Native Multi-tenancy Support by Abhishek357 · Pull Request #4055 · thanos-io/thanos

Abhishek357 · 2021-04-14T04:34:29Z

Signed-off-by: Abhishek357 abhisheksinghchauhan357@gmail.com

Signed-off-by: Abhishek357 <abhisheksinghchauhan357@gmail.com>

bwplotka

Amazing work!

And good idea to put things into GitHub - this way we can be easier aware of what's the proposal status.

Very well format proposal and I love the verbose sentences. Thanks @Abhishek357 💪🏽 Some small things to figure out still - in comments.

bwplotka · 2021-04-15T09:23:36Z

+### Goals
+
+1. Introduce a mechanism to identify tenants to support multi-tenancy use cases.
+2. Isolate all the incoming query requests.


Nice! Is query API the only goal? If yes let's change this to:

Suggested change

2. Isolate all the incoming query requests.

2. Isolate the incoming read and write API requests across tenants if needed.

If only query then let's make sure to put Metadata, Exemplars APIs etc to ### Non-Goals section

yes, for now, let's focus on query API.
other opinions are welcomed

We can make a note about store API(only the querier part) that we are focussing on.

Well it would be nice to put it in other APIs too, but we can extend that later. Just important @Abhishek357 to decide now (:

bwplotka · 2021-04-15T09:24:35Z

+
+### Solution
+
+* Introduce a new flag to enable or disable the multi-tenancy mode.


I would love to have something more organic. E.g if you specify a certain tenant label the mode will be enabled (:

Would be nice to say exactly how this flag could named, or the injection of tenant ID header/flag/parameter

Would be nice to say exactly how this flag could named

receiver already have tenant flags and we will adopt them to be consistent.

Ack, can you add note about that? And maybe provide the flag name here?

bwplotka · 2021-04-15T09:25:03Z

+
+* Introduce a new flag to enable or disable the multi-tenancy mode.
+* When this flag is enabled we will modify the metrics which can be used to track usage characteristics of a tenant by injecting the tenant’s label (a const. Label ).
+* We will make sure that the query request received has a tenant label and if not we have some default tenant label configured.


Why not blocking if tenant is not specified?

we can but we can also have a default tenant label to use when none is specified.

If we have switched on multi-tenancy mode, let's have no default label. Users know that if they have enabled multitenancy, they should provide a label for that, and not depend on a default tenant label. If they want a default behaviour, then it does not make sense to have multitenancy enabled. WDYT?

Yes, I think that would be good. But we need to ensure how we do that on Receiver too (it would be good to do so)

bwplotka · 2021-04-15T09:26:52Z

+* We will enforce the tenant labels inside the Thanos code base(i.e we assume the authentication is done and the query has access to the tenant's data which it specifies) when we run components with a specified mode(i.e multi-tenancy flag is enabled). We will no longer require any label enforcing proxy.
+* Support the cross tenant queries using regular expressions.
+* Store API will advertise the tenant label and this how it will help us -
+    * Let's assume a case of hard tenancy, we have two store-gateway advertising different sets of tenants.


Do we plan to have a special Info method field in proto. If yes, do you think we could design it here in this doc?

Alternatively, why not reusing LabelSet? (I am not suggesting we should, but it would be nice to mention why not e.g in Alternatives section below)

Is this suggestion about this proposal - https://thanos.io/tip/proposals/202101_endpoint_discovery.md/

Correct! cc @hitanshu-mehta

bwplotka · 2021-04-15T09:28:03Z

+### Solution
+
+* Introduce a new flag to enable or disable the multi-tenancy mode.
+* When this flag is enabled we will modify the metrics which can be used to track usage characteristics of a tenant by injecting the tenant’s label (a const. Label ).


How we inject?

For storing the tenant external label, we follow the current implementation of how Thanos does multitenancy, but for querying, we certainly need to define the implementation details 👍

Ack, can we add this here? (:

bwplotka · 2021-04-15T09:29:39Z

+2. We can use a prom-label proxy for label enforcement.
+    * It does not deal with cross-tenant queries. But with the proposed approach we can use regular expression for cross tenant queries.
+    * Difficult to configure and increases complexity.
+


Alternative, reuse LabelSet in InfoAPI for Store and others API tenancy info propagation

can you suggest why should we not use it?

We can, was that a plan? I think we did not talk about advertising tenant labels yet, right? I just proposed the alternative, maybe it's worth moving to main idea, let's think about it (:

kakkoyun · 2021-04-19T15:37:45Z

It'd be great if you could take a look @brancz :)

Signed-off-by: Abhishek357 <abhisheksinghchauhan357@gmail.com>

bwplotka · 2021-04-22T17:56:25Z

+
+1. Allow admins to obtain tenant information on per tenant queries, operations, and ingestion.
+2. Sending tenant-specific alerts to different alert managers.
+3. Support cross tenant views.


I think we should elaborate on this, what exactly this use case looks like in practice?

"User can query for multiple tenants" -> How you user specify those tenants?

cc @onprem

Ideally we extend those use case (more descriptive) more life examples, showing also the value of this feature

e.g We can enable running the same Querier/Store for multiple shared tenants

bwplotka · 2021-04-22T17:57:43Z

+
+### Goals
+
+1. Introduce a mechanism to identify tenants to support multi-tenancy use cases.


I think another goal is to have a way for user to specify which tenant it wants to access?

stale · 2021-06-22T14:42:22Z

Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

added multi-tenancy proposal

9c1e253

Signed-off-by: Abhishek357 <abhisheksinghchauhan357@gmail.com>

bwplotka reviewed Apr 15, 2021

View reviewed changes

kakkoyun changed the title ~~Added multi-tenancy proposal~~ Proposal: Native Multi-tenancy Support Apr 19, 2021

kakkoyun requested a review from brancz April 19, 2021 15:37

nit picks

f293b88

Signed-off-by: Abhishek357 <abhisheksinghchauhan357@gmail.com>

bwplotka reviewed Apr 22, 2021

View reviewed changes

Abhishek357 mentioned this pull request May 3, 2021

Querier: Added native multi-tenancy support #4141

Closed

2 tasks

stale Bot added the stale label Jun 22, 2021

stale Bot closed this Jun 30, 2021

douglascamata mentioned this pull request Apr 26, 2023

Proposal: query path tenancy #6320

Merged

	2. Isolate all the incoming query requests.
	2. Isolate the incoming read and write API requests across tenants if needed.


		### Solution

		* Introduce a new flag to enable or disable the multi-tenancy mode.


		### Goals

		1. Introduce a mechanism to identify tenants to support multi-tenancy use cases.

Conversation

Abhishek357 commented Apr 14, 2021

Uh oh!

bwplotka left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yashrsharma44 Apr 20, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Abhishek357 Apr 20, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kakkoyun commented Apr 19, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stale Bot commented Jun 22, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

bwplotka left a comment •

edited

Loading

yashrsharma44 Apr 20, 2021 •

edited

Loading

Abhishek357 Apr 20, 2021 •

edited

Loading