Implement WITHATTRIBS for VSIM.#14065
Conversation
🎉 Snyk checks have passed. No issues have been found so far.✅ security/snyk check is complete. No issues have been found. (View Details) ✅ license/snyk check is complete. No issues have been found. (View Details) |
|
hi @antirez, also need to add the new argument into modules/vector-sets/commands. |
|
Thank you @sundb! Will do :) |
| results = self.redis.execute_command(*cmd_args) | ||
|
|
||
| # Results format differs depending on RESP version | ||
| resp3 = True if hasattr(self.redis, 'connection_pool') and hasattr(self.redis.connection_pool, 'response_callbacks') else False |
There was a problem hiding this comment.
Is that the proper way to test if a client is using resp3? Should we use HELLO command instead?
try:
hello_info = r.execute_command('HELLO')
# Check the 'proto' field in the response
if isinstance(hello_info, dict): # RESP3 returns a dict
print(f"Current protocol version: {hello_info.get('proto')}")
print("Connection is using RESP3")
else: # RESP2 returns an array
# Find the 'proto' value in the array
for i in range(0, len(hello_info), 2):
if hello_info[i] == b'proto':
print(f"Current protocol version: {hello_info[i+1]}")
print("Connection is using RESP2")
break
except Exception as e:
print(f"Error checking protocol: {e}")There was a problem hiding this comment.
Yes, what you suggested is better. But perhaps the Python Redis object has an explicit resp3 field actually, I need to check if it is exported.
There was a problem hiding this comment.
Even better: I believe the test should create both RESP2 and RESP3 connections at startup, so that the tests can decide what to use and test the behavior in both sides. No test will be needed, we just use what we want based on the protocol to test, in the case when (like this one) there are divergences in the behavior.
There was a problem hiding this comment.
Test updated creating a RESP3 connection and testing the behavior separately for RESP2 / RESP3.
| @@ -0,0 +1,153 @@ | |||
| from test import TestCase, generate_random_vector | |||
There was a problem hiding this comment.
Should these tests be part of CI? Or only here to show the expected behavior?
There was a problem hiding this comment.
The whole Python test of Vector Sets should be part of the CI, but its integration is pending right now. Somebody will do it in the next future apparently.
| * item, score, attribute. For RESP3 instead item -> [score, attribute] | ||
| */ | ||
| if (resp3 && withscores && withattribs) | ||
| RedisModule_ReplyWithArray(ctx,2); |
There was a problem hiding this comment.
Isn't it supposed to be a map rather than an array, based on the comment above? Also, why do it only for resp3 and not for resp2 as well?
There was a problem hiding this comment.
For RESP3, we create a map that is: map -> attribute_or_score if just one was selected, or map -> [score,attrib] if multiple will be selected. Basically the behavior will be that if just one extra information other than the element is requested, we return a simple map, otherwise we return a map mapping to an array. This is backward compatible, produces less protocol and so forth, but changes the reply format in case multiple additional items are requested. It is a tradeoff. The alternative would be that the case case would return item -> [score], a single item array, which is a bit odd.
| hnsw_release_read_slot(vset->hnsw,slot); | ||
|
|
||
| if (withscores) | ||
| if (resp3 && (withscores || withattribs)) { |
There was a problem hiding this comment.
Why testing if resp3 here? The outer map was opened either way, no?
There was a problem hiding this comment.
Because for RESP3, we always return a map, and for RESP3 we always return a flat array of item, score, attrib, ..., so the two code paths have different logic. In Redis you are not used to see this explicit check because we don't have any other data type that has multiple attributes attached to the same set-alike element. For instance with sorted sets we just have the score.
There was a problem hiding this comment.
I see. In that case, the response should start with the same logic, right? Currently we have
/* Return results */
if (withscores)
RedisModule_ReplyWithMap(ctx, REDISMODULE_POSTPONED_LEN);
else
RedisModule_ReplyWithArray(ctx, REDISMODULE_POSTPONED_LEN);
but I believe it should be:
/* Return results */
if (resp3 && (withscores || withattribs))
RedisModule_ReplyWithMap(ctx, REDISMODULE_POSTPONED_LEN);
else
RedisModule_ReplyWithArray(ctx, REDISMODULE_POSTPONED_LEN);
There was a problem hiding this comment.
@alonre24 I think you are right, thanks, probably it was impossible to catch this with tests, since I believe that the functions were refactored at some point into moduleReplyWithCollection(), so now basically if the reply is postponed, calling one or the other is exactly the same, and later the calls that set the length are the correct ones, so it works regardless. Verifying and pushing the fix.
There was a problem hiding this comment.
@alonre24 fix pushed. Indeed right now RedisModule_ReplyWithMap() or RedisModule_ReplyWithArray() are equivalent if the reply is postponed, so everything worked but was conceptually wrong anyway. Fixed and refactored to make it more explicit.
There was a problem hiding this comment.
I see, it makes sense.
I Approved the PR
| { | ||
| "token": "WITHATTRIBS", | ||
| "name": "withattribs", | ||
| "type": "pure-token", | ||
| "optional": true | ||
| } |
There was a problem hiding this comment.
hi @antirez does it belong to VSIM not VLINKS?
Hi, as described, this implements WITHATTRIBS, a feature requested by a few users, and indeed needed. This was requested the first time by @rowantrollope but I was not sure how to make it work with RESP2 and RESP3 in a clean way, hopefully that's it. The patch includes tests and documentation updates.
Hi, as described, this implements WITHATTRIBS, a feature requested by a few users, and indeed needed. This was requested the first time by @rowantrollope but I was not sure how to make it work with RESP2 and RESP3 in a clean way, hopefully that's it. The patch includes tests and documentation updates.
|
@redis/client-developers fyi. We need to add this to the client libraries. |
# Description The `WITHATTRIBS` token was incorrectly documented under the `vlinks` command in #14065
Hi, as described, this implements WITHATTRIBS, a feature requested by a few users, and indeed needed.
This was requested the first time by @rowantrollope but I was not sure how to make it work with RESP2 and RESP3 in a clean way, hopefully that's it.
The patch includes tests and documentation updates.