Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add store proto #277

Merged
merged 7 commits into from
Apr 18, 2024
Merged

feat: add store proto #277

merged 7 commits into from
Apr 18, 2024

Conversation

honeyAcorn
Copy link
Contributor

New Store proto for Persistent Store


message _GetRequest {
bytes key = 1;
string store_id = 2;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't need a store_id for client facing protos. We will need the store_name I think but I am not sure because in the cache get/set/delete apis we don't pass the cache_name.

@cprice404 could you comment on this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That rides in a header on cache APIs. That does not mean that is what we should do here, but that's how it works there. I'd personally prefer it to be in the proto, but if Chris has other thoughts I'm open to them.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tbh I'd prefer it to be in the proto too. But I feel like someone must have had some motivation for putting the cache name in the header originally. @kvcache ?

If @kvcache doesn't remember a compelling reason for why it was in the header for cache then I'm #TeamProto

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There were compelling reasons but they no longer exist. Putting it in the message is better in this case, and most cases.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay fff5d40

Comment on lines 33 to 34
bytes key = 1;
bytes value = 2;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following the same naming convention as our cache apis this should be:

bytes store_key = 1;
bytes store_value = 2;

But I am not sure if we want to name the value as store_value or store_body as we call it cache_body in the cache apis. @cprice404 thoughts?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

idk where body came from, value makes sense to me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think body came from cache_body in cacheclient.proto

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, what i'm saying is that I don't know how we ultimately chose body in the cache proto. value makes more sense to me. And overall I do not feel compelled to make this proto match the cache proto, I'd rather just take everything we've learned and make this one the best it can be.

@honeyAcorn
Copy link
Contributor Author

So we have some options:
store_id vs store_name or even remove it?
key vs store_key
value vs store_value or store_body

If we want to keep the naming convention as cacheclient,
Then it should be store_key and store_body, and no store_name? But from Dylan's reply I think having store_name won't hurt.

}
message Miss { }
message Hit {
bytes value = 1;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to explicitly model other data types here while we have the chance? There has been a lot of consternation about how in the cache proto we didn't model numeric values in the proto and that forced us to do weird things with them, like requiring them to be strings and then calling increment on strings. I think that if we had to do over with again we probably would have done a oneof for the values, and supported string/bytes/int/float as discrete types. cc: @kvcache

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

like this? fff5d40

Comment on lines 16 to 21
message _ValueType {
bytes bytes_value = 1;
string string_value = 2;
int64 integer_value = 3;
double double_value = 4;
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
message _ValueType {
bytes bytes_value = 1;
string string_value = 2;
int64 integer_value = 3;
double double_value = 4;
}
message _ValueType {
oneof {
bytes bytes_value = 1;
string string_value = 2;
int64 integer_value = 3;
double double_value = 4;
}
}

Comment on lines 16 to 23
message _ValueType {
oneof type {
bytes bytes_value = 1;
string string_value = 2;
int64 integer_value = 3;
double double_value = 4;
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would call this message just Value

message _Value {
  oneof value {
    bytes bytes_value = 1;
    string string_value = 2;
    int64 integer_value = 3;
    double double_value = 4;
  }
}

Or may be we can directly use oneof in the request and response like this:

message _SetRequest {
  bytes key = 1;
  oneof value {
    bytes bytes_value = 2;
    string string_value = 3;
    int64 integer_value = 4;
    double double_value = 5;
  }
}

message _GetResponse {
  oneof value {
    bytes bytes_value = 1;
    string string_value = 2;
    int64 integer_value = 3;
    double double_value = 4;
  }
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially I was separating like this, but then I thought, maybe it makes more sense to define them outside since we don't want these two to have chance to diverge. What do you think?

@honeyAcorn
Copy link
Contributor Author

honeyAcorn commented Apr 18, 2024

@cprice404 , @kvcache , @anp13 and I had a meeting to talk about this. Here's the summary:

  1. We have the store_name in header instead of proto message.
  2. We will change the bytes key to string key for more clarity
  3. We will not need hit/miss for GET cause this is not like cache, store don't have ttl concept. We will throw NOT FOUND grpc error if the key doesn't exist.
  4. We will first try to change to bytes value to oneof value where value can be of int/string/double/bytes type. Once it make sense on the sdk and on the server side, we will stick to it and support more type if needed. This would allow us to do operations to numeric types directly without having to convert from string.

Here's the change: 7b8fdf2

@kvcache
Copy link
Collaborator

kvcache commented Apr 18, 2024

@honeyAcorn
1 - we flip flopped on this. Store name belongs in the header.
2 - would you like to do that now?
3 - yep.
4 - yep.

@honeyAcorn
Copy link
Contributor Author

honeyAcorn commented Apr 18, 2024

@honeyAcorn 1 - we flip flopped on this. Store name belongs in the header. 2 - would you like to do that now? 3 - yep. 4 - yep.

@kvcache Thanks for correcting it for me! I've modified previous comment too. Now it's all good to be reviewed! It looks nice and clean now :) 7b8fdf2

Copy link
Collaborator

@kvcache kvcache left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me.

Copy link
Collaborator

@cprice404 cprice404 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shot

@honeyAcorn honeyAcorn merged commit dd58dda into main Apr 18, 2024
7 checks passed
@honeyAcorn honeyAcorn deleted the store_proto branch April 18, 2024 23:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants