Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce KeySpace interface for API v2. #353

Open
wants to merge 46 commits into
base: master
Choose a base branch
from

Conversation

iosmanthus
Copy link
Member

@iosmanthus iosmanthus commented Jun 20, 2022

Signed-off-by: iosmanthus [email protected]

This pull request is still working in progress, and the interfaces for request encoding are stable. However, lots of the request structs need to implement with these interfaces.

The core interfaces to encode the request are:

/// Abstracts any request sent to a TiKV server.
#[async_trait]
pub trait KvRequest<C>: Request + Sized + Clone + Sync + Send + 'static {
    /// The expected response to the request.
    type Response: HasKeyErrors + HasLocks + Clone + Send + 'static;
    fn encode_request(&self, _codec: &C) -> Cow<Self>;
    fn decode_response(&self, _codec: &C, _resp: Self::Response) -> crate::Result<Self::Response>;
}

When the C is bound to RequestCodec, the requests are able to be encoded:

pub trait RequestCodec: Sized + Clone + Sync + Send + 'static {
    fn encode_key(&self, key: Vec<u8>) -> Vec<u8> { key }
    fn decode_key(&self, key: Vec<u8>) -> Result<Vec<u8>> { Ok(key) }
    fn encode_range(&self, start: Vec<u8>, end: Vec<u8>) -> (Vec<u8>, Vec<u8>) { (start, end) }
    fn encode_pd_query(&self, key: Vec<u8>) -> Vec<u8> { key }
    fn decode_region(&self, region: Region) -> Result<Region> { Ok(region) }
}

TODO:

  • Compile pass for API V1
  • Fix test for API v1
  • Impl V2 codec
  • Write tests for V2 codec.

@iosmanthus iosmanthus changed the title ·WIP]: Introduce KeySpace interface for API v2. [WIP]: Introduce KeySpace interface for API v2. Jun 20, 2022
Signed-off-by: iosmanthus <[email protected]>
@@ -204,22 +209,27 @@ pub trait PdClient: Send + Sync + 'static {
async fn update_leader(&self, ver_id: RegionVerId, leader: metapb::Peer) -> Result<()>;

async fn invalidate_region_cache(&self, ver_id: RegionVerId);

fn get_request_codec(&self) -> Self::RequestCodec;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it should return reference to Self::RequestCodec?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the only use of RequestCodec is in the construction of Dispatch, and it requires a move, I think Self::RequestCodec is more direct.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Then we can probably rename it to create_request_codec or new_request_codec which means it's actually a factory method. It's not going to be called many times, and move is the right semantic.

@@ -20,7 +23,7 @@ async fn main() {
};

// init
let client = Client::new_with_config(args.pd, config, None)
let client = Client::new_with_config(args.pd, config, TxnApiV1, None)
Copy link
Member

@sunxiaoguang sunxiaoguang Jun 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Client itself can tell we are using TxnKV already, can we hide things like TxnApiV1, TxnApiV2, RawApiV1, RawApiV2. Just let user chose the appropriate client and use ApiV1 or ApiV2.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may reexport them in their own module, like: raw::ApiV1 , txn::ApiV1

Signed-off-by: iosmanthus <[email protected]>
src/lib.rs Outdated
//! let client = RawClient::new(vec!["127.0.0.1:2379"], None).await?;
//! # use tikv_client::request::request_codec::RawApiV1;
//! futures::executor::block_on(async {
//! let client = RawClient::new(vec!["127.0.0.1:2379"], RawApiV1, None).await?;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might break the original API, we should add a default implementation for API V1.

}

#[macro_export]
macro_rules! impl_kv_request_for_scan_op {
Copy link
Member Author

@iosmanthus iosmanthus Jul 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These macro rules have the same patterns which are to take some field out and then encode or decode it. We should extract them into one rule.

@iosmanthus iosmanthus changed the title [WIP]: Introduce KeySpace interface for API v2. Introduce KeySpace interface for API v2. Jul 6, 2022
@iosmanthus iosmanthus mentioned this pull request Jul 15, 2022
3 tasks
}
}

impl KeySpaceId {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sunxiaoguang PTAL. This is the logic for constructing keyspace from the name.

@adamf
Copy link

adamf commented Sep 4, 2023

I am very excited for this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants