Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for deserialization of owned values #8

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

thomaseizinger
Copy link

Resolves #7.

There is some duplication between the visitors so there is potential for follow-up work. The feature works however and is tested with 2 additional tests I added.

Let me know what you think!

bors bot added a commit to comit-network/comit-rs that referenced this pull request Apr 7, 2020
2405: Remove dependency on `primitive_types` for Ethereum types such as Address and TransactionHash r=da-kami a=da-kami

`H256` (Hash) and `H160` (Address) are converted to our own types.
Note that for deserialization we always have to pass the Hash/Address by reference, otherwise we don't reach the right code-path in serde hex. @thomaseizinger already [tackled that problem](fspmarshall/serde-hex#8). 

Unfortunately, the `U256` [value in Ethereum is not fixed hex](https://github.com/ethereum/wiki/wiki/JSON-RPC#hex-value-encoding) (although it actually is of fixed size, namely `[u8, 32]`). 
This leads to the problem, that we would have to implement our own `Serializer` and `Deserializer` because serde-hex does not implement `Compact` for `[u8, 32]`.

Implementing a custom `Serializer` and `Deserializer` is, however, not trivial, because our hex library does not support compacted hex neither.

I started pulling in the [serialization and deserialization code from the parity codebase](https://github.com/paritytech/parity-common/blob/master/primitive-types/impls/serde/src/serialize.rs), but it is quite a lot of code to be pulled in, that is rather complicated.
Given, that this ticket was under the premise that we (most likely...) can just use serde-hex or our hex library for encoding/decoding the value, this is getting a rabbit hole. Since I already wasted quite some time on this ticket due to a tricky [serde-hex bug](fspmarshall/serde-hex#8) I would put the implementation of `U256` aside now and keep the dependency on `primitive_types` for it. 

We can decide how/if we want to deal with `U256` later. I am not sure we want to own the custom serialization/deserialization code, or if it is not better to use `primitive_types::U256`.

Co-authored-by: Daniel Karzel <[email protected]>
@gagath
Copy link

gagath commented Jul 24, 2020

Many thanks for this patch!

I have spend a lot of time trying to figure out why this toy example:

use serde::{Deserialize};
use serde_hex::{SerHex, CompactPfx};

#[derive(Debug, Deserialize)]
struct Foo {
    #[serde(with = "SerHex::<CompactPfx>")]
    bar: u32
}

const INI: &str = r#"bar = 0x42"#;

fn main() {
    let foo: Foo = serde_ini::from_str(INI).unwrap();
    println!("{:?}", foo);
}

Was providing the following result:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Custom("invalid type: string \"0x42\", expected a borrowed byte array")', src/main.rs:17:20
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Cheers!

thomaseizinger added a commit to comit-network/comit-rs that referenced this pull request Aug 18, 2020
This introduces the ethereum::Address type into our config::File
struct. Unfortunately, this opens a can of worms with a bug in
the serde-hex library. See [0] for more details.

We work around this by removing the dependency on serde-hex from our
Ethereum deserialization code and instead hand-roll everything for
our usecase.

This also has the advantage that we can now again just use our structs
directly in the route handlers of warp instead of serde_json::Value.
See [1] for more on this.

[0]: fspmarshall/serde-hex#8
[1]: #2405
thomaseizinger added a commit to comit-network/comit-rs that referenced this pull request Aug 18, 2020
This introduces the ethereum::Address type into our config::File
struct. Unfortunately, this opens a can of worms with a bug in
the serde-hex library. See [0] for more details.

We work around this by removing the dependency on serde-hex from our
Ethereum deserialization code and instead hand-roll everything for
our usecase.

This also has the advantage that we can now again just use our structs
directly in the route handlers of warp instead of serde_json::Value.
See [1] for more on this.

[0]: fspmarshall/serde-hex#8
[1]: #2405
thomaseizinger added a commit to comit-network/comit-rs that referenced this pull request Aug 18, 2020
This introduces the ethereum::Address type into our config::File
struct. Unfortunately, this opens a can of worms with a bug in
the serde-hex library. See [0] for more details.

We work around this by removing the dependency on serde-hex from our
Ethereum deserialization code and instead hand-roll everything for
our usecase.

This also has the advantage that we can now again just use our structs
directly in the route handlers of warp instead of serde_json::Value.
See [1] for more on this.

[0]: fspmarshall/serde-hex#8
[1]: #2405
thomaseizinger added a commit to comit-network/comit-rs that referenced this pull request Aug 19, 2020
This introduces the ethereum::Address type into our config::File
struct. Unfortunately, this opens a can of worms with a bug in
the serde-hex library. See [0] for more details.

We work around this by removing the dependency on serde-hex from our
Ethereum deserialization code and instead hand-roll everything for
our usecase.

This also has the advantage that we can now again just use our structs
directly in the route handlers of warp instead of serde_json::Value.
See [1] for more on this.

[0]: fspmarshall/serde-hex#8
[1]: #2405
thomaseizinger added a commit to comit-network/comit-rs that referenced this pull request Aug 20, 2020
This introduces the ethereum::Address type into our config::File
struct. Unfortunately, this opens a can of worms with a bug in
the serde-hex library. See [0] for more details.

We work around this by removing the dependency on serde-hex from our
Ethereum deserialization code and instead hand-roll everything for
our usecase.

This also has the advantage that we can now again just use our structs
directly in the route handlers of warp instead of serde_json::Value.
See [1] for more on this.

[0]: fspmarshall/serde-hex#8
[1]: #2405
thomaseizinger added a commit to comit-network/comit-rs that referenced this pull request Aug 20, 2020
This introduces the ethereum::Address type into our config::File
struct. Unfortunately, this opens a can of worms with a bug in
the serde-hex library. See [0] for more details.

We work around this by removing the dependency on serde-hex from our
Ethereum deserialization code and instead hand-roll everything for
our usecase.

This also has the advantage that we can now again just use our structs
directly in the route handlers of warp instead of serde_json::Value.
See [1] for more on this.

[0]: fspmarshall/serde-hex#8
[1]: #2405
thomaseizinger added a commit to comit-network/comit-rs that referenced this pull request Aug 20, 2020
This introduces the ethereum::Address type into our config::File
struct. Unfortunately, this opens a can of worms with a bug in
the serde-hex library. See [0] for more details.

We work around this by removing the dependency on serde-hex from our
Ethereum deserialization code and instead hand-roll everything for
our usecase.

This also has the advantage that we can now again just use our structs
directly in the route handlers of warp instead of serde_json::Value.
See [1] for more on this.

[0]: fspmarshall/serde-hex#8
[1]: #2405
thomaseizinger added a commit to comit-network/comit-rs that referenced this pull request Aug 21, 2020
This introduces the ethereum::Address type into our config::File
struct. Unfortunately, this opens a can of worms with a bug in
the serde-hex library. See [0] for more details.

We work around this by removing the dependency on serde-hex from our
Ethereum deserialization code and instead hand-roll everything for
our usecase.

This also has the advantage that we can now again just use our structs
directly in the route handlers of warp instead of serde_json::Value.
See [1] for more on this.

[0]: fspmarshall/serde-hex#8
[1]: #2405
thomaseizinger added a commit to comit-network/comit-rs that referenced this pull request Aug 21, 2020
This introduces the ethereum::Address type into our config::File
struct. Unfortunately, this opens a can of worms with a bug in
the serde-hex library. See [0] for more details.

We work around this by removing the dependency on serde-hex from our
Ethereum deserialization code and instead hand-roll everything for
our usecase.

This also has the advantage that we can now again just use our structs
directly in the route handlers of warp instead of serde_json::Value.
See [1] for more on this.

[0]: fspmarshall/serde-hex#8
[1]: #2405
bors bot added a commit to comit-network/comit-rs that referenced this pull request Aug 21, 2020
3071: Infer DAI token from Ethereum chain ID r=mergify[bot] a=thomaseizinger

We are going to use this to avoid including the token contract address in the orders that we are sending around.

This introduces the ethereum::Address type into our config::File
struct. Unfortunately, this opens a can of worms with a bug in
the serde-hex library. See [0] for more details.

We work around this by removing the dependency on serde-hex from our
Ethereum deserialization code and instead hand-roll everything for
our usecase.

This also has the advantage that we can now again just use our structs
directly in the route handlers of warp instead of serde_json::Value.
See [1] for more on this.

[0]: fspmarshall/serde-hex#8
[1]: #2405

Co-authored-by: Thomas Eizinger <[email protected]>
@nklhtv
Copy link

nklhtv commented Sep 13, 2022

Awesome! Exactly what i needed!
@fspmarshall Any chance to merge this soon?

@elpiel
Copy link
Contributor

elpiel commented Oct 11, 2023

ping @fspmarshall

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support deserialization from an owned value
4 participants