Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ICRC-3 #128

Merged
merged 46 commits into from
Apr 2, 2024
Merged
Changes from 2 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
4b322bb
add icrc-3 draft
MarioDfinity Jul 11, 2023
9b44eb0
add requirements for Values representing Transactions
MarioDfinity Jul 11, 2023
4319f1a
Update standards/ICRC-3/README.md
MarioDfinity Jul 11, 2023
8dd28f1
Update standards/ICRC-3/README.md
MarioDfinity Jul 11, 2023
570a200
fix graph
MarioDfinity Jul 11, 2023
3cdbaac
add ICRC1 schemas examples
MarioDfinity Jul 11, 2023
01252e5
ICRC-2
MarioDfinity Jul 11, 2023
ebec6c7
move schemas for ICRC-1 and ICRC-2 to ICRC-3 README.md
MarioDfinity Jul 25, 2023
29f90e8
Update standards/ICRC-3/README.md
MarioDfinity Sep 19, 2023
e50331b
Update standards/ICRC-3/README.md
MarioDfinity Sep 19, 2023
bfa5ca3
Merge
MarioDfinity Nov 8, 2023
db51936
pseudocode for hashing values
bogwar Nov 8, 2023
9f6dc65
output for hashing an array test vector
bogwar Nov 14, 2023
d6fd4ae
Incorporated changes from 14th Nov 2023 working group
MarioDfinity Nov 27, 2023
0e4b103
add "the hash of the" for Nat and Int
MarioDfinity Nov 27, 2023
25f86b7
expected_allowance switched to Nat
MarioDfinity Nov 27, 2023
1199b1f
s/u64/Nat64/
MarioDfinity Nov 27, 2023
880e549
Update standards/ICRC-3/README.md
MarioDfinity Nov 27, 2023
1d42f10
Update standards/ICRC-3/README.md
MarioDfinity Nov 27, 2023
ef995e0
missing of
MarioDfinity Nov 27, 2023
4547854
why
MarioDfinity Nov 28, 2023
968fa14
examples
MarioDfinity Nov 28, 2023
a09e60a
add get_archives
MarioDfinity Nov 28, 2023
70aa2ee
Update standards/ICRC-3/README.md
MarioDfinity Nov 28, 2023
101b90c
Update standards/ICRC-3/README.md
MarioDfinity Nov 28, 2023
dd4b51a
add icrc3_get_archives to did file
MarioDfinity Nov 28, 2023
9cc6ce1
fix did file
MarioDfinity Nov 28, 2023
362a4d5
Update standards/ICRC-3/README.md
MarioDfinity Nov 28, 2023
1c96c87
Update standards/ICRC-3/README.md
MarioDfinity Nov 29, 2023
c1cfd5f
Update standards/ICRC-3/README.md
MarioDfinity Nov 29, 2023
2d53c86
tx.op schema for ICRC standards
MarioDfinity Nov 29, 2023
be22f71
block typee and supported blocks
MarioDfinity Mar 11, 2024
ab3e9aa
add icrc3_supported_block_types to did file
MarioDfinity Mar 11, 2024
2736d79
fix ICRC-2
MarioDfinity Mar 11, 2024
8977901
fix GetArchivesResult
MarioDfinity Mar 13, 2024
3c29c17
add missing ;
MarioDfinity Mar 13, 2024
d06826e
Update README.md
MarioDfinity Mar 15, 2024
8d80565
Update README.md
MarioDfinity Mar 15, 2024
1467044
use btype instead of type
MarioDfinity Mar 18, 2024
96245c6
fallsback -> falls back
MarioDfinity Mar 18, 2024
c2531e9
better description
MarioDfinity Mar 18, 2024
b74946e
use rfc5234
MarioDfinity Mar 18, 2024
be84cad
fix
MarioDfinity Mar 18, 2024
3a2d197
1xfer for icrc1_transfer
MarioDfinity Mar 19, 2024
4871a8c
Accepted
MarioDfinity Apr 2, 2024
65ed86d
Update ICRC-3.did
MarioDfinity Apr 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
115 changes: 115 additions & 0 deletions standards/ICRC-3/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
# `ICRC-3`: Transaction Log

| Status |
|:------:|
| Draft |

`ICRC-3` is a standard for accessing the transaction log of a Ledger on the [Internet Computer](https://internetcomputer.org).

`ICRC-3` specifies:
1. A generic format for sharing the transaction log without information loss
2. A mechanism to verify the transaction log on the client side to allow downloading the transaction log via query calls
3. A way for new standards to define new transaction types compatible with ICRC-3

## Transaction Log

The transaction log is a list of transactions where each transaction contains the hash of its parent (`phash`). The parent of a transaction `i` is transaction `i-1` for `i>0` and `null` for `i=0`.

```
┌────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ │◄─────────┤phash = hash(i) │◄─────────┤phash = hash(i+1)│
├────────────────┤ ├─────────────────┤ ├─────────────────┤
│ │ │ │ │ │
│ Transaction i │ │ Transaction i+1 │ │ Transaction i+2 │
│ │ │ │ │ │
└────────────────┘ └─────────────────┘ └─────────────────┘
```

## Value

The [candid](https://github.com/dfinity/candid) format supports sharing information even when the client and the server involved do not have the same schema (see the [Upgrading and subtyping](https://github.com/dfinity/candid/blob/master/spec/Candid.md#upgrading-and-subtyping) section of the candid spec). While this mechanism allows to evolve services and clients
independently without breaking them, it also means that a client may not receive all the information that the server is sending, e.g. in case the client schema lacks some fields that the server schema has.

This loss of information is not an option for `ICRC-3`. The client must receive the same exact data the server sent. For this reason, `ICRC-3` introduces the `Value` type which never changes:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why? What problem does this solve?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's explained before.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The candid format supports sharing information even when the client and the server involved do not have the same schema (see the Upgrading and subtyping section of the candid spec). While this mechanism allows to evolve services and clients
independently without breaking them, it also means that a client may not receive all the information that the server is sending, e.g. in case the client schema lacks some fields that the server schema has.

This loss of information is not an option for ICRC-3. The client must receive the same exact data the server sent.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's the text that I commented on. It simply states that it's not acceptable, but it does not explain why.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added the following:

This loss of information is not an option for ICRC-3. The client must receive the same exact data the server sent in order to verify it. Verification is done by hashing the data and checking that the result is consistent with what has been certified by the server.

Is it better?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Why isn't it up to the client if it wants to verify the hashes? If it wants to verify the hashes, it should make sure to have an up-to-date schema. If it only wants certain data, it can use any schema that provides that data. By verifying the hash it can even make sure that its schema is up-to-date so.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The client cannot check that the schema is identical to the one of the canister. There is no guarantee that the schema will be identical even if the client first fetches it from the canister. Candid was built with a different goal in mind: the schema of the server can evolve independently from the client.
The solution to this problem is to have a type that will never change over time so it is guaranteed that clients and canisters will always know the full structure. That's what Value is.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it wants to verify the hashes, it should make sure to have an up-to-date schema.

Because it's not a sustainable solution in a decentralized system. If we went this way, releasing a new Ledger feature would immediately break block validation for all existing clients. We don't have a full list of our clients, so we cannot ask them to upgrade in advance.
The main clients of ICRC-3 standard are the on-chain and off-chain indexers, such as the index canister and Rosetta. We want these to continue working correctly with no intervention as we evolve the ledger.


```
type Value = variant {
Blob : blob;
Text : text;
Nat : nat; // do we need this or can we just use Int?
Int : int;
Array : vec Value;
Map : vec record { text; Value };
};
```

Servers must serve the transaction log as list of `Value` where each `Value` represent a single transaction in the transaction log.

## Value Hash

`ICRC-3` specifies a standard hash function over `Value`.

This hash function should be used by Ledgers to calculate the hash of the parent of a transaction and by clients to verify the downloaded transaction log.

The hash function works is the [representation-independent hashing of structured data](https://internetcomputer.org/docs/current/references/ic-interface-spec#hash-of-map) used by the IC:
MarioDfinity marked this conversation as resolved.
Show resolved Hide resolved
- the hash of a `Blob` is the hash of the bytes themselves
- the hash of a `Text` is the hash of the bytes representing the text
- the hash of a `Nat` is the [`leb128`](https://en.wikipedia.org/wiki/LEB128#Unsigned_LEB128) encoding of the number
MarioDfinity marked this conversation as resolved.
Show resolved Hide resolved
MarioDfinity marked this conversation as resolved.
Show resolved Hide resolved
- the hash of an `Int` is the [`sleb128`](https://en.wikipedia.org/wiki/LEB128#Signed_LEB128) encoding of the number
MarioDfinity marked this conversation as resolved.
Show resolved Hide resolved
- the hash of an `Array` is the hash of the concatenation of the hashes of all the elements of the array
- the hash of a `Map` is the hash of the concatenation of all the hashed items of the map sorted. An hashed item is the tuple composed by the hash of the key and the hash of the value.
MarioDfinity marked this conversation as resolved.
Show resolved Hide resolved

MarioDfinity marked this conversation as resolved.
Show resolved Hide resolved
## Standards Transactions
MarioDfinity marked this conversation as resolved.
Show resolved Hide resolved

Each standard that adheres to `ICRC-3` must define the list of transactions types they define and/or extend together with the function that converts a [`Value`](#value) to that type. Transaction types are well-typed records that are easy to consume by clients.

`Value`s representing transactions must have the following properties:
1. they must be of type `Map`
2. all transactions with index >0 must have a top-level field called `phash: Blob` which contains the [hash](#value-hash) of the previous transactions.

For instance, [`ICRC-1`](https://github.com/dfinity/ICRC-1/tree/main/standards/ICRC-1) should define three transactions types - `ICRC1_Mint`, `ICRC1_Burn` and `ICRC1_Transfer` - and the function to convert a `Value` to them in order to adhere to the `ICRC-3` standard.

## Specification

### `icrc3_get_transactions`

```
type Value = variant {
Blob : blob;
Text : text;
Nat : nat; // do we need this or can we just use Int?
Int : int;
Array : vec Value;
Map : vec record { text; Value };
};

type GetTransactionsArgs = vec record { start : Nat; length : Nat };

type Transactions = vec record { id : Nat; transaction: Value };

// A function for fetching archived transactions.
type GetArchivedTransactionsFn = func (GetTransactionsArgs) -> (Transactions) query;

type GetTransactionsResult = record {
// Total number of transactions in the
// transaction log
log-length : Nat;

// System certificate for the hash of the
// latest transaction in the chain.
// Only present if `icrc3_get_transactions`
// is called in a non-replicated query context.
certificate : opt blob;

transactions : vec record { id : Nat; transaction: Value };

archived_transactions : vec record {
args : GetTransactionsArgs;
callback : GetArchivedTransactionsFn;
};
};

service {
icrc3_get_transactions : (GetTransactionsArgs) -> (GetTransactionsResult) query;
};
```