Offloading stuff to the nodes

[RiccardoM]

Currently inside BDJuno we query pretty much all the data from the nodes at specific intervals, and then the fronted fetches these data using Hasura, which exposes the methods to access the underlying DB in which the data is stored.

Although this method currently works, there are some instances where it's particularly difficult to keep the data correctly updated. This is particularly true for the delegations data, which are queried only when the user wants to see their wallet details, but should be kept updated at every block (due to the staking rewards).

For this reason, I think what we could do is explore whether we can replace the periodic parsing of some data with the implementation of custom Hasura services that allow us to query the node rather than relying on the stored data. This would then look like the following:

1. The frontend queries Hasura as it has always done
2. Instead of relying on the data stored inside the database, Hasura queries the full node for those data
3. The data are returned to the frontend like any other GraphQL query

This is explained in detail here: Turn your Go REST API to GraphQL using Hasura Actions.

PRos

• Offloading some data retrieval could make it easier for the parser to get the data
• We could rely on the node for some data without worrying about constantly updating the parser to be compatible with future changes

Cons

• Some data (ie. account balance) should be queried anyway to have an history that can be used by tools such as Forbole X to display graphs
• We might have some problems with things such as filtering and sorting, as the nodes might not implement them properly. This would require the frontend to adapt and perform the filtering/sorting by itself rather then relying on Hasura.

[ryuash]

[calvinkei]

yes, that's how i do it currently: 1 api call per datapoint.
but do you mean querying the node directly for account balance at a certain height? is it possible?

[huichiaotsou]

Should we decide what parts to add in hasura action for the 0.1.0 tag release? currently it's account balances, total supply, validator commission, and delegator rewards.

[huichiaotsou]

total supply calls bankClient which kills the node. validator_commission was added by Monika, I suppose there was some issue on another chain?

[huichiaotsou]

leaving a note from campfire, please add opinions if any :)

Directly from the node, particularly user-specific data:

• account details
• account balance
• delegations

BDjuno, particularly publicly available info:

• home
• blocks
• transactions
• proposals
• params

[RiccardoM]

Note: delegations include current delegations, redelegations and unbonding delegations

[kwunyeung]

I am wondering if it's worth storing delegations data in a not that frequent scenario. If the delegations are not being stored, we will not be able to show the delegations of a validator efficiently. We will not be able to show the number of delegations on the validator list as well.

If we query the delegations directly when a user visit validator page, it generates too many queries if there are multiple users viewing a validator page. It turns out we will have to remove the whole feature.

The delegations do show some interesting relationships between accounts and the changes of bonded tokens. It helps us to understand more on the token distribution and influence of the validators.

I know this is why Cosmostation don't show any delegation and I heard of a few voices asking for showing those detail.

Could we have a solution that would not remove delegations entirely?

[RiccardoM]

The problem with keeping the delegations (and unbonding delegations and redelegations) on the database is that they need to be constantly updated. And on chains like Terra with hundreds of thousands of users this is not possible right now due to the fact that Cosmos cannot handle all those queries periodically. I think we could simply store the delegations without any information about the rewards, but I'm not very sure about this honestly. It might be more worth to simply have a load balancer in front of the chains for which we expect to have a higher traffic honestly

[kwunyeung]

Ok then let’s try first. My experience was quite not possible to query the /validator/delegations endpoint from the frontend with a responsive time and it may hang the node when there are multiple queries on delegations.

[RiccardoM]

Maybe it was due to the fact that queries were routed to Tendermint and blocked there using a mutex lock? It might be reasonable now that lock has been removed

[huichiaotsou]

• tag v0.1.0, staging version:
  Hasura actions implemented for account-related and delegation-related (current delegations, redelegations, unbonding delegations, delegation rewards) data. The previous logic/tables will still be present during the transition.

• tag v1.0.0, official release:
  Account-related and delegation-related data will be dropped from BDjuno and depend completely on Hasura actions.

Graphql staging server for testing purpose (Sifchain): https://gql-dev.sifchain.forbole.com/console/

Will prepare a graphql staging server of Desmos for @calvinkei 's deployment on Forbole X

[RiccardoM]

@huichiaotsou @MonikaCat I just realized one thing during a call with Ravi: one of the most impacting things right now are delegators and validator rewards. Did you take care of moving them to be a node-based query as well instead of a periodic operation run every X blocks?

[MonikaCat]

@RiccardoM Yes I moved it to hasura actions and created those two actions for it:

• action_delegation_reward
• action_validator_commission_amount

I also removed periodic operations part in upcoming bdjuno v1.0.0