Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

if results too large get each result one at a time #1118

Closed
wants to merge 13 commits into from
Closed

if results too large get each result one at a time #1118

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
c278fff
updated method so that if it gets that the file is too large we get e…
bjartek Jul 4, 2023
f75ee9a
flatten out code
bjartek Jul 11, 2023
b181049
added error message from quicknode an, it is panic: client: rpc error…
bjartek Jul 11, 2023
78eabac
cannot use tx list as it might contain transactions that are not ther…
bjartek Jul 17, 2023
ca4fe4c
use the new get results by index method from flow-go-sdk
bjartek Jul 17, 2023
9e2dba7
update to 60 mb
bjartek Jul 17, 2023
e2886fb
remove comment
bjartek Jul 17, 2023
2e669ac
Merge branch 'master' into tx-results-too-large
bjartek Jul 21, 2023
c4c3aa0
add back replace line
bjartek Jul 28, 2023
f8f263e
go mod tidy
bjartek Jul 28, 2023
8533de7
Merge branch 'master' into tx-results-too-large
bjartek Jul 28, 2023
23b3f73
updated to released version
bjartek Jul 28, 2023
62bb32f
update flow-go-sdk
bjartek Jul 28, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 17 additions & 4 deletions flowkit/flowkit.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -864,16 +864,29 @@ func (f *Flowkit) GetTransactionsByBlockID(
_ context.Context,
blockID flow.Identifier,
) ([]*flow.Transaction, []*flow.TransactionResult, error) {

tx, err := f.gateway.GetTransactionsByBlockID(blockID)
if err != nil {
return nil, nil, err
}

txRes, err := f.gateway.GetTransactionResultsByBlockID(blockID)
if err != nil {
return nil, nil, err
if err == nil {
return tx, txRes, nil
}
errorMessage := err.Error()

if strings.Contains(errorMessage, "received message larger than max") || strings.Contains(errorMessage, "trying to send message larger than max") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we have access to these errors? so we could import them by type instead of hardcoding error message?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

they look like standard GRPC error messages to me, so i doubt it.

txRes := []*flow.TransactionResult{}
for index := range tx {
txr, err := f.gateway.GetTransactionResultByIndex(blockID, uint32(index))
if err != nil {
return nil, nil, errors.Wrapf(err, "failed getting result for index %d", index)
}
txRes = append(txRes, txr)
}
return tx, txRes, nil
}
return tx, txRes, nil
return nil, nil, err
}

// BuildTransaction builds a new transaction type for later signing and submitting to the network.
Expand Down
5 changes: 5 additions & 0 deletions flowkit/gateway/emulator.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,11 @@ func (g *EmulatorGateway) GetTransactionResultsByBlockID(_ flow.Identifier) ([]*
panic("GetTransactionResultsByBlockID not implemented")
}

func (g *EmulatorGateway) GetTransactionResultByIndex(_ flow.Identifier, _ uint32) (*flow.TransactionResult, error) {
// TODO: implement
panic("GetTransactionResultsByIndex not implemented")
}

func (g *EmulatorGateway) GetTransactionsByBlockID(_ flow.Identifier) ([]*flow.Transaction, error) {
// TODO: implement
panic("GetTransactionResultsByBlockID not implemented")
Expand Down
1 change: 1 addition & 0 deletions flowkit/gateway/gateway.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ type Gateway interface {
SendSignedTransaction(*flow.Transaction) (*flow.Transaction, error)
GetTransaction(flow.Identifier) (*flow.Transaction, error)
GetTransactionResultsByBlockID(blockID flow.Identifier) ([]*flow.TransactionResult, error)
GetTransactionResultByIndex(blockID flow.Identifier, index uint32) (*flow.TransactionResult, error)
GetTransactionResult(flow.Identifier, bool) (*flow.TransactionResult, error)
GetTransactionsByBlockID(blockID flow.Identifier) ([]*flow.Transaction, error)
ExecuteScript([]byte, []cadence.Value) (cadence.Value, error)
Expand Down
6 changes: 5 additions & 1 deletion flowkit/gateway/grpc.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ import (

// maxGRPCMessageSize 20mb, matching the value set in onflow/flow-go
// https://github.com/onflow/flow-go/blob/master/utils/grpc/grpc.go#L5
const maxGRPCMessageSize = 1024 * 1024 * 20
const maxGRPCMessageSize = 1024 * 1024 * 60

// GrpcGateway is a gateway implementation that uses the Flow Access gRPC API.
type GrpcGateway struct {
Expand Down Expand Up @@ -116,6 +116,10 @@ func (g *GrpcGateway) GetTransaction(ID flow.Identifier) (*flow.Transaction, err
return g.client.GetTransaction(g.ctx, ID)
}

func (g *GrpcGateway) GetTransactionResultByIndex(blockID flow.Identifier, index uint32) (*flow.TransactionResult, error) {
return g.client.GetTransactionResultByIndex(g.ctx, blockID, index)
}

func (g *GrpcGateway) GetTransactionResultsByBlockID(blockID flow.Identifier) ([]*flow.TransactionResult, error) {
return g.client.GetTransactionResultsByBlockID(g.ctx, blockID)
}
Expand Down
3 changes: 2 additions & 1 deletion flowkit/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ require (
github.com/onflow/cadence v0.39.14
github.com/onflow/flow-emulator v0.53.0
github.com/onflow/flow-go v0.31.1-0.20230720190804-2782d45706ac
github.com/onflow/flow-go-sdk v0.41.9
github.com/onflow/flow-go-sdk v0.42.0
github.com/onflow/flow-go/crypto v0.24.9
github.com/pkg/errors v0.9.1
github.com/rs/zerolog v1.29.0
Expand Down Expand Up @@ -107,6 +107,7 @@ require (
github.com/multiformats/go-multistream v0.4.1 // indirect
github.com/multiformats/go-varint v0.0.7 // indirect
github.com/onflow/atree v0.6.0 // indirect
github.com/onflow/crypto v0.24.9 // indirect
github.com/onflow/flow-core-contracts/lib/go/contracts v1.2.4-0.20230703193002-53362441b57d // indirect
github.com/onflow/flow-core-contracts/lib/go/templates v1.2.3 // indirect
github.com/onflow/flow-ft/lib/go/contracts v0.7.0 // indirect
Expand Down
902 changes: 898 additions & 4 deletions flowkit/go.sum
View file
Open in desktop

Large diffs are not rendered by default.

3 changes: 2 additions & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ require (
github.com/onflow/flow-cli/flowkit v1.3.2-0.20230714155736-8c42ef6a9f45
github.com/onflow/flow-core-contracts/lib/go/templates v1.2.3
github.com/onflow/flow-emulator v0.53.0
github.com/onflow/flow-go-sdk v0.41.9
github.com/onflow/flow-go-sdk v0.42.0
github.com/onflowser/flowser/v2 v2.0.14-beta
github.com/pkg/errors v0.9.1
github.com/psiemens/sconfig v0.1.0
Expand Down Expand Up @@ -145,6 +145,7 @@ require (
github.com/multiformats/go-varint v0.0.7 // indirect
github.com/onflow/atree v0.6.0 // indirect
github.com/onflow/cadence-tools/lint v0.10.1 // indirect
github.com/onflow/crypto v0.24.9 // indirect
github.com/onflow/flow-archive v1.3.4-0.20230503192214-9e81e82d4dcc // indirect
github.com/onflow/flow-core-contracts/lib/go/contracts v1.2.4-0.20230703193002-53362441b57d // indirect
github.com/onflow/flow-ft/lib/go/contracts v0.7.0 // indirect
Expand Down
9 changes: 9 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -488,6 +488,8 @@ github.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFb
github.com/grpc-ecosystem/grpc-gateway/v2 v2.7.0/go.mod h1:hgWBS7lorOAVIJEQMi4ZsPv9hVvWI6+ch50m39Pf2Ks=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.11.3 h1:lLT7ZLSzGLI08vc9cpd+tYmNWjdKDqyr/2L+f6U12Fk=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.11.3/go.mod h1:o//XUCC/F+yRGJoPO/VU0GSB0f8Nhgmxx0VIRUvaC0w=
github.com/gxed/hashland/keccakpg v0.0.1/go.mod h1:kRzw3HkwxFU1mpmPP8v1WyQzwdGfmKFJ6tItnhQ67kU=
github.com/gxed/hashland/murmur3 v0.0.1/go.mod h1:KjXop02n4/ckmZSnY2+HKcLud/tcmvhST0bie/0lS48=
github.com/hashicorp/consul/api v1.3.0/go.mod h1:MmDNSzIMUjNpY/mQ398R4bk2FnqQLoPndWW5VkKPlCE=
github.com/hashicorp/consul/sdk v0.3.0/go.mod h1:VKf9jXwCTEY1QZP2MOLRhb5i/I/ssyNV1vwHyQBF0x8=
github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
Expand Down Expand Up @@ -588,6 +590,7 @@ github.com/json-iterator/go v1.1.8/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/u
github.com/json-iterator/go v1.1.9/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=
Expand Down Expand Up @@ -719,6 +722,7 @@ github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJ
github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/mr-tron/base58 v1.1.0/go.mod h1:xcD2VGqlgYjBdcBLw+TuYLr8afG+Hj8g2eTVqeSzSU8=
github.com/mr-tron/base58 v1.1.3/go.mod h1:BinMc/sQntlIE1frQmRFPUoPA1Zkr8VRgBdjWI2mNwc=
github.com/mr-tron/base58 v1.2.0 h1:T/HDJBh4ZCPbU39/+c3rRvE0uKBQlU27+QI8LJ4t64o=
Expand Down Expand Up @@ -778,6 +782,8 @@ github.com/onflow/cadence-tools/lint v0.10.1 h1:EAtLDiwtA7gnZ1grwmQkiCu487dTpH1H
github.com/onflow/cadence-tools/lint v0.10.1/go.mod h1:Rz3QhLseu6SnvZgBvyeJ0qzzz38Wxlev1lFwLiZtoPQ=
github.com/onflow/cadence-tools/test v0.10.0 h1:tcNjOFXl7ZuuvgQ3uS36ENd8l3RkSBv4toC/J302rhY=
github.com/onflow/cadence-tools/test v0.10.0/go.mod h1:eIvZOzFdi4JR2x6ekQfN8veQy04ZlZYbeeiQF/oZ0zM=
github.com/onflow/crypto v0.24.9 h1:jYP1qdwid0qCineFzBFlxBchg710A7RuSWpTqxaOdog=
github.com/onflow/crypto v0.24.9/go.mod h1:J/V7IEVaqjDajvF8K0B/SJPJDgAOP2G+LVLeb0hgmbg=
github.com/onflow/fcl-dev-wallet v0.7.2 h1:ZwhpzDakcZn9rHiIr52LwkdPh1MpUPbxA/PwCVaZ2SA=
github.com/onflow/fcl-dev-wallet v0.7.2/go.mod h1:kc42jkiuoPJmxMRFjfbRO9XvnR/3XLheaOerxVMDTiw=
github.com/onflow/flow-archive v1.3.4-0.20230503192214-9e81e82d4dcc h1:C4ZniFeOv+pHlDLJdGc/4e3NklSjVuvaXKN47980gnY=
Expand All @@ -795,6 +801,8 @@ github.com/onflow/flow-go v0.31.1-0.20230720190804-2782d45706ac/go.mod h1:x+d1uf
github.com/onflow/flow-go-sdk v0.24.0/go.mod h1:IoptMLPyFXWvyd9yYA6/4EmSeeozl6nJoIv4FaEMg74=
github.com/onflow/flow-go-sdk v0.41.9 h1:cyplhhhc0RnfOAan2t7I/7C9g1hVGDDLUhWj6ZHAkk4=
github.com/onflow/flow-go-sdk v0.41.9/go.mod h1:e9Q5TITCy7g08lkdQJxP8fAKBnBoC5FjALvUKr36j4I=
github.com/onflow/flow-go-sdk v0.42.0 h1:n5EQMYChVsRMzEDDckBZTy5wQKejeXsvatVbNC11fAU=
github.com/onflow/flow-go-sdk v0.42.0/go.mod h1:QNEJ8amKeIZZWAvo7I2Mn/o0sPQ21H1iEdox0t94anY=
github.com/onflow/flow-go/crypto v0.21.3/go.mod h1:vI6V4CY3R6c4JKBxdcRiR/AnjBfL8OSD97bJc60cLuQ=
github.com/onflow/flow-go/crypto v0.24.9 h1:0EQp+kSZYJepMIiSypfJVe7tzsPcb6UXOdOtsTCDhBs=
github.com/onflow/flow-go/crypto v0.24.9/go.mod h1:fqCzkIBBMRRkciVrvW21rECKq1oD7Q6u+bCI78lfNX0=
Expand Down Expand Up @@ -1035,6 +1043,7 @@ github.com/vmihailenco/msgpack/v4 v4.3.11 h1:Q47CePddpNGNhk4GCnAx9DDtASi2rasatE0
github.com/vmihailenco/msgpack/v4 v4.3.11/go.mod h1:gborTTJjAo/GWTqqRjrLCn9pgNN+NXzzngzBKDPIqw4=
github.com/vmihailenco/tagparser v0.1.1 h1:quXMXlA39OCbd2wAdTsGDlK9RkOk6Wuw+x37wVyIuWY=
github.com/vmihailenco/tagparser v0.1.1/go.mod h1:OeAg3pn3UbLjkWt+rN9oFYB6u/cQgqMEUPoW2WPyhdI=
github.com/whyrusleeping/go-logging v0.0.0-20170515211332-0457bb6b88fc/go.mod h1:bopw91TMyo8J3tvftk8xmU2kPmlrt4nScJQZU2hE5EM=
github.com/wsddn/go-ecdh v0.0.0-20161211032359-48726bab9208/go.mod h1:IotVbo4F+mw0EzQ08zFqg7pK3FebNXpaMsRy2RT+Ees=
github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM=
github.com/x448/float16 v0.8.4/go.mod h1:14CWIYCyZA/cWjXOioeEpHeN/83MdbZDRQHoFcYsOfg=
Expand Down
Loading