Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pb/model based testing #19913

Draft
wants to merge 51 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
8ac9629
first working prototype
paulbrauner-da Apr 3, 2024
273cb28
run against actual canton
paulbrauner-da Apr 4, 2024
4cf149f
refactoring
paulbrauner-da Apr 9, 2024
7ac30b3
rework errors
paulbrauner-da Apr 9, 2024
c5f9a4e
extract projections from canton
paulbrauner-da Apr 9, 2024
2f8fceb
add a proper projection datatype
paulbrauner-da Apr 10, 2024
1105ade
refactor ToProjection
paulbrauner-da Apr 10, 2024
23ec022
compute projections from the IDE ledger
paulbrauner-da Apr 10, 2024
fd600bd
hide the two runners behind an interface, compare projections
paulbrauner-da Apr 10, 2024
3b537fd
implementation of FEAT in scala
paulbrauner-da Apr 10, 2024
6f7fbd4
add ledger skeleton enumerations
paulbrauner-da Apr 11, 2024
3a16f4b
add a 'fixer' for ledgers
paulbrauner-da Apr 12, 2024
e2da9a4
print ledgers in scala syntax before exiting when finding a bug
paulbrauner-da Apr 16, 2024
b4dfa05
introduce skeletons
paulbrauner-da Apr 16, 2024
5310088
add symbolic solver
paulbrauner-da Apr 16, 2024
bda97a9
remove warnings
paulbrauner-da Apr 16, 2024
6fbe630
add random hash function to symbolic solver
paulbrauner-da Apr 16, 2024
5849521
split parties into participants, disjoint for now
paulbrauner-da Apr 17, 2024
27f8556
Add scenarios to skeletons
paulbrauner-da Apr 18, 2024
2f2a557
generate toplogies using z3
paulbrauner-da Apr 18, 2024
4fe3ae3
use z3 to also validate concrete ledgers
paulbrauner-da Apr 18, 2024
04c55fc
allow for parallel execution of tests
paulbrauner-da Apr 19, 2024
e959748
properly wait for party propagation
paulbrauner-da Apr 19, 2024
961e751
prepare the interpreter for party replication
paulbrauner-da Apr 19, 2024
a45b610
implement party replication
paulbrauner-da Apr 19, 2024
a25eb12
compare all projections
paulbrauner-da Apr 19, 2024
e88c457
no longer generate disjoint sets in the fixer
paulbrauner-da Apr 19, 2024
52b7077
add keys to the symbolic solver
paulbrauner-da Apr 22, 2024
b2842c8
update commented out code
paulbrauner-da Apr 22, 2024
6c8e8ab
add lookupbykey, symbolic solver only
paulbrauner-da Apr 22, 2024
1e6be5c
assert lookup by keys
paulbrauner-da Apr 23, 2024
c7d151f
add FetchByKey
paulbrauner-da Apr 23, 2024
4b130dd
simplify validity checker and add shrinking
paulbrauner-da Apr 23, 2024
8dc8840
add support for disclosure, but can't fetch disclosures in some cases
paulbrauner-da Apr 26, 2024
e7a1307
make shrinking work much better
paulbrauner-da Apr 26, 2024
a242977
fix disclosure
paulbrauner-da Apr 26, 2024
7228978
parser
paulbrauner-da Apr 28, 2024
e7f2ec5
initial support for upgrades
paulbrauner-da Apr 30, 2024
0039c7c
introduce proper support for upgrades but something's broken
paulbrauner-da Apr 30, 2024
b768512
mark dars as upgrading the previous one
paulbrauner-da May 7, 2024
e7cf445
found an upgrade bug
paulbrauner-da May 7, 2024
34fef5f
improve the hash function in the symbolic solver
paulbrauner-da May 14, 2024
54825e2
fix build after rebase
paulbrauner-da May 21, 2024
9ecb9e5
remove obsolete and wrong comment
paulbrauner-da May 22, 2024
43aa24c
fix build after rebase
paulbrauner-da May 22, 2024
a3d72e4
add package list to participant but no constraint or interpretation yet
paulbrauner-da May 22, 2024
a593069
add constraints for package lists
paulbrauner-da May 22, 2024
3b96962
make it compile again after rebase
paulbrauner-da Sep 4, 2024
08231a2
add script to run canton
paulbrauner-da Sep 13, 2024
2d9103f
Fix build after rebase.
paulbrauner-da Oct 17, 2024
f1c7112
quick fix for disclosures, needs porting
paulbrauner-da Oct 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions sdk/bazel-java-deps.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,7 @@ def install_java_deps():
"org.typelevel:paiges-core_{}:0.4.2".format(scala_major_version),
"org.wartremover:wartremover_{}:2.4.21".format(scala_version),
"org.xerial:sqlite-jdbc:3.36.0.1",
"tools.aqua:z3-turnkey:4.13.0",
maven.artifact("com.github.pureconfig", "pureconfig-macros_2.12", "0.14.0", neverlink = True),
],
fetch_sources = True,
Expand Down
1 change: 1 addition & 0 deletions sdk/canton/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,7 @@ da_scala_library(
tags = ["maven_coordinates=com.daml:canton-community-util-logging:__VERSION__"],
unused_dependency_checker_mode = "error",
visibility = [
"//daml-lf/model-test-lib:__subpackages__",
"//daml-lf/validation:__subpackages__",
"//daml-script:__subpackages__",
"//language-support:__subpackages__",
Expand Down
120 changes: 120 additions & 0 deletions sdk/daml-lf/model-test-lib/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
# Copyright (c) 2024 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

load(
"//bazel_tools:scala.bzl",
"da_scala_binary",
"da_scala_library",
"lf_scalacopts_stricter",
)
load(
"//rules_daml:daml.bzl",
"daml_compile",
"default_damlc_opts",
)
load(
"//daml-lf/language:daml-lf.bzl",
"ENGINE_LF_VERSIONS",
"mangle_for_damlc",
)

PKG_VERSIONS = [
(1, None),
(2, 1),
(3, 2),
]

[
daml_compile(
name = "universal-pkgv{pkg_version}-lf{lf_version}".format(
lf_version = mangle_for_damlc(lf_version),
pkg_version = pkg_version,
),
srcs = ["src/main/daml/Universal.daml"],
ghc_options = default_damlc_opts + [
"--ghc-option=-DPACKAGE_VERSION={}".format(pkg_version),
],
project_name = "universal",
target = lf_version,
typecheck_upgrades = True,
upgrades = "//daml-lf/model-test-lib:universal-pkgv{pkg_version}-lf{lf_version}.dar".format(
lf_version = mangle_for_damlc(lf_version),
pkg_version = upgraded,
) if upgraded != None else None,
version = "{}.0.0".format(pkg_version),
)
for lf_version in ENGINE_LF_VERSIONS
for (pkg_version, upgraded) in PKG_VERSIONS
]

da_scala_library(
name = "model-test-lib",
srcs = glob(
["src/main/**/*.scala"],
exclude = ["src/main/scala/com/digitalasset/daml/lf/model/test/Demo.scala"],
),
data = ["//canton:community_app_deploy.jar"],
scala_deps = [
"@maven//:org_scalacheck_scalacheck",
"@maven//:org_scalaz_scalaz_core",
"@maven//:org_typelevel_cats_core",
"@maven//:org_scala_lang_modules_scala_parser_combinators",
"@maven//:org_apache_pekko_pekko_stream",
"@maven//:org_typelevel_cats_kernel",
"@maven//:org_typelevel_paiges_core",
],
scalacopts = lf_scalacopts_stricter,
tags = ["maven_coordinates=com.daml:daml-lf-model-test-lib:__VERSION__"],
visibility = ["//visibility:public"],
deps = [
"//bazel_tools/runfiles:scala_runfiles",
"//canton:bindings-java",
"//canton:community_ledger_ledger-common",
"//canton:community_util-logging",
"//canton:ledger_api_proto_scala",
"//daml-lf/archive:daml_lf_archive_reader",
"//daml-lf/data",
"//daml-lf/interpreter",
"//daml-lf/language",
"//daml-lf/scenario-interpreter",
"//daml-lf/transaction",
"//daml-script/runner:script-runner-lib",
"//language-support/java/bindings-rxjava",
"//libs-scala/contextualized-logging",
"//libs-scala/rs-grpc-bridge",
"@maven//:io_grpc_grpc_netty",
"@maven//:io_reactivex_rxjava2_rxjava",
"@maven//:org_reactivestreams_reactive_streams",
"@maven//:tools_aqua_z3_turnkey",
],
)

da_scala_binary(
name = "demo",
srcs = ["src/main/scala/com/digitalasset/daml/lf/model/test/Demo.scala"],
data = [":universal-pkgv{pkg_version}-lf{lf_version}.dar".format(
lf_version = mangle_for_damlc(lf_version),
pkg_version = pkg_version,
) for lf_version in ENGINE_LF_VERSIONS for (pkg_version, _) in PKG_VERSIONS],
main_class = "com.digitalasset.daml.lf.model.test.Demo",
resources = ["//daml-script/runner:src/main/resources/logback.xml"],
scala_deps = [
"@maven//:org_scalacheck_scalacheck",
"@maven//:org_scala_lang_modules_scala_parser_combinators",
"@maven//:org_apache_pekko_pekko_actor",
"@maven//:org_apache_pekko_pekko_stream",
#"@maven//:org_typelevel_cats_core",
],
runtime_deps = [
"@maven//:ch_qos_logback_logback_classic",
],
deps = [
":model-test-lib",
"//bazel_tools/runfiles:scala_runfiles",
"//canton:community_util-logging",
"//daml-lf/data",
"//daml-lf/language",
"//libs-scala/rs-grpc-bridge",
"//libs-scala/rs-grpc-pekko",
],
)
17 changes: 17 additions & 0 deletions sdk/daml-lf/model-test-lib/bootstrap.canton
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
import com.digitalasset.canton.version.ProtocolVersion

val staticDomainParameters = StaticDomainParameters.defaults(sequencer1.config.crypto, ProtocolVersion.dev)

// Bootstrap the domain
bootstrap.domain(
domainName = "da",
sequencers = Seq(sequencer1),
mediators = Seq(mediator1),
domainOwners = Seq(sequencer1, mediator1),
domainThreshold = PositiveInt.one,
staticDomainParameters = staticDomainParameters,
)

participant1.domains.connect_local(sequencer1, alias = "da")
participant2.domains.connect_local(sequencer1, alias = "da")
participant3.domains.connect_local(sequencer1, alias = "da")
65 changes: 65 additions & 0 deletions sdk/daml-lf/model-test-lib/canton.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
canton {

features.enable-testing-commands = yes
features.enable-preview-commands = yes

sequencers {
sequencer1 {
storage.type = memory
public-api.port = 5001
admin-api.port = 5002
}
}

mediators {
mediator1 {
storage.type = memory
admin-api.port = 5202
}
}

participants {
participant1 {
storage.type = memory
admin-api.port = 5012
ledger-api.port = 5011
ledger-api.user-management-service.enabled = yes
}

participant2 {
storage.type = memory
admin-api.port = 5014
ledger-api.port = 5013
ledger-api.user-management-service.enabled = yes
}

participant3 {
storage.type = memory
admin-api.port = 5016
ledger-api.port = 5015
ledger-api.user-management-service.enabled = yes
}

}
}

# alpha version

canton.parameters {
non-standard-config = yes
alpha-version-support = yes
}

canton.participants.participant1.parameters = {
alpha-version-support = yes
}


canton.participants.participant2.parameters = {
alpha-version-support = yes
}


canton.participants.participant3.parameters = {
alpha-version-support = yes
}
9 changes: 9 additions & 0 deletions sdk/daml-lf/model-test-lib/run_canton.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#!/bin/bash
# Copyright (c) 2024 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
# SPDX-License-Identifier: Apache-2.0


DIR="$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"

cd "$DIR"/../..
bazel run canton/community_app -- daemon -c ${DIR}/canton.conf --bootstrap ${DIR}/bootstrap.canton --debug --log-file-name=/tmp/canton.log
Loading