Snapshot error order

[DanChaltiel]

Hi,

I'm sorry if that is documented somewhere, but I could not find it in ?expect_snapshot or vignette("snapshotting").

When an error occurs inside a snapshot, it seems that the whole code is interpreted even when an error is thrown, and then only the last error is shown in the build pane.

Here is a reprex (testthat v3.2.1.1):

test_that("Snapshot error order", {
  expect_snapshot({
    stop("foo")
    w = stop("bar")
    print(w)
  })
})

── Error (test-quicktest.R:6:3): Snapshot error order ──────────────────────────
Error in `eval(expr, envir, enclos)`: object 'w' not found
Backtrace:
    ▆
 1. └─testthat::expect_snapshot(...) at test-quicktest.R:6:3
 2.   └─rlang::cnd_signal(state$error)
[ FAIL 1 | WARN 0 | SKIP 0 | PASS 0 ]

In my case, w is built from another function that failed with an error, but the snapshot failed at print(w) like in my reprex.

Is this an expected behavior? Wouldn't it make more sense to stop at the first error, like in a natural R console?

EDIT:

Surprisingly, running testthat::test_file("tests/testthat/test-quicktest.R") from the console gives another output, still wrong:

── Error (test-quicktest.R:6:3): Snapshot error order ──────────────────────────
Error in `eval(expr, envir, enclos)`: bar
Backtrace:
    ▆
 1. └─testthat::expect_snapshot(...) at test-quicktest.R:6:3
 2.   └─rlang::cnd_signal(state$error)
[ FAIL 1 | WARN 0 | SKIP 0 | PASS 0 ]

Note that the row/col number is still the same (6:3)

[hadley]

For better or worse expect_snapshot() copies the behaviour as if you typed the code into the console. We rely on this in a bunch of our snapshot tests so there's no way to change it now.