vere-v3.1

Description

vere-v3.1 is a a minor runtime release with the following highlighted features:

• Rebinding eyre HTTP cache entires.
• The runtime now automatically tries to open the Ames port on a home network using the NAT-PMP protocol. This will improve connectivity on self-hosted setups.
• Restore the ctrl+z functionality to instantly force kill the urbit process.
• Plug memory leaks related to DNS queries. This has resulted in significant uptime improvements for select Urbit Foundation infrastructure ships.
• More reliable lockfile and replay error handling.
• Miscellaneous error message improvements.

What's Changed

• jets: make a real jet for swp by @pkova in #577
• build: recent macOS updates clang by @mrdomino in #579
• Revert "build: recent macOS updates clang" by @pkova in #588
• build: bump openssl dep by @mrdomino in #586
• BLAKE3 jets by @lukechampine in #535
• Revert "Revert "build: recent macOS updates clang"" by @pkova in #598
• vere: adds --gc-early flag to pack and meld subcommands by @joemfb in #613
• http: dynamic responses for empty cache entries by @Fang- in #603
• vere: add flag to continue running after behn: queue blocked by @joemfb in #618
• vere: disable systemd and apparmor for the dbus dependency by @pkova in #582
• nix: update flake.lock to fix FHSUserEnv by @eamsden in #565
• vere: bring your own boot sequence by @Fang- in #544
• stun: corrects response decoding, adds tests by @joemfb in #623
• macos: fix lldb by @pkova in #611
• events: better error messages for partial write by @pkova in #622
• ames: add libnatpmp for automatic port forwarding by @pkova in #593
• add more helpful chop message by @jalehman in #635
• disk: fix broken chop error message by @pkova in #642
• Revert "ames: add libnatpmp for automatic port forwarding" by @pkova in #644
• Revert "disk: fix broken chop error message" by @pkova in #643
• main: restore SIGTSTP handler after replay by @pkova in #656
• disk: parameterize lmdb map size by @matthew-levan in #655
• ames: free packet on successful galaxy dns resolution by @pkova in #658
• fix nock 9 crashing pier if axis is cell by @Quodss in #661
• ames: libnatpmp for real this time by @pkova in #646
• boot: replay in subprocess by @matthew-levan in #654
• ames: resolve galaxy dns in batches; plug leak by @joemfb in #639
• vere: support .run in subprocess replay by @joemfb in #663
• vere: respect --loom in subprocess replay by @joemfb in #664
• ames: only print packet drop-count on drop by @joemfb in #665
• ames: factor out stun implementation and basic de/serialization functions by @joemfb in #662
• docs: lsp integration by @matthew-levan in #670
• nix: fix vere compilation by @h33p in #633
• Update notice about swap space URL. by @sigilante in #625
• u3: better recovery from crashes during snapshot patch application by @joemfb in #651
• gitignore MODULE.bazel and lock by @belisarius222 in #680
• bazel: change mirror for libnatpmp because it's always down by @pkova in #681
• bazel: change openssl mirror to github since openssl.org is down by @pkova in #683
• disk: update epoc.txt and vere.txt atomically by @pkova in #669
• play: use libuv for replay by @matthew-levan in #682
• Update VERSION to 3.1 by @pkova in #686
• events: even better error messages for partial write by @jalehman in #636
• stun: fix stun by @pkova in #694
• docs: add autoconf-archive to installation instructions by @matthew-levan in #692
• vere: use fcntl advisory locks on pidfile by @joemfb in #697
• Release vere-v3.1 by @pkova in #705

New Contributors

• @mrdomino made their first contribution in #579
• @Quodss made their first contribution in #661

Full Changelog: vere-v3.0...vere-v3.1

vere-v3.1-rc4

Description

During pre-release testing of vere-v3.1-rc3 we discovered that replaying in a subprocess introduced a failure case in Native Planet lockfile handling heuristics.

Vere lockfile handling is meant to prevent multiple vere processes from starting the same pier at the same time. Unfortunately this lockfile handling has historically been poor and sometimes the lockfile sticks around after a vere process exits abnormally. Native Planet was using PID heuristics to delete these lingering lockfiles, but this method was broken after #654.

This release candidate contains #697 that uses advisory locking syscalls to significantly improve lockfile reliability. The fix has also been tested to work across the Docker host -> container barrier.

What's Changed

• docs: add autoconf-archive to installation instructions by @matthew-levan in #692
• vere: use fcntl advisory locks on pidfile by @joemfb in #697

Full Changelog: vere-v3.1-rc3...vere-v3.1-rc4

vere-v3.1-rc3

Description

During pre-release testing of vere-v3.1-rc2 it was discovered that informal pinging was broken. This remained undetected because informal pinging falls back to formal pinging when malfunctioning. This release candidate includes #694 to fix the issue.

What's Changed

• stun: fix stun by @pkova in #694

Full Changelog: vere-v3.1-rc2...vere-v3.1-rc3

vere-v3.1-rc2

Description

During pre-release testing of vere-v3.1-rc1 we discovered that a PR for improving error messages when the disk is full was mistakenly not included. This release candidate includes #636 with the improved error messages.

Full Changelog: vere-v3.1-rc1...vere-v3.1-rc2

What's Changed

• events: even better error messages for partial write by @jalehman in #636

Full Changelog: vere-v3.1-rc1...vere-v3.1-rc2

vere-v3.1-rc1

Description

vere-v3.1 is a a minor runtime release with the following highlighted features:

• Rebinding eyre HTTP cache entires.
• The runtime now automatically tries to open the Ames port on a home network using the NAT-PMP protocol. This will improve connectivity on self-hosted setups.
• Restore the ctrl+z functionality to instantly force kill the urbit process.
• Plug memory leaks related to DNS queries. This has resulted in significant uptime improvements for select Urbit Foundation infrastructure ships.

What's Changed

• jets: make a real jet for swp by @pkova in #577
• build: recent macOS updates clang by @mrdomino in #579
• Revert "build: recent macOS updates clang" by @pkova in #588
• build: bump openssl dep by @mrdomino in #586
• BLAKE3 jets by @lukechampine in #535
• Revert "Revert "build: recent macOS updates clang"" by @pkova in #598
• vere: adds --gc-early flag to pack and meld subcommands by @joemfb in #613
• http: dynamic responses for empty cache entries by @Fang- in #603
• vere: add flag to continue running after behn: queue blocked by @joemfb in #618
• vere: disable systemd and apparmor for the dbus dependency by @pkova in #582
• nix: update flake.lock to fix FHSUserEnv by @eamsden in #565
• vere: bring your own boot sequence by @Fang- in #544
• Merge master back to develop after vere-v3.0 by @pkova in #621
• stun: corrects response decoding, adds tests by @joemfb in #623
• macos: fix lldb by @pkova in #611
• events: better error messages for partial write by @pkova in #622
• ames: add libnatpmp for automatic port forwarding by @pkova in #593
• add more helpful chop message by @jalehman in #635
• disk: fix broken chop error message by @pkova in #642
• Revert "ames: add libnatpmp for automatic port forwarding" by @pkova in #644
• Revert "disk: fix broken chop error message" by @pkova in #643
• main: restore SIGTSTP handler after replay by @pkova in #656
• disk: parameterize lmdb map size by @matthew-levan in #655
• ames: free packet on successful galaxy dns resolution by @pkova in #658
• fix nock 9 crashing pier if axis is cell by @Quodss in #661
• ames: libnatpmp for real this time by @pkova in #646
• boot: replay in subprocess by @matthew-levan in #654
• ames: resolve galaxy dns in batches; plug leak by @joemfb in #639
• vere: support .run in subprocess replay by @joemfb in #663
• vere: respect --loom in subprocess replay by @joemfb in #664
• ames: only print packet drop-count on drop by @joemfb in #665
• ames: factor out stun implementation and basic de/serialization functions by @joemfb in #662
• docs: lsp integration by @matthew-levan in #670
• nix: fix vere compilation by @h33p in #633
• Update notice about swap space URL. by @sigilante in #625
• u3: better recovery from crashes during snapshot patch application by @joemfb in #651
• gitignore MODULE.bazel and lock by @belisarius222 in #680
• bazel: change mirror for libnatpmp because it's always down by @pkova in #681
• bazel: change openssl mirror to github since openssl.org is down by @pkova in #683
• disk: update epoc.txt and vere.txt atomically by @pkova in #669
• play: use libuv for replay by @matthew-levan in #682
• Update VERSION to 3.1 by @pkova in #686
• Merge develop to release by @pkova in #688

New Contributors

• @mrdomino made their first contribution in #579
• @Quodss made their first contribution in #661

Full Changelog: vere-v3.0...vere-v3.1

vere-v3.0

Description

The big ticket items for this vere release include the epoch system, support for persistent nock caching, informal pings and MDNS local routing. The epoch system is functional immediately after upgrading to vere-v3.0, the other features are reliant on the 411k release slated for ~2024.3.18.

This release is not backwards compatible with vere-v2.12 and earlier. A ship cannot be downgraded after upgrading to vere-v3.0.

Epoch System

The epoch system cuts the previously uniform event log into "epochs" associated with a specific runtime version. The epochs are represented as directories in the file system. Here is an example of epochs under the Foundation galaxy ~deg that has been running the vere-v3.0 pre-release for a while:

ls ./deg/.urb/log/
0i0  0i95419173  0i98550959  0i99182407  0i99543593  data.mdb  lock.mdb

Each epoch folder is identified by an event number and includes a snapshot and a part of the event log. If the snapshot of a ship gets corrupted the ship may have to replay; the epoch system makes this process more pleasant since the ship no longer has to replay the entire event log. A replay can be initiated from the latest valid epoch, significantly speeding up the process.

The latest snapshot for a ship is situated in the latest epoch, 0i99543593 in the above case. All other epochs are safe to relocate elsewhere or even delete.

The default behavior for the epoch system is to create a new epoch every time the vere runtime version changes. A user may also manually create a new epoch by using the urbit roll command.

The urbit chop command now deletes all epochs except for the last two.

What's Changed

• jets: match hoon changes to bif:by by @ashelkovnykov in #439
• Merge develop to next/kelvin/411 by @pkova in #505
• pier: decrement arvo kelvin by @pkova in #506
• Merge master back to develop after vere-v2.12 by @pkova in #517
• Update VERSION to 3.0 by @pkova in #518
• Epoch System by @matthew-levan in #459
• u3: rewrites jam atom encoding to maximize its range by @joemfb in #363
• Fixed next/kelvin workflow to not upload to docker by @mopfel-winrux in #529
• vere: support stack traces on failing boot / full replay by @joemfb in #523
• Fix advisory types in dor/mor jets. by @sigilante in #534
• hoon: 138k by @joemfb in #526
• epoc: cleanup by @joemfb in #531
• u3: inline cell deconstruction by @joemfb in #536
• u3: declares %139 jets for +scot and +scow by @joemfb in #537
• ames: resolve galaxy ips using ipv4 only by @pkova in #543
• vere: refactors event log metadata reading to avoid the loom by @joemfb in #547
• disk: check for null pointer when writing through fak_o by @pkova in #548
• serf: adds new, stateful memory-pressure thresholds by @joemfb in #546
• UIP-0103: Persistent Nock Caching + Loom Migration Framework + Pointer Compression Migration Refactor by @matthew-levan in #508
• u3: adjust free-list sizes by @joemfb in #539
• serf: reclaim from persistent memoization cache by @joemfb in #549
• ames: handle %nail gift, lanes to vere cache by @pkova in #541
• u3: adds unconditional freelist migration as workaround by @joemfb in #551
• jets: refactors and optimizes tree-math jets by @joemfb in #397
• Fix typo in ames.c by @sigilante in #550
• u3: redeclare inline u3x_atom for linkability by @joemfb in #552
• jet +mate arm by @sidnym-ladrut in #512
• fixes +fitz jet mismatch by @mikolajpp in #444
• build: fetch urcrypt instead of vendoring it by @matthew-levan in #524
• u3: removes free-list ream from v3 migration by @joemfb in #553
• build: install autoconf-archive in docker workflow by @matthew-levan in #554
• build: fix docker workflow by @matthew-levan in #555
• u3: restore old-style, efficient reap to u3m_love() by @joemfb in #538
• u3: optimizes u3i_edit (nock opcode 10) by @joemfb in #362
• merge develop into next/kelvin/411 by @joemfb in #560
• ames: account for variable-size num fragments by @yosoyubik in #561
• lick: Shut bug by @mopfel-winrux in #563
• mdns: add mdns for local network routing by @pkova in #511
• stun: minimal working client/server by @yosoyubik in #545
• ames: clean up libuv timers by @pkova in #568
• jets: declare %138 separately by @joemfb in #567
• ames, http: declare gang correctly when scrying by @pkova in #569
• bazel: change dbus source since the official site is down by @pkova in #572
• bazel: looks like this makes the docker CI work by @pkova in #573
• Merge develop to next/kelvin/411 by @pkova in #571
• ames: independent galaxy dns resolution for stun by @yosoyubik in #574
• ames: fix galaxy domain check by @yosoyubik in #575
• Merge next/kelvin/411 to develop by @pkova in #576
• Merge develop to release for vere-v3.0-rc1 by @pkova in #578
• vere: add --serf-bin option to provide path to serf by @eamsden in #566
• Revert "vere: add --serf-bin option to provide path to serf" by @matthew-levan in #583
• mdns: distinguish fake ships in the mdns namespace by @pkova in #584
• bazel: bump zlib by @pkova in #590
• fix epoch migration bug by @matthew-levan in #592
• ames: fix ames czar prints by @yosoyubik in #595
• epoc: fix log metadata write bug, make pre-release repair more robust by @joemfb in #596
• vere: fix chop by @joemfb in #599
• vere: fix epoc lifecycle error handling, enforce invariants by @joemfb in #602
• http: do not double free scry cache path by @pkova in #606
• v3-migrate: ream before reclaiming bytecode caches by @pkova in #608
• Revert "u3: restore old-style, efficient reap to u3m_love()" by @pkova in #612
• vere: adds --gc-early flag to pack and meld subcommands by @pkova in #615
• Release vere-v3.0 by @pkova in #619

New Contributors

• @sidnym-ladrut made their first contribution in #512
• @mikolajpp made their first contribution in #444
• @yosoyubik made their first contribution in #561

Full Changelog: vere-v2.12...vere-v3.0

vere-v3.0-rc5

Description

During pre-release testing of vere-v3.0-rc4 the ship ~dinleb-rambep experienced loom corruption caused by the refcount of a jet registration noun being off by one. We were unable to reproduce the issue but decided to revert #538 out of an abundance of caution. We also added the --gc-early subcommand to recover the potentially existing ships that had been running release candidate with the issue.

What's Changed

• Revert "u3: restore old-style, efficient reap to u3m_love()" by @pkova in #612
• vere: adds --gc-early flag to pack and meld subcommands by @pkova in #615

Full Changelog: vere-v3.0-rc4...vere-v3.0-rc5

vere-v3.0-rc4

Description

During pre-release testing of vere-v3.0-rc3 a bug was discovered with the free list migration. This bug caused spurious "loom: corrupt" warnings for some ships when upgrading from vere-v2.12 to vere-v3.0. Another bug was discovered in error handling of the urbit chop command, causing problems if the ship crashed exactly during a chop.

This release candidate includes #602 and #608 to fix these bugs.

What's Changed

• vere: fix epoc lifecycle error handling, enforce invariants by @joemfb in #602
• http: do not double free scry cache path by @pkova in #606
• v3-migrate: ream before reclaiming bytecode caches by @pkova in #608

Full Changelog: vere-v3.0-rc3...vere-v3.0-rc4

vere-v3.0-rc3

Description

During pre-release testing of vere-v3.0-rc2 a bug was discovered with LMBD initialization where if your ship does not fit in a direct atom certain migration scenarios could crash. We also noticed a regression in the chop command. This release candidate fixes those issues in addition to some typos in ames czar printfs.

What's Changed

• ames: fix ames czar prints by @yosoyubik in #595
• epoc: fix log metadata write bug, make pre-release repair more robust by @joemfb in #596
• vere: fix chop by @joemfb in #599

Full Changelog: vere-v3.0-rc2...vere-v3.0-rc3

vere-v3.0-rc2

Description

During pre-release testing of vere-v3.0-rc1 a bug was discovered in situations where a ship would crash exactly when performing the migration to the epoch system. This scenario together with an attempt to downgrade back to vere-v2.12 would result in the pier being left in a broken state. #592 fixes this problem.

This release candidate also includes a fix for MDNS to distinguish between fake ships and real ships on the local network.

What's Changed

• mdns: distinguish fake ships in the mdns namespace by @pkova in #584
• bazel: bump zlib by @pkova in #590
• fix epoch migration bug by @matthew-levan in #592

Full Changelog: vere-v3.0-rc1...vere-v3.0-rc2