release-3759

What's Changed

• Release 2023-08-25 by @conradludgate in #5113

Full Changelog: release-3756...release-3759

release-3756

What's Changed

• Added deferrable transaction option to http batch queries by @jawj in #4993
• Upload test results to DB right after generation by @bayandin in #4967
• fsync directory before mark file removal by @LizardWizzard in #4986
• Allow to enable http/pg/pg tenant only auth separately in safekeeper. by @arssher in #4988
• tests: extend timeout in timeline deletion test by @jcsp in #4992
• Disable neon-pool-opt-in by @petuhovskiy in #4995
• Fix test_s3_wal_replay flakiness. by @arssher in #4470
• Fix delete_objects in UnreliableWrapper by @LizardWizzard in #5002
• ephemeral_file: remove FileExt impl (was only used by tests) by @problame in #5003
• Correctly remove orphaned objects in RemoteTimelineClient::delete_all by @LizardWizzard in #5000
• Compaction: unify key and value reference vecs by @arpad-m in #4888
• proxy: reduce global conn pool contention by @conradludgate in #4747
• page cache: newtype the blob_io and ephemeral_file file ids by @problame in #5005
• delta_layer: Restore generic from last week by @koivunej in #5014
• WIP Get ext index from spec by @lubennikovaav in #4978
• ephemeral file: refactor write_blob impl to concentrate mutable state by @problame in #5004
• test_compatibility: run amcheck unconditionally by @bayandin in #4985
• Do not attach deleted tenants by @LizardWizzard in #5008
• proxy: add more sasl logs by @conradludgate in #5012
• Increase timeout once again. by @LizardWizzard in #5021
• fix: format context by @koivunej in #5022
• test: cleanup remote_timeline_client tests by @koivunej in #5013
• proxy: pool connection logs by @conradludgate in #5020
• Make the compaction warning more tolerant by @arpad-m in #5024
• Fix safekeeper recovery with auth by @petuhovskiy in #5035
• test: allow race with flush and stopped queue by @koivunej in #5027
• simplify page-caching of EphemeralFile by @problame in #4994
• rfcs: archive my MVCC S3 Notion Proposal by @problame in #5040
• resume timeline deletions on attach by @LizardWizzard in #5030
• Avoid flakiness in test_timeline_delete_fail_before_local_delete by @LizardWizzard in #5032
• bump VM_BUILDER_VERSION to v0.16.2 by @fprasx in #5031
• refactor: failpoint support by @koivunej in #5033
• Fix DeltaLayer dumping by @arpad-m in #5045
• Resume tenant deletion on attach by @LizardWizzard in #5039
• Remove initial timeline id troubles by @koivunej in #5044
• pageserver: do not read redundant timeline_layers from IndexPart, so that we can remove it later by @jcsp in #4972
• pageserver: refactor Tenant/Timeline args into structs by @jcsp in #5053
• bump vm-builder version to v0.16.3 by @fprasx in #5055
• proxy: remove connections on shutdown by @conradludgate in #5051
• Release 2023-08-22 by @github-actions in #5060
• Enable neon-pool-opt-in by @vadim2404 in #5062

Full Changelog: release-3714...release-3756

release-3714

What's Changed

• Disable neon-pool-opt-in by @kelvich in #4996
• Fix lint by @kelvich in #4997

Full Changelog: release-3710...release-3714

release-3710

What's Changed

• pageserver: avoid logging the "ERROR" part of DbErrors that are successes by @jcsp in #4902
• Use WebIdentityTokenCredentialsProvider to access remote extensions by @lubennikovaav in #4921
• pageserver: expose prometheus metrics for startup time by @jcsp in #4893
• Add new flag for advertising pg address by @ctring in #4898
• proxy: delay auth on retry by @conradludgate in #4929
• Revert "bump vm-builder to v0.15.0-alpha1 (#4895)" by @fprasx in #4931
• Improve test results format by @bayandin in #4549
• bump vm-builder version to v0.15.0-alpha1 by @fprasx in #4934
• Explain why VirtualFile stores tenant_id and timeline_id as strings by @arpad-m in #4930
• remove_remote_layer: uninteresting refactorings by @koivunej in #4936
• Fix extensions download error handling by @lubennikovaav in #4941
• test_runner: remove excessive timeouts by @bayandin in #4659
• fix pkglibdir path for remote extensions by @lubennikovaav in #4943
• refactor: make {Delta,Image}LayerInners usable without {Delta,Image}Layer by @koivunej in #4937
• Measure compute-pageserver latency by @bojanserafimov in #4901
• if control file already exists ignore the remote version of the extension by @awestover in #4945
• test: mark test_download_extensions.py skipped for now by @koivunej in #4948
• Make the implementation of DiskBtreeReader::visit non-recursive by @arpad-m in #4884
• test: make pg_regress less flaky, hopefully by @awestover in #4903
• test: allow slow shutdown warning by @koivunej in #4953
• delete already present files from library index by @awestover in #4955
• Delete tenant's data from s3 by @LizardWizzard in #4855
• Increase max sql-over-http response size from 1MB to 10MB by @jawj in #4961
• proxy: http logging to 11 by @conradludgate in #4950
• Grant BypassRLS to new neon_superuser roles by @save-buffer in #4935
• proxy: propagate ws span by @petuhovskiy in #4966
• tests: remove redundant wait_while by @LizardWizzard in #4952
• proxy: improve http logs by @petuhovskiy in #4976
• InMemoryLayer: move end_lsn out of the lock by @arpad-m in #4963
• bump vm-builder version to v0.15.4 by @fprasx in #4980
• Add term=n option to safekeeper START_REPLICATION command. by @arssher in #4925
• Remove dead code from walproposer_utils.c by @petuhovskiy in #4525
• pageserver: avoid incrementing access time when reading layers for compaction by @jcsp in #4971
• Add neon_local to docker image. by @arssher in #4981
• Harden retries on tenant/timeline deletion path. by @LizardWizzard in #4973
• Changed batch query body to from [...] to { queries: [...] } by @jawj in #4975
• Update Postgres to v15.4 and v14.9 by @tristan957 in #4965
• Turn BlockCursor::{read_blob,read_blob_into_buf} async fn by @arpad-m in #4905
• Turn BlockLease associated type into an enum by @arpad-m in #4982
• Remote storage metrics follow-up by @koivunej in #4957
• Release 2023-08-15 by @github-actions in #4990

New Contributors

• @ctring made their first contribution in #4898

Full Changelog: release-3668...release-3710

release-3668

What's Changed

• Release: proxy retry fixup by @kelvich in #4940

Full Changelog: release-3666...release-3668

release-3666

What's Changed

• release fix: revert vm builder bump from 0.13.1 -> 0.15.0-alpha1 by @fprasx in #4932

Full Changelog: release-3665...release-3666

release-3665

What's Changed

• Deffine NEON_SMGR to make it possible for extensions to use Neon SMG API by @knizhnik in #4840
• Prepare k-merge in compaction for async I/O by @arpad-m in #4836
• pageserver: tweak period of imitate_layer_accesses by @jcsp in #4859
• test: written_size_bytes_delta by @koivunej in #4857
• refactor: tidy consumption metrics by @koivunej in #4860
• wait for tenant to be active before polling for timeline absence by @LizardWizzard in #4856
• fix(consumption): rename size => data by @koivunej in #4866
• Tar Remote Extensions by @awestover in #4715
• Bump cryptography from 41.0.2 to 41.0.3 by @bayandin in #4870
• Upgrade pg_embedding to 0.3.5 by @vadim2404 in #4873
• build: ca-certificates need to be present by @koivunej in #4880
• compute_ctl: Prewarm before starting http server by @bojanserafimov in #4867
• pageserver: suppress error logs in shutdown/detach by @jcsp in #4876
• Compaction: sort on slices directly instead of kmerge by @arpad-m in #4839
• test_runner: set AWS credentials for endpoints by @bayandin in #4887
• tests: split neon_fixtures.py by @LizardWizzard in #4871
• README: make it a bit clearer how to get regression tests running by @jcsp in #4885
• Make DiskBtreeReader::{visit, get} async by @arpad-m in #4863
• fix: shutdown logging again by @koivunej in #4886
• bump vm-builder to v0.15.0-alpha1 by @fprasx in #4895
• Make {DeltaLayer,ImageLayer}::{load,load_inner} async by @arpad-m in #4883
• proxy: update logs and span data to be consistent and have more info by @conradludgate in #4878
• pageserver: remove no-op attempt to report fail/failpoint feature by @jcsp in #4879
• Remote storage metrics by @koivunej in #4892
• fix: silence not found error when removing ephmeral by @koivunej in #4900
• Change log levels on various operations by @MMeent in #4914
• pagectl: skip metadata file in pagectl draw-timeline by @jcsp in #4872
• Sort by (key, lsn), not just key by @arpad-m in #4918
• tests: enable higher concurrency and adjust tests with outlier runtime by @jcsp in #4904
• Release 2023-08-08 by @github-actions in #4923

Full Changelog: release-3634...release-3665

release-3634

What's Changed

• Define NEON_SMGR to make it possible for extensions to use Neon SMG API by @vadim2404 in #4889

Full Changelog: release-3633...release-3634

release-3633

What's Changed

• Release 2023-08-02: update pg_embedding by @bayandin in #4877

Full Changelog: release-3632...release-3633

release-3632

What's Changed

• fix some typos by @cuishuang in #4662
• support non-interactive transactions in sql-over-http by @nicksrandall in #4654
• Use mark file to allow for deletion operations to continue through restarts by @LizardWizzard in #4552
• proxy: add tests for batch http sql by @skyzh in #4793
• proxy: record connection type by @conradludgate in #4802
• cold starts: Add sync-safekeepers fast path by @bojanserafimov in #4804
• Init Timeline in Stopping state in create_timeline_struct when Cause::Delete by @LizardWizzard in #4780
• compute_ctl: Parse sk connstring by @bojanserafimov in #4809
• Upload Test Remote Extensions by @awestover in #4792
• Bump certifi from 2022.12.7 to 2023.7.22 by @bayandin in #4815
• Fix allure report overwriting for different Postgres versions by @bayandin in #4806
• proxy: add some connection init logs by @conradludgate in #4812
• Use OnceLock instead of manually implementing it by @arpad-m in #4805
• proxy: retry compute wake in auth by @conradludgate in #4817
• dedup cleanup fs traces by @LizardWizzard in #4778
• test: allow normal warnings in test_threshold_based_eviction by @koivunej in #4801
• add list of public extensions and lookup table for libraries by @awestover in #4807
• test: allow future image layer warning by @koivunej in #4818
• fix(compute_ctl): remove stray variable in error message by @koivunej in #4823
• Move tenant/delete.rs to tenant/timeline/delete.rs by @LizardWizzard in #4825
• test: fix test_neon_cli_basics flakyness without making it better for future by @koivunej in #4827
• make download function behavior more intuitive for prefix=None by @awestover in #4795
• test_runner: add amcheck to test_compatibility by @bayandin in #4772
• test_runner: clean up test_compatibility by @bayandin in #4770
• test: raise timeout to avoid flakyness by @koivunej in #4832
• Make it possible to grant self perfmissions to self created roles by @knizhnik in #4821
• proxy: add unit test for wake_compute by @skyzh in #4819
• fix: demote warnings, fix flakyness by @koivunej in #4837
• Make DiskBtreeReader::dump async by @arpad-m in #4838
• pageserver: add unit test for deleted_at in IndexPart by @jcsp in #4844
• proxy: div by zero by @conradludgate in #4845
• fix: fix error message for listen_pg_addr_tenant_only binding by @yaoyinnan in #4787
• Update pg_embedding to 0.3.1 version by @vadim2404 in #4811
• test_compatibility: fix pg_tenant_only_port port collision by @bayandin in #4850
• Prewarm compute nodes by @bojanserafimov in #4828
• consumption_metrics: send timeline_written_size_delta by @koivunej in #4822
• support isolation level + read only for http batch sql by @skyzh in #4830
• Release 2023-08-01 by @github-actions in #4858
• ci: fix upload-postgres-extensions-to-s3 job by @problame in #4861

New Contributors

• @cuishuang made their first contribution in #4662
• @nicksrandall made their first contribution in #4654
• @yaoyinnan made their first contribution in #4787

Full Changelog: release-3592...release-3632