Skip to content

Releases: neondatabase/neon

release-6514

29 Aug 12:24
aa8c5d1
Compare
Choose a tag to compare

What's Changed

Full Changelog: release-6506...release-6514

release-6506

26 Aug 10:55
4355dba
Compare
Choose a tag to compare

What's Changed

  • proxy: Fix some warnings by extended clippy checks by @cloneable in #8748
  • l0_flush: remove support for mode page-cached by @problame in #8739
  • Migrate physical GC and scan_metadata to remote_storage by @arpad-m in #8673
  • Update aws -> infra for repo rename by @arpad-m in #8755
  • scrubber: remove _generic postfix and two unused functions by @arpad-m in #8761
  • safekeeper: check for non-consecutive writes in safekeeper.rs by @arssher in #8640
  • task_mgr::spawn: require a TenantId by @problame in #8462
  • Make Postgres 16 default version by @bayandin in #8745
  • Revert "safekeeper: check for non-consecutive writes in safekeeper.rs" by @jcsp in #8771
  • storcon: run db migrations after step down sequence by @VladLazar in #8756
  • proxy: make TLS and WS handling optional by @conradludgate in #8752
  • storcon: add peer jwt token by @VladLazar in #8764
  • pageserver: add metric pageserver_secondary_heatmap_total_size by @jcsp in #8768
  • impr(pageserver): abort on fatal I/O writer error by @skyzh in #8777
  • Benchmarking: need to checkout actions to download Neon artifacts by @Bodobolero in #8770
  • pageserver: add counter for wait time on background loop semaphore by @problame in #8769
  • storcon: handle heartbeater shutdown gracefully by @koivunej in #8767
  • test: test_timeline_ancestor_detach_errors rare allowed_error by @koivunej in #8782
  • CI(build-and-test): run regression tests on arm by @bayandin in #8552
  • remove workspace hack from libs by @conradludgate in #8780
  • Add compute_logical_snapshot_files metric by @tristan957 in #8762
  • safekeeper: add listing timelines by @arssher in #8789
  • test: fix more flaky due to graceful shutdown by @koivunej in #8787
  • Add compute_receive_lsn metric by @tristan957 in #8750
  • test: avoid some too long shutdowns by flushing before shutdown by @koivunej in #8772
  • fix(pageserver): unify initdb optimization for sparse keyspaces; fix force img generation by @skyzh in #8776
  • pageserver: batch InMemoryLayer puts, remove need to sort items by LSN during ingest by @jcsp in #8591
  • Sleep in test_scrubber_physical_gc by @arpad-m in #8798
  • Revert "Use sycnhronous commit for logical replicaiton worker (#8645)" by @ololobus in #8792
  • proxy: Enable stricter/pedantic clippy checks by @cloneable in #8775
  • test_compatibility: flush in the end by @koivunej in #8804
  • Fix race condition in LRU list update in get_cached_relsize by @knizhnik in #8807
  • feat(pageserver): warn on aux v1 tenants + default to v2 by @skyzh in #8625
  • Upgrade Postgres to 14.13, 15.8, and 16.4 by @tristan957 in #8704
  • Document how to use "git merge" for PostgreSQL minor version upgrades. by @hlinnaka in #8692
  • Require poetry >=1.8 by @MMeent in #8812
  • Update serde_json to 1.0.125 by @arpad-m in #8813
  • CI(autocomment): add arch to build type by @bayandin in #8809
  • feat(pageserver): remove features=testing restriction for compact by @skyzh in #8815
  • fix(pageserver): l0 check criteria by @skyzh in #8797
  • test: copy dict to avoid error on retry by @koivunej in #8811
  • storcon_cli: add support for drain and fill operations by @VladLazar in #8791
  • update diesel by @conradludgate in #8816
  • impr(pageserver): error if keys are unordered in merge iter by @skyzh in #8818
  • proxy: flesh out JWT verification code by @conradludgate in #8805
  • controller: use PageserverUtilization for scheduling by @jcsp in #8711
  • storage controller: enable timeline CRUD operations to run concurrently with reconciliation & make them safer by @jcsp in #8783
  • proxy: local auth backend by @conradludgate in #8806
  • proxy: new local-proxy application by @conradludgate in #8736
  • chore: improve markdown formatting by @mrl5 in #8825
  • Storage & Compute release 2024-08-26 by @vipvap in #8827

New Contributors

Full Changelog: release-6450...release-6506

release-6450

21 Aug 04:09
cdd8014
Compare
Choose a tag to compare

What's Changed

  • Adopt list_streaming in tenant deletion by @arpad-m in #8504
  • Add a test for clickhouse as a logical replication consumer by @a-masterov in #8408
  • test: deflake test_duplicate_creation by @koivunej in #8536
  • pageserver: fix return code from secondary_download_handler by @jcsp in #8508
  • pageserver: fail if id is present in pageserver.toml by @problame in #8489
  • l0_flush: use mode=direct by default => coverage in automated tests by @problame in #8534
  • scrubber: enable cleaning up garbage tenants from known deletion bugs, add object age safety check by @jcsp in #8461
  • Add metrics for input data considered and taken for compression by @arpad-m in #8522
  • scrubber: add remote_storage based listing APIs and use them in find-large-objects by @arpad-m in #8541
  • CI(benchmarking): make neonvm default provisioner by @bayandin in #8538
  • [neon/acr] impr: push to ACR while building images by @antonyc in #8545
  • feat(storcon): store scrubber metadata scan result by @yliang412 in #8480
  • feat(scrubber): post scan_metadata results to storage controller by @yliang412 in #8502
  • cleanup(compact_level0_phase1): some commentary and wrapping into block expressions by @problame in #8544
  • cicd: change Azure storage details [1/2] by @fcdm in #8553
  • compaction_level0_phase1: bypass PS PageCache for data blocks by @problame in #8543
  • CI(regress-tests): run less regression tests by @bayandin in #8561
  • feat(pageserver): support btm-gc-compaction for child branches by @skyzh in #8519
  • refactor(page_service): Timeline gate guard holding + cancellation + shutdown by @problame in #8339
  • Fix negative replication delay metric by @tristan957 in #8520
  • cicd: change Azure storage details [2/2] by @fcdm in #8562
  • controller: simplify reconciler generation increment logic by @jcsp in #8560
  • storage_scrubber: migrate FindGarbage to remote_storage by @arpad-m in #8548
  • CI(create-test-report): fix missing benchmark results in Allure report by @bayandin in #8540
  • test(pageserver): add test_gc_feedback_with_snapshots by @skyzh in #8474
  • pageserver: shutdown all walredo managers 8s into shutdown by @problame in #8572
  • Reduce linux-raw-sys duplication by @arpad-m in #8577
  • pageserver: add layer visibility calculation by @jcsp in #8511
  • storage-scrubber: log version on start by @skyzh in #8571
  • fix(pageserver): skip existing layers for btm-gc-compaction by @skyzh in #8498
  • test_runner: don't create artifacts if Allure is not enabled by @bayandin in #8580
  • pageserver: refine how we delete timelines after shard split by @jcsp in #8436
  • storage_controller: start adding chaos hooks by @jcsp in #7946
  • Wait for completion of the upload queue in flush_frozen_layer by @arpad-m in #8550
  • Improve safekeepers eviction rate limiting by @petuhovskiy in #8456
  • tests: add test_historic_storage_formats by @jcsp in #8423
  • fix(pageserver): deadlock in gc-compaction by @skyzh in #8590
  • feat(pageserver): support auto split layers based on size by @skyzh in #8574
  • safekeeper: remove unused partial_backup_enabled option by @dotdister in #8547
  • feat(pageserver): support split delta layers by @skyzh in #8599
  • Update Postgres versions to use smgrexists() instead of access() to check if Oid is used by @knizhnik in #8597
  • CI(trigger-e2e-tests): wait for promote-images job from the last commit by @bayandin in #8592
  • CI(trigger-e2e-tests): fix deadlock with Build and Test workflow by @bayandin in #8606
  • fix: allow awaiting logical size for root timelines by @koivunej in #8604
  • CI(benchmarking): set pub/sub projects for LR tests by @bayandin in #8483
  • feat(pageserver): support dry-run for gc-compaction, add statistics by @skyzh in #8557
  • fix: make Timeline::set_disk_consistent_lsn use fetch_max by @koivunej in #8311
  • feat: persistent gc blocking by @koivunej in #8600
  • pageserver: remove legacy read path by @VladLazar in #8601
  • chore: bump index part version by @koivunej in #8611
  • storage_scrubber: migrate scan_safekeeper_metadata to remote_storage by @arpad-m in #8595
  • Add package-mode=false to poetry. by @arssher in #8609
  • Add a test using Debezium as a client for the logical replication by @a-masterov in #8568
  • enable rum test by @a-masterov in #8380
  • tests: improve stability of test_storage_controller_many_tenants by @jcsp in #8607
  • pageserver: fixes for layer visibility metric by @jcsp in #8603
  • pageserver: use layer visibility when composing heatmap by @jcsp in #8616
  • pageserver: add bench_ingest by @jcsp in #7409
  • scrubber: clean up scan_metadata before prod by @yliang412 in #8565
  • storage broker: only print one line for version and build tag in init by @arpad-m in #8624
  • Lower level for timeline cancellations during gc by @arpad-m in #8626
  • Add missing colon to ArchivalConfigRequest specification by @arpad-m in #8627
  • proxy: random changes by @conradludgate in #8602
  • fix: EphemeralFiles can outlive their Timeline via enum LayerManager by @koivunej in #8229
  • fix(pageserver): dump the key when it's invalid by @skyzh in #8633
  • refactor(timeline_detach_ancestor): replace ordered reparented with a hashset by @koivunej in #8629
  • cicd: push build-tools image to ACR as well by @fcdm in #8638
  • fix: drain completed page_service connections by @koivunej in #8632
  • feat(pageserver): add direct io pageserver config by @yliang412 in #8622
  • pageserver: make bench_ingest build (but panic) on macOS by @jcsp in #8641
  • Use sycnhronous commit for logical replicaiton worker by @knizhnik in #8645
  • fix(test): do not fail test for filesystem race by @koivunej in #8643
  • fix: stop leaking BackgroundPurges by @koivunej in #8650
  • pageserver: remove paranoia double-calculation of retain_lsns by @jcsp in #8617
  • tests: don't require kafka client for regular tests by @jcsp in #8662
  • CI(build-tools): update Rust, Python, Mold by @bayandin in #8667
  • Dockerfiles: fix LegacyKeyValueFormat & JSONArgsRecommended by @bayandin in #8664
  • Run a subset of benchmarking job steps on GitHub action runners in Azure - closer to the system under test by @Bodobolero in #8651
  • Revert "proxy: update tokio-postgres to allow arbitrary config params (#8076)" by @conradludgate in #8654
  • fix(neon): disable create tablespace stmt by @skyzh in #8657
  • CI(pin-build-tools-image): fix permissions for Azure login by @bayandin in #8671
  • pageserver: don't treat NotInitialized::Stopped as unexpected by @jcsp in #8675
  • storcon: skip draining shard if it's secondary is lagging too much by @VladLazar in #8644
  • Dockerfiles: remove cachepot by @bayandin in #86...
Read more

release-6299

12 Aug 13:33
5090281
Compare
Choose a tag to compare

What's Changed

Full Changelog: release-6250...release-6299

release-proxy-6107

11 Aug 12:29
73935ea
Compare
Choose a tag to compare
Merge pull request #8647 from neondatabase/rc/proxy/2024-08-08

Proxy release 2024-08-08

release-6250

07 Aug 19:33
7b7d77c
Compare
Choose a tag to compare

What's Changed

Full Changelog: release-6248...release-6250

release-6248

05 Aug 13:11
1a36516
Compare
Choose a tag to compare

What's Changed

Full Changelog: release-6210...release-6248

release-6210

29 Jul 17:45
df07482
Compare
Choose a tag to compare

What's Changed

  • [proxy/redis] impr: use redis_auth_type to switch between auth types by @antonyc in #8428
  • fix(docs): race on monotonic rfc id by @skyzh in #8445
  • tests: more generous memory allowance in test_compaction_l0_memory by @jcsp in #8446
  • pageserver: remove absolute-order disk usage eviction by @jcsp in #8454
  • tests: make test_change_pageserver more robust by @jcsp in #8442
  • Start using remote_storage in S3 scrubber for PurgeGarbage by @arpad-m in #7932
  • fix(scrubber): more robust metadata consistency checks by @yliang412 in #8344
  • fix(docs): clearify the admin URL and token used in scrubber by @skyzh in #8441
  • refactor(pageserver) remove task_mgr for most global tasks by @problame in #8449
  • build: mark target/ and pg_install/ with CACHEDIR.TAG by @problame in #8448
  • Mark body of archival_config endpoint as required by @arpad-m in #8458
  • Change default version of Neon extensio to 1.4 by @knizhnik in #8405
  • vm-image: Expose new LFC working set size metrics by @sharnoff in #8298
  • fix(docs): remove incorrect flags for scrubber purge-garbage command by @yliang412 in #8463
  • pageserver: use identity file as node id authority and remove init command and config-override flags by @VladLazar in #7766
  • tests: make test_scrubber_physical_gc_ancestors more stable by @jcsp in #8453
  • build(deps): bump openssl from 0.10.64 to 0.10.66 in /test_runner/pg_clients/rust/tokio-postgres by @dependabot in #8464
  • pageserver: add LayerVisibilityHint by @jcsp in #8432
  • Use smgrexists() instead of access() to enforce uniqueness of generated relfilenumber by @knizhnik in #7992
  • Fix to use a tab instead of spaces by @shinyaaa in #8394
  • tests: allow-list a controller heartbeat error by @jcsp in #8471
  • temporarily disable cache saving in the registry as it is very slow by @cicdteam in #8475
  • feat(pageserver): support retain_lsn in bottommost gc-compaction by @skyzh in #8328
  • remote_storage: add list_streaming API call by @arpad-m in #8466
  • followup(#8475): also disable 'cache-to' for neon-image-arch and neon-test-extensions by @problame in #8478
  • Make async connect work on MacOS: it is necessary top call WaitLatchOrSocket before PQconnectPoll by @knizhnik in #8472
  • pageserver: simplify LayerAccessStats by @jcsp in #8431
  • Implement retry support for list_streaming by @arpad-m in #8481
  • tests: simplify code around unstable test_basebackup_with_high_slru_count by @jcsp in #8477
  • pageserver: maintain gc_info incrementally by @jcsp in #8427
  • proxy: switch to leaky bucket by @conradludgate in #8470
  • CI(build-and-test): move part of the pipeline to a reusable workflow by @bayandin in #8241
  • pageserver: faster and simpler inmem layer vec read by @VladLazar in #8469
  • pageserver: fix a compilation error by @jcsp in #8487
  • devx: nicer diff hunk headers by @koivunej in #8482
  • CI: Run ARM checks in the main pipeline by @bayandin in #8185
  • followup(#8359): pre-initialize circuitbreaker metrics by @problame in #8491
  • fix: remote timeline client shutdown trips circuit breaker by @problame in #8495
  • Temporarily remove week-end test for res-aurora from pgbench-compare benchmarking runs by @Bodobolero in #8493
  • storcon: make heartbeats restart aware by @VladLazar in #8222
  • tests: always scrub on test exit when using S3Storage by @jcsp in #8437
  • tests: adjust threshold in test_partial_evict_tenant by @jcsp in #8509
  • storcon: shutdown with clean observed state by @VladLazar in #8494
  • fix(pageserver): allow incomplete history in btm-gc-compaction by @skyzh in #8500
  • Update pgrx to v 0.11.3 by @lubennikovaav in #8515
  • Update links in markdown. by @MihaiBojin in #8501
  • Handle NotInitialized::ShuttingDown error in shard split by @arpad-m in #8506
  • Using own registry to cache layers in docker build workflows by @cicdteam in #8521
  • remote_storage: expose last_modified in listings by @jcsp in #8497
  • Update Rust to 1.80.0 by @arpad-m in #8518
  • tests: checkpoint instead of compact in test_sharding_split_compaction by @jcsp in #8473
  • .github: reduce test concurrency by @jcsp in #8444
  • pageserver: make vectored get the default read path for the pageserver by @VladLazar in #8384
  • CODEOWNERS: collapse safekeepers into storage by @jcsp in #8510
  • storcon: introduce step down primitive by @VladLazar in #8512
  • Fix sql-exporter-autoscaling for pg < 16 by @sharnoff in #8523
  • refactor(layer load API): all errors are permanent by @problame in #8527
  • raise wait_lsn timeout from 60s to 300s by @problame in #8529
  • fix(pageserver): give L0 compaction priorities over image layer creation by @skyzh in #8443
  • Persist archival information by @arpad-m in #8479
  • CI(test_runner): Upload all test artifacts if preserve_database_files is enabled by @bayandin in #7990
  • safekeeper: include application name in wal service root span by @problame in #8525
  • test_runner: fix user_property usage in benchmarks by @bayandin in #8531
  • Storage & Compute release 2024-07-29 by @vipvap in #8533

New Contributors

Full Changelog: release-6144...release-6210

release-6144

26 Jul 17:33
6026cbf
Compare
Choose a tag to compare

What's Changed

Full Changelog: release-6142...release-6144

release-6142

25 Jul 23:08
dbcfc01
Compare
Choose a tag to compare

What's Changed

Full Changelog: release-6138...release-6142