Release v2.2.1

Release v2.2.1 is a minor bug fix release.

What's Changed

• Fix not found request for forward response by @lzydmxy in #370
• Fix clang 20 build error: no-switch-default-label @JackyWoo #374

Full Changelog: v2.2.0...v2.2.1

Release v2.2.0

What's Changed

v2.2.0 is a major version in which we enhance the stablility on raft log and provide a convenient method to profile memory by jemalloc.

Feature

• Add ability to enable, disable, dump jemalloc profiling and flush unused memory by @JackyWoo in #354

Performance

• Use shared mutex for fowrard_response_callback by @JackyWoo in #343

Bug Fix

• Fix core dump when request forwarder receiving data by @JackyWoo in #335
• Add min parallel to four by @JackyWoo in #345
• Fix logical error: last_segment not exist by @JackyWoo in #347
• Fix unexpected max latency in output of stat by @lzydmxy in #349

Enhancement

• Add ability to turn OFF -Werror by @azat in #336
• Fail fast for log error by @JackyWoo in #340
• Use local time instead of UTC for snapshot file by @lzydmxy in #338
• Optimize memory usage by removing term and length info for a log by @JackyWoo in #342
• Support Dwarf 5 by @lzydmxy in #356
• Add configuration max_log_segment_file_size and fix illegal open segment detection by @JackyWoo in #366
• Store snapshots objects count in snapshot_obj_1 by @lzydmxy in #367

Code Refactor

• Remove raftkeeper 1.0 only code by @JackyWoo in #352
• Use llvm project by @lzydmxy in #361
• Add raftkeeper robot to generate test report by @JackyWoo in #357
• Skip report test result if event is not pull request by @JackyWoo in #360
• Fix duplicate test report in PR by @JackyWoo in #362

Others

• Add benchmark data by @JackyWoo in #337
• Start version v2.2.0 by @JackyWoo in #339

New Commiters

• @lzydmxy became RaftKeeper commiter for the excellent work on the read-request performance optimization and snapshot optimization. Welcome!

New Contributors

• @azat made first contribution in #336. Welcome!

Full Changelog: v2.1.1...v2.2.0

v2.1.1

RaftKeeper v2.1.1 is a minor bug fix release.

What's Changed

• Update build type to RelWithDebInfo in build.sh by @JackyWoo in #325
• Refactor the Docker file for integration testing by @lzydmxy in #331
• Don't send watch reponse if we got error in muiti writes ops by @lzydmxy in #327
• Fix replay session_request log error by @lzydmxy in #333
• Clear sessions_and_watchers after watch response by @lzydmxy in #329

Full Changelog: v2.1.0...v2.1.1

Release v2.1.0

We are excited to announce RaftKeeper version 2.1.0 has been officially released.

This is a major release following version 2.0.0. In this version, the performance of RaftKeeper has been further enhanced, with create requests seeing an 11% improvement and mixed requests experiencing a significant 118% increase.

You can refer to this article RaftKeeper v2.1.0 Released: Significant Performance Improvements!

1. TPS(create-100% )
   

2. TPS(create-1% set-8% get-45% list-45% delete-1%)
   

Benchmark details see benchmark.

In 2.1.0 we also provide asynchronously creating snapshot which will exreamly reduce user request blocking time.

What's Changed

Update notion

In v2.1.0 we dropped the old snapshot version logic, so

• If update from v2.0.4, feel free to update
• If update from version lower than v2.0.4, please update to v2.0.4 first, then create snapshot by echo csnp | nc node port for every nodes, then update to v2.1.0

Feature

• Add metric is_leader by @JackyWoo in #220
• Add more metrics for mntr command by @lzydmxy in #230
• Implement asynchronous snapshot by copying dataTree by @lzydmxy in #247
• Add metric snap_blocking_time_ms by @JackyWoo in #254

Performance

• Make snapshot load faster by @lzydmxy in #222
• Speed up list by using compacted strings by @JackyWoo in #297
• Remove the use of thread pools for parallel processing of read request by @lzydmxy in #235
• Remove unnecessary locks in keeperStore by @lzydmxy in #238
• Speed up data tree copying with SIMD and prefetching. by @JackyWoo in #252
• Fix high CPU usage caused by last committed log index manager. by @JackyWoo in #253
• Reserve childs vector size for simpleListResponse by @lzydmxy in #265
• Add configuration socket_option_no_delay by @JackyWoo in #281
• Opimize too many getsockname and getsockopt system calls by @JackyWoo in #283
• Fix endless loop in last commmitted index thread by @JackyWoo in #275
• Use atomic instead of lock for connections statistics by @lzydmxy in #289
• Avoid ZooKeeperResponsePtr destruction in responses_queue by @lzydmxy in #292
• Handle response serialization in the IO thread by @lzydmxy in #250

Bug Fix

• NIO framework should support macos by @JackyWoo in #214
• Fix built package name by @JackyWoo in #215
• Fix flaky test test_four_word_command by @JackyWoo in #226
• Send FIN to 4lw client, wait 4lw client to close connection by @lzydmxy in #223
• Fix loading snapshots bug when set log level higher than INFO by @lzydmxy in #236
• Release memory when loading snapshot completes by @JackyWoo in #242
• Fix extremely large max latency for error session request by @JackyWoo in #244
• Use read-write-lock for user_response_callbacks by @lzydmxy in #268
• Fix data race by using write-lock for session requests callback by @JackyWoo in #277
• Fix Apple silicon MacOS build error: 32bit platforms are not supported by @JackyWoo in #304
• Fix creating snapshot too frequently caused by int32 overflow by @JackyWoo in #307
• Fix stale log index when there is an snapshot but no log in disk by @JackyWoo in #311
• Fix snapshot never clear by @lzydmxy in #323
• Save term info in snapshot filename by @lzydmxy in #296

Enhancement

• Optimize committed log index manager by @JackyWoo in #216
• Add client_req_timeout parameter to Raft configuration by @lzydmxy in #246
• Removed submodule zlib-ng by @JackyWoo in #257
• Refactor cmake scripts by @JackyWoo in #258
• Support build with clang 16 and newer by @JackyWoo in #269
• Reduce a series of default_timeout_ms for RaftKeeper by @lzydmxy in #272
• Fix too many heart beat logs by @JackyWoo in #286
• Add system nodes for keeper by @lzydmxy in #301
• Disable build with GCC by @JackyWoo in #271
• Use cpp standard 23 by @JackyWoo in #274

Code Refactor

• Fix too many warning log when restarting by @JackyWoo in #218
• Remove lib protobuf by @JackyWoo in #228
• Delete proto from git ignore file by @JackyWoo in #229
• Remove libs re2 and fast_float by @JackyWoo in #232
• Add session and watch manager by @JackyWoo in #262

Document

• Update benchmark result by @JackyWoo in #324
• Adding how to deploy learner mode to how-to-deploy document by @JackyWoo in #260

Others

• Starting v2.0.5 by @JackyWoo in #212
• Add new issue template performance by @JackyWoo in #284
• Remove benchmark tool for we split it as a new project by @JackyWoo in #316

Full Changelog: v2.0.4...v2.1.0

Release v2.0.4

RaftKeeper v2.0.4 contains some bug fixes, code refactoring and enhancements. Also we add some documents here.

Please note that we optimized the log and snapshot file format, but it is backforward compitable.

What's Changed

Bug

• Handle eof when read magic for snapshot v0 by @lzydmxy #146
• Fix IO thread name by @JackyWoo #151
• Print timezone for built time in command mntr by @JackyWoo #154
• Fix duplicated error log in forwarder by @JackyWoo #158
• Fix ehpemeral nodes mutex by @JackyWoo in #163
• Fix wait time unit in request queue by @JackyWoo in #165
• Enable function pthread_setname_np by @JackyWoo in #173
• Sending session response in IO thread by @lzydmxy in #177
• Make log append FSYNC_BATCH work by @lzydmxy in #195

Enhancement

• Improve the NIO frame work by @JackyWoo #135
• Update nuraft version by @JackyWoo #138
• Forward session request with forwarder by @JackyWoo #147
• Enable creating RaftKeeper snapshot logs for Converter by @JackyWoo #152
• Upgrade Zookeeper client version to 3.7.1 for benchmark tool by @JackyWoo #155
• Use unordered_map to store Watches by @lzydmxy in #167
• Use concurrent queue in accumulator by @JackyWoo in #166
• Add build a option to support build mode by @JackyWoo in #186
• Waiting for zk connection to be established in benchmark by @lzydmxy in #191
• Remove third party lib protobuf from tests by @JackyWoo in #193
• Replace protobuf snapshot batch by @JackyWoo in #196
• Avoid memory copy when parsing log by @JackyWoo in #202
• Optimize lock for acl map by @JackyWoo in #204
• Update slow request threshold by @JackyWoo in #205
• Forwarding error may cause request not found in request processor pending queue by @JackyWoo in #208

Documentation

• Update readme by @JackyWoo in #179
• Add some docs by @JackyWoo in #183
• Add doc how-to-monitor-and-manage by @JackyWoo in #188
• Update documents by @JackyWoo in #189

Code refactor

• Code refactoring to make it more readable @JackyWoo #157
• Move NIO to source root directory by @JackyWoo in #178
• Fix typo ReadBufferFromNuRaftBuffer by @JackyWoo in #194
• Fail fast when there are something wrong with log and snapshot by @JackyWoo in #200
• Fix forward request tostring by @JackyWoo in #207
• Remove service types.h by @JackyWoo in #209

Full Changelog: v2.0.3...v2.0.4

Release v2.0.3

RaftKeeper v2.0.3 is a bug fix release and contains 2 features.

What's Changed

Feature

• Support multi-read requests by @lzydmxy in #94
• Support FilteredList request by @lzydmxy in #102

Enhancement

• Update benchmark docs by @JackyWoo in #98

Bug

• Fix request processor endless loop by @JackyWoo in #90
• Fix data race by @JackyWoo in #92
• Fix data race error in request processor by @JackyWoo in #96
• Reset state machine when apply snapshot from leader by @JackyWoo in #100
• Fix test_four_word_command occasional failure by @JackyWoo in #105
• Fix too much warning log by @JackyWoo in #109
• reorder raft server shutdown by @lzydmxy in #111
• Fix shutdown_timeout time unit by @JackyWoo in #112
• add cluster_config_lock_ to avoid data race by @lzydmxy in #106
• Fix cmake generation warning by @JackyWoo in #115
• Fix epoll writable event leak by @JackyWoo in #116
• Enable manually creating snapshot by default by @JackyWoo in #122
• Adjust raft timeout by @JackyWoo in #125
• Fix illegal reconnection by @JackyWoo in #126
• Set default dead_session_check_period_ms to 500 by @JackyWoo in #128
• Fix watch not triggered for muilt request by @lzydmxy in #130
• Fix logs by @JackyWoo in #132
• Fix request hunging in some case by @JackyWoo in #131
• Fix already registered after forward expiration by @lzydmxy in #118

Full Changelog: v2.0.2...v2.0.3

New Contributors

• Welcome @lzydmxy, he made his first contribution in #94.

Release v2.0.2

RaftKeeper v2.0.2 contains 2 features and some enhancements in stability.

What's Changed

Feature

• Set lower bound and upper bound for client session timeout by @Hooollin in #38
• Add uptime to 4lw command by @JackyWoo in #66

Enhancement

• Add benchmark build.sh by @nicelulu in #69
• Add core dump size configuratuin to config.xml by @JackyWoo in #70
• Fix code style tools install error by @JackyWoo in #76
• Update gtest version to 1.14.0 by @JackyWoo in #73
• Ignore test_multinode_simple temporarily by @JackyWoo in #79
• Set default thread_count to CPU core size. by @JackyWoo in #75
• Add name to NIO threads by @JackyWoo in #74
• Disable memory tracker by @JackyWoo in #72
• Add git commit id by @JackyWoo in #82
• Add thread name by @JackyWoo in #83
• Remove session_consistent by @JackyWoo in #88

Bug

• Fix min_latency by @JackyWoo in #78

Full Changelog: v2.0.1...v2.0.2

Release v2.0.1

RaftKeeper v2.0.1 has made some improvements in stability.

What's Changed

Bug

• Nuraft error log by @JackyWoo in #15
• Fix some core dump and add asan to CI by @nicelulu in #47
• fix TSAN build error by @JackyWoo in #48
• Fix data races by @JackyWoo in #55
• Add ubsan to CI by @nicelulu in #59
• Add msan to CI by @nicelulu in #56

Enhancement

• set default charset to java files by @JackyWoo in #9
• Refact project structure by @JackyWoo in #13
• CI check code style by @nicelulu in #16
• Add build and test jobs to CI by @JackyWoo in #17
• Revert poco nio by @nicelulu in #20
• Organize the integration tests log output by @JackyWoo in #21
• Add PR template by @JackyWoo in #25
• chore: code style first part by @JackyWoo in #27
• clear migrate doc by @nicelulu in #46
• Refactor forwarder by @nicelulu in #31
• print build flags by @nicelulu in #49
• chore: code style 2nd part by @JackyWoo in #28
• chang build type to release by @JackyWoo in #51
• make sanitizer configurable by @JackyWoo in #54
• Wrap lambda block by @Hooollin in #64
• Optimize memory copying on request and response chains by @nicelulu in #57
• Update clang format by @JackyWoo in #50

New Contributors

• @Henry2SS
• @Hooollin

Full Changelog: v2.0.0...v2.0.1

Release v2.0.0

RaftKeeper v2.0.0 is a main version which contains a lot of features.

What's Changed

Feature

1. Support connection reconnect to server within session timeout.
2. NIO network framework which can support hundreds of thousands of connections.
3. New Snapshot & Log storage format who contains more meta datas.
4. New fsync mode FSYNC_PARALLEL in which leader can write log asynchronously, but aolso keep data safety.
5. New pipeline framework who exreamelly improve throughput.
6. Persistency improvement : requests response return in order in one session.

Full Changelog: https://github.com/JDRaftKeeper/RaftKeeper/commits/v2.0.0