Add sticky backend query hint

[zsparal]

Background

Sometimes we want to make sure that queries in a single connection all hit the same DB replica. This helps us remain in a consistent state in the face of small replication delays between the different hosts. In many cases, it's preferable to read a consistent state from a single replica, even if that replica might not contain the most up to date version of the world.

For example, let's say that we quickly insert two records (A and B) into the database and B causally depends on A. Once the application reads B, it assumes that A already exists and complains if it does not find it. This feature ensures read consistency by making sure that the application talks to the same replica during the lifetime of the connection.

The change

Sticky backends are disabled by default, and can be enabled by using the /* sticky_backend=1 */ query hint. This is then persisted for the entire session, so the client only has to set the value once. The setting takes effect with the current query being executed:

• when enabling sticky backends a load-balanced server will be selected for the query and the following queries will be executed against the same server
• when disabling sticky backends, this and all subsequent queries will be executed against a load-balanced server

We support three different sticky modes:

• 0: no sticky backends (default)
• 1: stick to the same replica as long as it is alive
• 2: strict sticky, if the sticky replica is not alive no other replica will be chosen until it errors out

Production experience

We (Mollie) have been running a patched version of proxysql in production since last August and it has been completely stable, while reducing the above-described problem in our systems.

Questions & notes

• We would be very happy to contribute an e2e test for this functionality if you could provide us with some guidance on the testing setup. We tried setting up the tests locally, but some of the test suite automation resides in a private repository under: https://github.com/proxysql/jenkins-build-scripts
• I would be happy to contribute an update to the documentation website as well (https://proxysql.com/documentation/query-annotations/) but I could not find the source code for it. If you could point me to the correct repository, I will also open a PR there with the changes

Fixes #4558

[mirostauder]

Automated message: PR pending admin approval for build testing

[zsparal]

@renecannao or @mirostauder Could you please take a look at this PR (and the related issue)? Do you think it is a feature that could benefit other users of ProxySQL?

Thank you 🙂

[mirostauder]

Can one of the admins verify this patch?

[zsparal]

Is there anything I can do to move this forward?