Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(sql): use a connection pool named "read" for some read operations in SqlExecutionRepository #4803

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

kirangodishala
Copy link
Contributor

@kirangodishala kirangodishala commented Nov 20, 2024

Introduce a read pool for some read-only database queries in SqlExecutionRepository. Enable this by configuring an additional connection pool named "read", like this:

sql:
  connectionPools:
    default:
      <...>
    read:
      jdbcUrl: jdbc:...
      user: orca_service
      password: 
      connectionTimeoutMs: 
      validationTimeoutMs: 
      maxPoolSize:
      minIdle:
      maxLifetimeMs:
      idleTimeoutMs:

Additional read-only database queries happen in places that expect to see "just-written" contents. They're not safe to convert to use the read replica until there's logic in place to be aware of replication lag, and wait until the read replica is up-to-date.

Copy link

@dzhengg dzhengg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we include an example (maybe in one of the commit messages or the description) for how to configure orca to use a read pool?

@dbyron-sf dbyron-sf changed the title feat(sql): use a connection pool named "read" for read operations in SqlExecutionRepository feat(sql): use a connection pool named "read" for some read operations in SqlExecutionRepository Nov 21, 2024
@dbyron-sf dbyron-sf marked this pull request as draft November 21, 2024 22:09
… DataSource bean available

depending on configuration
@kirangodishala kirangodishala force-pushed the use-read-replicas-in-execution-repository branch 2 times, most recently from d2a56fe to 53927e2 Compare December 24, 2024 14:38
@kirangodishala kirangodishala marked this pull request as ready for review December 24, 2024 15:16
dbyron-sf and others added 5 commits December 25, 2024 10:32
…SqlExecutionRepository if configured to do so.

WIP: so far this is only configuration, nothing actually uses the read pool
…nRepository

to pave the way to use the read pool for selecting by correlation id
- correlation ids for completed pipelines/orchestrations
- a stage
@kirangodishala kirangodishala force-pushed the use-read-replicas-in-execution-repository branch from 53927e2 to 942d48c Compare December 25, 2024 05:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants