Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: use redis as backend for koa-session #2693

Merged
merged 10 commits into from
May 13, 2024
2 changes: 2 additions & 0 deletions localenv/cloud-nine-wallet/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -85,8 +85,10 @@ services:
TRUST_PROXY: ${TRUST_PROXY}
AUTH_DATABASE_URL: postgresql://cloud_nine_wallet_auth:cloud_nine_wallet_auth@shared-database/cloud_nine_wallet_auth
AUTH_SERVER_DOMAIN: ${CLOUD_NINE_AUTH_SERVER_DOMAIN:-http://localhost:3006}
REDIS_URL: redis://shared-redis:6379/1
depends_on:
- shared-database
- shared-redis
shared-database:
image: 'postgres:15' # use latest official postgres version
restart: unless-stopped
Expand Down
3 changes: 2 additions & 1 deletion localenv/happy-life-bank/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ services:
WEBHOOK_URL: http://happy-life-bank/webhooks
OPEN_PAYMENTS_URL: ${HAPPY_LIFE_BANK_OPEN_PAYMENTS_URL:-http://happy-life-bank-backend}
EXCHANGE_RATES_URL: http://happy-life-bank/rates
REDIS_URL: redis://shared-redis:6379/1
REDIS_URL: redis://shared-redis:6379/2
WALLET_ADDRESS_URL: ${HAPPY_LIFE_BANK_WALLET_ADDRESS_URL:-https://happy-life-bank-backend/.well-known/pay}
ENABLE_TELEMETRY: false
depends_on:
Expand All @@ -73,6 +73,7 @@ services:
NODE_ENV: development
AUTH_DATABASE_URL: postgresql://happy_life_bank_auth:happy_life_bank_auth@shared-database/happy_life_bank_auth
AUTH_SERVER_DOMAIN: ${HAPPY_LIFE_BANK_AUTH_SERVER_DOMAIN:-http://localhost:4006}
REDIS_URL: redis://shared-redis:6379/3
depends_on:
- cloud-nine-auth
happy-life-admin:
Expand Down
1 change: 1 addition & 0 deletions packages/auth/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@
"axios": "^1.6.8",
"dotenv": "^16.4.5",
"graphql": "^16.8.1",
"ioredis": "^5.3.2",
"knex": "^3.1.0",
"koa": "^2.15.2",
"koa-bodyparser": "^4.4.1",
Expand Down
25 changes: 23 additions & 2 deletions packages/auth/src/app.ts
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ import { AccessService } from './access/service'
import { AccessTokenService } from './accessToken/service'
import { InteractionRoutes } from './interaction/routes'
import { ApolloArmor } from '@escape.tech/graphql-armor'
import { Redis } from 'ioredis'
import { LoggingPlugin } from './graphql/plugin'

export interface AppContextData extends DefaultContext {
Expand Down Expand Up @@ -98,6 +99,7 @@ export interface AppServices {
accessTokenService: Promise<AccessTokenService>
grantRoutes: Promise<GrantRoutes>
interactionRoutes: Promise<InteractionRoutes>
redis: Promise<Redis>
}

export type AppContainer = IocContract<AppServices>
Expand Down Expand Up @@ -346,12 +348,31 @@ export class App {

koa.use(cors())
koa.keys = [this.config.cookieKey]

const redis = await this.container.use('redis')
const maxAgeMs = 60 * 1000
koa.use(
session(
{
key: 'sessionId',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this correspond to what is stored inside redis? The key I see in redis is a UUID -> where does "sessionId" come into play?

Copy link
Contributor Author

@BlairCurrey BlairCurrey May 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the original configuration and from what I can tell the key name does not correspond to anything in redis, it's just for the http cookie returned to the client:

image

The value there would be the UUID key you see in redis.

maxAge: 60 * 1000,
signed: true
maxAge: maxAgeMs,
signed: true,
store: {
async get(key) {
return await redis.hgetall(key)
},
async set(key, session) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see there is a third parameter maxAge possible in this function. Do we want to include that here?

Copy link
Contributor Author

@BlairCurrey BlairCurrey May 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assumed I would use that then purposefully did not after seeing it could be undefined or "session" depending on how we actually configure it. The type here doesnt account for how we actually configure it, so I didn't want to handle scenarios like session or undefined (which should never happen) when we know we want to expire it. I suppose an alternative would be handling session and undefined by defaulting back to expireInMs, but I figured it was just simpler to control both the session max age and redis ttl from 1 variable. If we did handle it by defaulting back to expireInMs then it would basically be: set it to maxAge when maxAge is a number (which will match our configuration) or set it to match our configuration.

Does that make sense? Open to feedback here if anyone sees a better way.

// Add a delay to cookie age to ensure redis record expires after cookie
const expireInMs = maxAgeMs + 10 * 1000
Comment on lines +365 to +366
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, this is to possibly allow the "final" request with the to-be-expired cookie parameter to complete before deleting it in redis?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah basically. could potentially be shorter I guess - just needs to be long enough to cover the time between finding the non-expired koa-session in the server and retrieving the redis record. no harm in having a little extra buffer though I figure.

const op = redis.multi()
op.hset(key, session)
op.expire(key, expireInMs)
await op.exec()
},
Copy link
Contributor

@mkurapov mkurapov May 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
},
await redis.set(key, JSON.stringify(session), 'PX', expireInMs)

I think psetex is deprecated based on the docs. (sorry for the odd diff)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

psetex is not deprecated per my knowledge, but it is not wise to use it, since it will be deprecated and removed.
From the docs:

Note: Since the SET command options can replace SETNX, SETEX, PSETEX, GETSET, it is possible that in future versions of Redis these commands will be deprecated and finally removed.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, no need to do stringify on session object. Instead, you can just use hmset(key, session)

Copy link
Contributor Author

@BlairCurrey BlairCurrey May 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to hmset as suggested except I used hset. apparently hmset, like psetex is marked as deprecated in redis docs https://redis.io/docs/latest/commands/hmset/

dda0079

async destroy(key) {
await redis.hdel(key)
}
}
},
koa
)
Expand Down
34 changes: 33 additions & 1 deletion packages/auth/src/config/app.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import * as crypto from 'crypto'
import dotenv from 'dotenv'
import * as fs from 'fs'
import { ConnectionOptions } from 'tls'

function envString(name: string, value: string): string {
const envValue = process.env[name]
Expand Down Expand Up @@ -53,5 +55,35 @@ export const Config = {
accessTokenDeletionDays: envInt('ACCESS_TOKEN_DELETION_DAYS', 30),
incomingPaymentInteraction: envBool('INCOMING_PAYMENT_INTERACTION', false),
quoteInteraction: envBool('QUOTE_INTERACTION', false),
listAllInteraction: envBool('LIST_ALL_ACCESS_INTERACTION', true)
listAllInteraction: envBool('LIST_ALL_ACCESS_INTERACTION', true),
redisUrl: envString('REDIS_URL', 'redis://127.0.0.1:6379'),
redisTls: parseRedisTlsConfig(
envString('REDIS_TLS_CA_FILE_PATH', ''),
envString('REDIS_TLS_KEY_FILE_PATH', ''),
envString('REDIS_TLS_CERT_FILE_PATH', '')
)
}

function parseRedisTlsConfig(
caFile: string,
keyFile: string,
certFile: string
): ConnectionOptions | undefined {
const options: ConnectionOptions = {}

// self-signed certs.
if (caFile !== '') {
options.ca = fs.readFileSync(caFile)
options.rejectUnauthorized = false
}

if (certFile !== '') {
options.cert = fs.readFileSync(certFile)
}

if (keyFile !== '') {
options.key = fs.readFileSync(keyFile)
}

return Object.keys(options).length > 0 ? options : undefined
}
6 changes: 6 additions & 0 deletions packages/auth/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import {
} from '@interledger/open-payments'
import { createInteractionService } from './interaction/service'
import { getTokenIntrospectionOpenAPI } from 'token-introspection'
import { Redis } from 'ioredis'

const container = initIocContainer(Config)
const app = new App(container)
Expand Down Expand Up @@ -197,6 +198,11 @@ export function initIocContainer(
}
)

container.singleton('redis', async (deps): Promise<Redis> => {
const config = await deps.use('config')
return new Redis(config.redisUrl, { tls: config.redisTls })
})

return container
}

Expand Down
Binary file modified packages/documentation/public/img/localenv-architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified packages/documentation/public/img/rafiki-architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ and the databases

- TigerBeetle or Postgres (accounting)
- Postgres (Open Payments resources, auth resources)
- Redis (STREAM details)
- Redis (STREAM details, auth sessions)

To integrate Rafiki with your own services, view the [integration documentation](/integration/getting-started).

Expand Down Expand Up @@ -128,6 +128,10 @@ Now, the Admin UI can be found on localhost:3010.
| `LOG_LEVEL` | auth.logLevel | `info` | [Pino Log Level](https://getpino.io/#/docs/api?id=levels) |
| `NODE_ENV` | auth.nodeEnv | `development` | node environment, `development`, `test`, or `production` |
| `QUOTE_INTERACTION` | auth.interaction.quote | `false` | flag - quote grants are interactive or not |
| `REDIS_TLS_CA_FILE_PATH` | auth.redis.tlsCaFile | `''` | [Redis TLS config](https://redis.io/docs/management/security/encryption/) |
| `REDIS_TLS_CERT_FILE_PATH` | auth.redis.tlsCertFile | `''` | [Redis TLS config](https://redis.io/docs/management/security/encryption/) |
| `REDIS_TLS_KEY_FILE_PATH` | auth.redis.tlsKeyFile | `''` | [Redis TLS config](https://redis.io/docs/management/security/encryption/) |
| `REDIS_URL` | `auth.redis.host`, `auth.redis.port` | `redis://127.0.0.1:6379` | The connection URL for Redis. For Helm, these components are provided individually. |
| `TRUST_PROXY` | | `false` | flag to use X-Forwarded-Proto header to determine if connections is secure |
| `WAIT_SECONDS` | auth.grant.waitSeconds | `5` | wait time included in grant request response (`grant.continue`) |
| `ENABLE_MANUAL_MIGRATIONS` | auth.enableManualMigrations | `false` | When set to true, user needs to run database manually with command `npm run knex -- migrate:latest --env production` |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ These services rely on four databases:
- A postgres database used by the `backend`
- A separate postgres database used by `auth`.
- [TigerBeetle](https://github.com/coilhq/tigerbeetle) used by `backend` for accounting balances at the ILP layer.
- Redis used by `backend` as a cache to share STREAM connection details across processes.
- Redis used by `backend` as a cache to share STREAM connection details across processes and `auth` to store session data.

## Backend

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ These packages depend on the following databases:

- TigerBeetle or Postgres (accounting)
- Postgres (Open Payments resources, auth resources)
- Redis (STREAM details)
- Redis (STREAM details, auth sessions)

We provide containerized versions of our packages together with two pre-configured docker-compose files ([Cloud Nine Wallet](https://github.com/interledger/rafiki/blob/main/localenv/cloud-nine-wallet/docker-compose.yml) and [Happy Life Bank](https://github.com/interledger/rafiki/blob/main/localenv/happy-life-bank/docker-compose.yml)) to start two Mock Account Servicing Entities with their respective Rafiki backend and auth servers. They automatically peer and 2 to 3 user accounts are created on both of them.

Expand Down
Loading
Loading