How to Verify if SQL Lab is Using Redis Cache in Apache Superset (Docker Setup)?

[Achintyarai22]

I am running Apache Superset in a Docker environment and have configured Redis as the cache backend. I want to verify if SQL Lab is effectively using Redis for caching queries.

configurations i have used.
REDIS_HOST = os.getenv("REDIS_HOST", "redis")
REDIS_PORT = os.getenv("REDIS_PORT", "6379")
REDIS_CELERY_DB = os.getenv("REDIS_CELERY_DB", "0")
REDIS_RESULTS_DB = os.getenv("REDIS_RESULTS_DB", "1")

RESULTS_BACKEND = FileSystemCache("/app/superset_home/sqllab")

CACHE_CONFIG = {
"CACHE_TYPE": "RedisCache",
"CACHE_DEFAULT_TIMEOUT": 300,
"CACHE_KEY_PREFIX": "superset_",
"CACHE_REDIS_HOST": REDIS_HOST,
"CACHE_REDIS_PORT": REDIS_PORT,
"CACHE_REDIS_DB": REDIS_RESULTS_DB,
}
DATA_CACHE_CONFIG = CACHE_CONFIG

QUERY_SEARCH_CACHE_CONFIG = {
'CACHE_TYPE': 'RedisCache',
'CACHE_DEFAULT_TIMEOUT': 5000, # 5 minutes
'CACHE_KEY_PREFIX': 'superset_query_cache_',
'CACHE_REDIS_HOST': REDIS_HOST,
'CACHE_REDIS_PORT': 6379,
"CACHE_REDIS_DB": REDIS_RESULTS_DB,
'CACHE_REDIS_URL': 'redis://superset-redis-master:6379'
}

class CeleryConfig:
broker_url = f"redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_CELERY_DB}"
imports = (
"superset.sql_lab",
"superset.tasks.scheduler",
"superset.tasks.thumbnails",
"superset.tasks.cache",
)
result_backend = f"redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_RESULTS_DB}"
worker_prefetch_multiplier = 10
task_acks_late = True
beat_schedule = {
"reports.scheduler": {
"task": "reports.scheduler",
"schedule": crontab(minute="", hour=""),
},
"reports.prune_log": {
"task": "reports.prune_log",
"schedule": crontab(minute=10, hour=0),
},
}

CELERY_CONFIG = CeleryConfig

[dosubot[bot]]

To verify if SQL Lab is using Redis for caching queries in your Apache Superset Docker setup, you can check the CACHE_CONFIG and QUERY_SEARCH_CACHE_CONFIG in your configuration. Both configurations specify the use of RedisCache, indicating that Redis is being used for caching.

Here's a breakdown of your configuration:

1. CACHE_CONFIG: This is set to use RedisCache with the specified Redis host, port, and database (REDIS_RESULTS_DB). This configuration is used for general caching purposes in Superset.

2. QUERY_SEARCH_CACHE_CONFIG: This is also set to use RedisCache with a different key prefix and timeout. It uses the same Redis host and port, and the same database as CACHE_CONFIG.

To ensure that SQL Lab is effectively using Redis for caching, you can:

• Check the Redis database (REDIS_RESULTS_DB) for cached keys using the Redis CLI. You can do this by connecting to your Redis instance and running the KEYS command to see if there are any keys with the prefixes specified in your configurations (superset_ and superset_query_cache_).

• Monitor the Redis instance to see if there is any activity when you run queries in SQL Lab. This can be done using Redis monitoring tools or by checking the Redis logs.

These steps will help you confirm that SQL Lab is utilizing Redis for caching queries in your setup ^[1][2].

To continue talking to Dosu, mention @dosu.

[sfirke]

Hm can you configure the Redis cache to cache queries performed in SQL Lab? I have not tried to do that. But regardless, here is how I confirmed that caching was working for other operations (e.g., loading a chart a second time):

• Enter the Redis cache's docker container with docker exec -it <container-name> bash
• Look at the Redis logs while I load a Superset dashboard repeatedly. It should show activity. Try redis-cli monitor once you are in the container.
• There's also a command in Redis to dump all keys, that will show you which caches are being populated (which would indicate they are set up correctly)

The second and third bullets can be done directly in Redis. If folks try this, feel free to reply with what worked.

Possibly useful links on cache config:

• https://stackoverflow.com/questions/73066916/understand-how-caching-helps-in-auto-refreshing-superset-dashboards
• https://stackoverflow.com/questions/72143877/my-superset-cache-is-set-to-redis-but-i-still-get-cache-warning-in-logs