You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm using a slightly different configuration of this repo but I noticed that my Scheduler is not queueing up tasks even though my CeleryExecutors are able to fire off tasks through the CLI or Web UI.
I'm wondering if there is good reason to hardcore the Celery Broker URL and backend to amqp://airflow:airflow@rabbitmq:5672/airflow rather than using something like amqp://$RABBITMQ_CREDS@$RABBITMQ_PORT_15672_TCP_ADDR:$RABBITMQ_SERVICE_PORT_NODE/$RABBITMQ_DEFAULT_VHOST where the host is dynamic? I've tried both methods and my scheduler is not queuing up tasks although it is constantly sending heartbeats.
The text was updated successfully, but these errors were encountered:
Thanks @stibbons I've been debugging and I've checked that the Workers are connected to the RabbitMQ - the worker pod shows the message that I expected when deploying on a single server. Flower also picked up the tasks and was able to log it as well.
How would I check if a Scheduler is dead? Do you know if the scheduler is heartbeating the Broker URL or the metaDB?
Some of my configs:
sql_alchemy_conn = postgresql+psycopg2://username:password@localhost/db
broker_url = amqp://airflow:airflow@$RABBITMQ_SERVICE_HOST:5672/airflow
celery_result_backend = postgresql+psycopg2://username:password@localhost/db # I've also tried with the amqp but no luck
I'm using a slightly different configuration of this repo but I noticed that my Scheduler is not queueing up tasks even though my CeleryExecutors are able to fire off tasks through the CLI or Web UI.
I'm wondering if there is good reason to hardcore the Celery Broker URL and backend to
amqp://airflow:airflow@rabbitmq:5672/airflow
rather than using something likeamqp://$RABBITMQ_CREDS@$RABBITMQ_PORT_15672_TCP_ADDR:$RABBITMQ_SERVICE_PORT_NODE/$RABBITMQ_DEFAULT_VHOST
where the host is dynamic? I've tried both methods and my scheduler is not queuing up tasks although it is constantly sending heartbeats.The text was updated successfully, but these errors were encountered: