"Failed to authenticate HTTP request: Authentication required" JWT token pulsar-manager adding new environment

[Marble879]

Hello,

Summary

When trying to use JWT authentication with pulsar-manager, I can not create new environments. I can use JWT tokens successfully via python clients, and I can also create new environments via the dashboard when JWT is disabled.

My docker-compose is as follows:

  version: '3'
networks:
  pulsar:
    driver: bridge

#secrets:
#  jwt_secret_key:
#    file: ./secrets/auth_secret.key
#  jwt_broker_key:
#    file: ./secrets/broker_token.txt

services:
  # Start zookeeper
  zookeeper:
    image: apachepulsar/pulsar:3.0.0
    container_name: zookeeper
    restart: on-failure
    networks:
      - pulsar

    environment:
      - metadataStoreUrl=zk:zookeeper:2181
      - PULSAR_MEM=-Xms256m -Xmx256m -XX:MaxDirectMemorySize=256m
    command: >
      bash -c "bin/apply-config-from-env.py conf/zookeeper.conf && \
             bin/generate-zookeeper-config.sh conf/zookeeper.conf && \
             exec bin/pulsar zookeeper"
    healthcheck:
      test: ["CMD", "bin/pulsar-zookeeper-ruok.sh"]
      interval: 10s
      timeout: 5s
      retries: 30

  # Init cluster metadata
  pulsar-init:
    container_name: pulsar-init
    hostname: pulsar-init
    image: apachepulsar/pulsar:3.0.0
    networks:
      - pulsar
    command: >
      bin/pulsar initialize-cluster-metadata \
               --cluster cluster-a \
               --zookeeper zookeeper:2181 \
               --configuration-store zookeeper:2181 \
               --web-service-url http://broker:8080 \
               --broker-service-url pulsar://broker:6650
    depends_on:
      zookeeper:
        condition: service_healthy

  # Start bookie
  bookie:
    image: apachepulsar/pulsar:3.0.0
    container_name: bookie
    restart: on-failure
    networks:
      - pulsar
    environment:
      - clusterName=cluster-a
      - zkServers=zookeeper:2181
      - metadataServiceUri=metadata-store:zk:zookeeper:2181
      # otherwise every time we run docker compose uo or down we fail to start due to Cookie
      # See: https://github.com/apache/bookkeeper/blob/405e72acf42bb1104296447ea8840d805094c787/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Cookie.java#L57-68
      - advertisedAddress=bookie
      - BOOKIE_MEM=-Xms512m -Xmx512m -XX:MaxDirectMemorySize=256m
    depends_on:
      zookeeper:
        condition: service_healthy
      pulsar-init:
        condition: service_completed_successfully

    command: bash -c "bin/apply-config-from-env.py conf/bookkeeper.conf && exec bin/pulsar bookie"

  # Start broker
  broker:
    image: apachepulsar/pulsar:3.0.0
    container_name: broker
    hostname: broker
    restart: on-failure
    networks:
      - pulsar
    environment:
      - metadataStoreUrl=zk:zookeeper:2181
      - zookeeperServers=zookeeper:2181
      - clusterName=cluster-a
      - managedLedgerDefaultEnsembleSize=1
      - managedLedgerDefaultWriteQuorum=1
      - managedLedgerDefaultAckQuorum=1
      - advertisedAddress=broker
      - advertisedListeners=external:pulsar://127.0.0.1:6650
      - PULSAR_MEM=-Xms512m -Xmx512m -XX:MaxDirectMemorySize=256m
      # Authentication
      - authenticationEnabled=true
      - authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderToken
      - brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationToken
      - brokerClientAuthenticationParameters={"token":"<token>"}
      - tokenSecretKey=file:/pulsar/auth-secret.key
    volumes:
      - ./secrets/auth-secret.key:/pulsar/auth-secret.key
    ports:
      - "6650:6650"
      - "8080:8080"
    depends_on:
      zookeeper:
        condition: service_healthy
      bookie:
        condition: service_started

    command: bash -c "bin/apply-config-from-env.py conf/broker.conf && exec bin/pulsar broker"

  dashboard:
    container_name: pulsar-manager
    hostname: pulsar-manager
    restart: on-failure
    image: apachepulsar/pulsar-manager:v0.4.0
    ports:
      - "9527:9527"
      - "7750:7750"
      - "7000:7000"
    depends_on:
      - broker
    networks:
      - pulsar
    environment:
      SPRING_CONFIGURATION_FILE: /pulsar-manager/pulsar-manager/application.properties
      ENV_JWT_TOKEN: <token>
      ENV_SECRET_KEY: file:///pulsar-manager/secret/auth-secret.key


    volumes:
      - ./.data/dashboard:/pulsar-manager/pulsar-manager/dbdata
      - ./secrets/auth-secret.key:/pulsar-manager/secret/auth-secret.key
    profiles:
      - local_dev

Error logs

The following pop-up appears on the dashboard after trying to create an environment:

I also get the following logs from the broker after trying to create an environment:
broker | 2023-06-26T12:36:35,322+0000 [pulsar-web-37-17] WARN org.apache.pulsar.broker.web.AuthenticationFilter - [ip] Failed to authenticate HTTP request: Authentication required broker | 2023-06-26T12:36:35,323+0000 [pulsar-web-37-17] INFO org.eclipse.jetty.server.RequestLog - ip - - [26/Jun/2023:12:36:35 +0000] "GET /admin/v2/clusters HTTP/1.1" 401 151 "-" "Pulsar-Java-v2.7.0" 1

I appreciate any guidance on how I can approach solving this. Please let me know if more information is required :)