What is the plan to release traefik v2 support

[trongchata]

The latest release tag still be 0.3.0, only support traefik v1.7
I saw there are lots of change to support latest Traefik v2 and etcd v3. So when is the plan for next version release ?

Thanks

[welcome]

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively.

You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉

[minrk]

Hopefully soon. I think we've done most of the changes we need to. The big outstanding issues are #155 and #151, neither of which need to be addressed before a release with what we have so far.

I think we should probably finish #163, so we have a sense of our current situation re: performance, because when I try to stress the current branch I get lots of timeouts and failures (CHP can handle far more transactions than traefik without failure).

Do you happen to use etcd or consul for your traefik deployment?

[trongchata]

Hi @minrk
Thank you for your reply. Yeah, we are using Joyterhub + Traefik 2.9 + etcd3, to implement the dynamic setting.
Because the support for traefik v2 has not been release, so we build traefik-proxy from main branch in local.

But during we test it, have met an issue. The Traefik proxy already get the response from traefik api, but it throw the error.
ValueError: No traefik entrypoint ports ([":8082", ":8080", ":8081"]) match public_url: http://:8000!

I know the http://:8000 is default value, but I have tried several attribute, still can't change it.
Here is my jupyter configuration

**c.Jupyterhub.proxy_class = "traefik_etcd"
c.TraefikEtcdPorxy.should_start = False
c.TraefikEtcdPorxy.public_url= "http://traefik_proxy_host:8001"

c.TraefikEtcdPorxy.etcd_url= "http://etcd:2379"
c.TraefikEtcdPorxy.traefik_api_url = "https://api_url"
c.TraefikEtcdPorxy.kv_traefik_prefix= "traefik"
c.TraefikEtcdPorxy.kv_jupyterhub_prefix= "/hub"**

Any suggestions for it?
Thanks

[trongchata]

The problem fixed, we have to setup
c.JupyterHub.bind_url = "http://:{port}"

In the traefik-proxy, it will use it to validate the entries response from traefik.

[minrk]

Yeah, I'm not sure we should be doing that check when traefik is run externally. I'll look into that.

[minrk]

If you'd like to test it, 1.0.0b1 is published: https://pypi.org/project/jupyterhub-traefik-proxy/1.0.0b1/

[alleny87]

So been hoping to see a version of this for traefik v2 for a while, wanted to test it out, running pypi pre-release version: 1.0.0b1 getting a error:

[W 2023-05-02 20:11:35.048 JupyterHub proxy:370] SSL Error checking for traefik static configuration: [Errno 99] Cannot assign requested address
.... Repeats many times.....
[W 2023-05-02 20:12:04.634 JupyterHub proxy:370] SSL Error checking for traefik static configuration: [Errno 99] Cannot assign requested address
[E 2023-05-02 20:12:04.635 JupyterHub app:3377]
    Traceback (most recent call last):
      File "/home/vscode/.local/lib/python3.10/site-packages/jupyterhub/app.py", line 3375, in launch_instance_async
        await self.start()
      File "/home/vscode/.local/lib/python3.10/site-packages/jupyterhub/app.py", line 3141, in start
        await self.proxy.get_all_routes()
      File "/home/vscode/.local/lib/python3.10/site-packages/jupyterhub_traefik_proxy/proxy.py", line 681, in get_all_routes
        await self._start_future
      File "/home/vscode/.local/lib/python3.10/site-packages/jupyterhub_traefik_proxy/proxy.py", line 501, in _start_external
        await self._wait_for_static_config()
      File "/home/vscode/.local/lib/python3.10/site-packages/jupyterhub_traefik_proxy/proxy.py", line 377, in _wait_for_static_config
        await exponential_backoff(
      File "/home/vscode/.local/lib/python3.10/site-packages/jupyterhub/utils.py", line 237, in exponential_backoff
        raise asyncio.TimeoutError(fail_message)
    asyncio.exceptions.TimeoutError: Traefik static configuration not available

Following externally managed instructions from here:
https://jupyterhub-traefik-proxy.readthedocs.io/en/latest/file.html

Running jupyter in a vscode container with exposed port 9000 for jupyterhub to run on, but also why the paths dont match, but they are mapped in correctly
Traefik is running as its own container with the config.yaml mapped in, and the shared /etc/traefik/config mapped into both the treafik container and vscode container. I can easily see files by running a docker exec so its not a mapping issue. I have tried with all the values below, to include the currently commented out ones. Configs below:

Simple jupyterhub config:

c.JupyterHub.port = 9000
c.JupyterHub.authenticator_class = 'jupyterhub.auth.DummyAuthenticator'
c.JupyterHub.proxy_class = "traefik_file"
c.TraefikFileProviderProxy.should_start = False
# Not needed since externally managed
# c.TraefikFileProviderProxy.static_config_file = "/workspaces/traefik-proxy/traefik.yaml"
c.TraefikFileProviderProxy.dynamic_config_file = "/workspaces/traefik-proxy/config/jupyter.toml"
c.TraefikFileProviderProxy.traefik_api_username = "jupyterhub"
c.TraefikFileProviderProxy.traefik_api_password = "jupyterhub"
#c.JupyterHub.bind_url = 'http://jupyter.localhost'
#c.JupyterHub.hub_bind_url = 'http://localhost:9000'
c.JupyterHub.subdomain_host = "http://jupyter.localhost"
c.JupyterHub.ssl_cert = 'externally managed'

traefik config:

api:
  insecure: true
  dashboard: true
global:
  checkNewVersion: false
  sendAnonymousUsage: false
entryPoints:
  web:
    address: ":80"
  auth_api:
    address: ":8099"
providers:
  docker:
    exposedByDefault: false
  file:
    directory: "/etc/traefik/config"
    watch: true

It seems to generate some of the dynamic config in jupyter.toml in the location in the config above /etc/traefik/config/jupyter.toml, which i can see in my traefik dashboard, but it doesn't generate any routes.

jupyter.toml:

[jupyterhub.routes]

[http.routers.route_api]
rule = "Host(`localhost`) && PathPrefix(`/api`)"
entryPoints = [ "auth_api",]
service = "api@internal"
middlewares = [ "auth_api",]

[http.middlewares.auth_api.basicAuth]
users = [ "jupyterhub:$apr1$zVq7lvQQ$YvJLv.S1AC3J3HeLIZ/t81",]

Any idea whats generating the error above?

[minrk]

@alleny87 thanks for testing!

Can you run jupyterhub with --debug and share more of the logs? Also the logs of the traefik container with --logging=debug?

If the Hub and treafik are running in different containers, you do need to make sure that they are listening on public ips within the container, and connecting to hostnames connectable from other containers (typically the container name).

The two relevant config values are:

# make sure the Hub is accessible from other containers
c.JupyterHub.hub_bind_url = 'http://:8000' 
# tell the hub where the proxy is
c.TraefikProxy.traefik_api_url = 'http://treafik-container-name:8099'

Also try disabling subdomain_host for now, as that adds complications to the routing while debugging.

Any idea whats generating the error above?

My guess is it's the use of localhost, since the URLs don't appear to have been configured.

The 'SSL' part is a mistake in our log messages, since I didn't realize OSErrors could occur here without SSL set up, but the important part is 'cannot assign requested address', which seems weird to me in a connect call, but seems common in docker when connecting to localhost that shouldn't be used for some reason.

[chuckmandu]

Adding a rule to traefik to filter both localhost and 127.0.0.1 for the api@internal service resolved the

asyncio.exceptions.TimeoutError: Traefik static configuration not available

error for me (using Traefik 2.9.8 and Proxy 1.0.0b2).

That is:

[http.routers.traefik_api]
rule = "(Host(`127.0.0.1`) || Host(`localhost`)) && PathPrefix(`/api`)"
entryPoints = [ "traefik_api",]
service = "api@internal"
middlewares = [ "traefik_api",]

[minrk]

Thanks for testing and reporting back!

[minrk]

1.0 is out!