OIDC authN rucio int server draft

[dciangot]

A starting point to address #61

[vkuznet]

@dciangot, the changes are fine but I want to clarify few things:

• does rucio codebase provide OIDC auth layer
• if so, does it support multiple OIDC providers
• if not, how we should handle tokens issued by different providers

I'm asking this question because on CMSWEB we'll use auth proxy server which does support multiple OIDC providers, in particular CERN SSO and IAM. And, token issued by one can be transparently used accessing services on another. I don't know if Rucio OIDC layer is capable of doing this. We have different use-case to support, e.g. if end-user access DAS with token issued by CERN SSO and DAS needs to access Rucio, will this token be used again. If not, we need to add extra code and logic to services like DAS, ReqMgr2, etc. to handle this. Similar, the reverse case is true, if grid job acquire IAM token this token should be usable on CMSWEB as well.

[dciangot]

Hi @vkuznet ,

indeed RUCIO supports multiple IdPs. Only one can be specified as admin_issuer though. The admin_issuer is the one that will be contacted for automatic User mapping. That can be overcome with a simple/dedicated periodic probe though, so not big of an issue I suppose.

It will only takes to generate the proper OIDC clients on CERN SSO and append it in the idpsecret json. Resulting in something like:

{
  "cms": {
    "redirect_uris": [
      "https://<server_name>/auth/oidc_token",
      "https://<server_name>/auth/oidc_code"
    ],
    "registration_access_token": "<RAT_string>",
    "client_secret": "<client_secret>",
    "SCIM": {
      "client_secret": "<client_secret>",
      "grant_type": "client_credentials",
      "registration_access_token": "<RAT_string>"
    },
    "issuer": "https://<issuer_server_name>/"
  },
  "cern-sso": {
    "redirect_uris": [
      "https://rucio-auth.cern.ch/auth/oidc_token",
      "https://rucio-auth.cern.ch/auth/oidc_code"
    ],
    "registration_access_token": "eyJraWQiOi ...",
    "client_id": "fdc297fc-09 ...",
    "client_secret": "APFVcga_X ...",
    "SCIM": {
      "client_secret": "IQqAcMOa ...",
      "grant_type": "client_credentials",
      "registration_access_token": "eyJraW ...",
      "client_id": "5b5e5d3 ..."
    },
    "issuer": "https://wlcg.cloud.cnaf.infn.it/"
  }
}

So I think we should take the occasion to test it. Adding it to the list of tasks in #61

[dciangot]

Regarding the management of different token. Each IdP can be added to the user RUCIO account via something like:

rucio-admin identity add --account admin_account_name \
  --type OIDC \
  --id "SUB=b3127dc7-2be3-417b-9647-6bf61238ad01, \
    ISS=https://wlcg.cloud.cnaf.infn.it/" \
  --email "[email protected]"

where SUB is the subject of the token.

I think that is then simply a matter of defining an automatic procedure for that.

One thing that I don't know is if RUCIO can accept tokens that are not in the form of WLCG profile (*). I'd say yes, but we need to find out.

(*) https://zenodo.org/record/3460258

[dciangot]

.... We have different use-case to support, e.g. if end-user access DAS with token issued by CERN SSO and DAS needs to access Rucio, will this token be used again..

Since we're here, can I ask why end-users are supposed to do that instead of going for cms-auth?
Also, in the case you mentioned, shouldn't the two tokens have a different audience ("das" and "rucio", or something like that)?

[vkuznet]

ok, user access CMSWEB. By default on CMSWEB we use CERN SSO. Now, user access DAS page and it is asked to authenticate with CERN SSO. As such (s)he gets CERN SSO token, and proceed to DAS page where (s)he put site dataset=... query which will be send to different services, like DBS and Rucio. We have two options here:

• use HTTP header of original client's request to cmsweb which will have CERN SSO token, or
• issue a new call to Rucio to obtain Rucio token

Another use-case, ReqMgr2 needs to place some request. It runs on CMSWEB where we obtain CERN SSO or IAM token, let's say the former. Then it can use this token to obtain DBS and Rucio info via some API calls.

Another use-case, grid job was submitted with IAM token, but inside of the job someone will make a call to DBS or use dasgoclient.

If we'll need to support all use-cases we will be forced to adjust our software stack at different places to handle (obtain) tokens. Instead, the token usage should be transparent from underlying software. Once you get your token and put it in HTTP header it should be usable anywhere in our systems.

The scope is a different story. It defines what you can do but not how you can access it. Therefore, if you have read rule you should be able to access all services using single token.

[dciangot]

Ok, I think that technically, on Rucio land, we should be able to handle all of this transparently, as mentioned above. It's matter of giving it a try.

My concern was on why we login in iam using cern sso basically, in some place, while directly on cern sso in some other place. But I suppose there are some historical reason.

[dciangot]

To start testing I think I'm only missing my DN (*) to be linked to a rucio account in the integration cluster and probably with some admin privileges. Also a client_id and secret for CERN SSO, @vkuznet do you think that CERN SSO can provide something similar to these (**)?

I think @ericvaandering you should have permissions to do the DN part, but I also do not know who else has the admin powers in the integration instance.

(*) /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=dciangot/CN=735100/CN=Diego Ciangottini

(**)
https://github.com/dmwm/rucio-flux/blob/e583d7e0725733262b1b38fcdf952d8065ab2e6b/scripts/create_iam_clients/client-configs/req-client-admin-scim.json
https://github.com/dmwm/rucio-flux/blob/e583d7e0725733262b1b38fcdf952d8065ab2e6b/scripts/create_iam_clients/client-configs/req-client-user-auth.json

[vkuznet]

@dciangot you can see it for yourself. Just visit cmsweb-test1.cern.ch/token, take your access token, paste it to jwt.io and see which attributes are available. We deployed auth proxy server (aps) on all cmsweb test clusters, cmsweb-test[1-7], and cmsweb-auth. Later this month it will be available on cmsweb-testbed.

[dciangot]

ok, thanks @vkuznet

I see a couple of possible issues. The first one is that I don't see any scopes claim in the token got from the endpoint you mentioned. While at least I would expect openid and profile, that would be the bare minimum for an OIDC authorization flow. RUCIO is checking the presence of these scopes as specified in the configuration (*), together with the aud claim that it has to trust (default is rucio) that in the your token is cms-go. For the latter problem it might be convenient to converge in a more general cms aud?

All in all looks to me, that is matter of define a spec to follow (I see scopes claim pretty everywhere with oidc, so I was surprised not to find it in the cmsweb token). Was the cmsweb token specification described somewhere? Do you envision any issue in introducing scopes claim and renaming aud at cmsweb level?

(*)
[oidc]
idpsecrets = /path/to/your/idpsecrets.json
admin_issuer = <IdP_nickname>
expected_audience = ''
expected_scope = 'openid profile'

[dciangot]

@dciangot you can see it for yourself. Just visit cmsweb-test1.cern.ch/token, take your access token, paste it to jwt.io and see which attributes are available. We deployed auth proxy server (aps) on all cmsweb test clusters, cmsweb-test[1-7], and cmsweb-auth. Later this month it will be available on cmsweb-testbed.

Also, I would need an actual oidc client (similar to what aps used I think) to be passed to Rucio server, how did you generate one?

[vkuznet]

Yes, it is matter of common convention. The aps uses CERN application portal where you can define the rules/scopes for your service. The aud is define there too. Originally, I used cms-go for testing go services, later I defined different application and aud now has different values in different aps servers, e.g. in monitoring one we use cms-monit, for cmsweb auth we have cmsweb-auth, for SONIC we have cms-sonic, etc. I think for cmsweb we will have cms-web or cmsweb, etc. The scopes again are defined in application portal and we need an agreement between services on a minimum set.

Regarding oidc client, you should use official oidc-agent toolkit along with oidc-gen tool which generate tokens. See it here: https://indigo-iam.github.io/v/v1.7.0/docs/tasks/user/getting-a-token/. But I also developed token manager within aps codebase, see https://github.com/dmwm/auth-proxy-server/tree/master/manager as well as decode-token tool (similar to jwt.io but CLI based), see it here https://github.com/dmwm/auth-proxy-server/tree/master/decode

[dciangot]

ok so you mean that I can generate cern sso clients with oidc-agent? If yes it would be partially ok, since I could be not allowed to create client with the grant_type rucio needs (at least for IAM it needs admin(myself) client edit). Indeed is not a problem of generating token, but rather produce a client with certain grant_type (e.g. exchange-token, scim:read (optional)... etc)

Looks good to me. Action items as I see them right now:

• DiegoC. try to create oidc client for cern-SSO and pass it to RUCIO configuration
• Manually test via token produced on oidc-agent
• Check if the token from cmsweb is working too

[dciangot]

so @vkuznet indeed I used oidc-agent and the client created via cern SSO to get an access token. I have one question though. I see the scope claim (*) information in the token now, while it was completely missing from the token got from cmsweb-test1.cern.ch/token . Is that expected? I think that everything should work with the one I get from oidc-agent, but not with the one from the endpoint, due to the lack of "scope" claim.

No hurry, but let me know if you have any insight.

(*)
"scope": "openid offline_access profile email"

[vkuznet]

@dciangot I'm not sure what you mean by oidc client, the client should be written in some software using some library. As I told you I written Token manager for aps which talks to CERN SSO to obtain the token. The oidc-agent is a an agent similar to ssh one, and oidc-gen is a tool to obtain IAM token. Therefore, with oidc-agent and oidc-gen you'll only get IAM token. While if you'll use web interface, e.g. cmsweb-test1.cern.ch/token, or you'll use token manager from aps you'll get CERN SSO token. Each token will be issued using token provider attributes, therefore in IAM token you'll get scopes defined by IAM, while in cmsweb you'll get scopes defined by CERN SSO and application portal. If you want to generate your own token, you may use any library listed on jwt.io and your favorite language, but they will not be acceptable in CMSWEB or Rucio until you'll adjust their configuration to accept new token provider.

[vkuznet]

regarding your second question, the web token end-point I think has a problem that instead of access token it shows intermediate token, I remember that I should fix it at some point.

[dciangot]

regarding your second question, the web token end-point I think has a problem that instead of access token it shows intermediate token, I remember that I should fix it at some point.

ok thanks, that was the whole point. Indeed should be fixed if that is the access token used by end-user to contact rucio

All the rest I think is a misunderstanding in terminology. I'm fully aware of all the chain that you mention (getting some headaches since a while already :) ) and it is under control for both cern sso and cms Iam on my side. For both I mange to get token on my own via oidc-agent.

What I mean by oidc-client is the client_id and client_secret needed to communicate with the provider, allowing for certain redirect uri and grant_types.

[vkuznet]

ok, the client_id and client_secrets are client's credentials and not oidc-client. These credentials are defined in CERN application portal and I can provide them to certain parties if necessary. In general, whoever who register application in portal can provide client's credentials.

And, here is aps issue I opened to fix the access token: dmwm/auth-proxy-server#3

[dciangot]

Yeah, inaccurate terms I brought from iam discussion then, sorry about that. Indeed I had already managed to create the creds via the portal and the use them to configure oidc-agent, so it's all good.

I'm ready to start testing as soon as @ericvaandering or any other integration admin give me rights to add oidc identities to rucio account. If some additional grant_types will be needed on client creds for CERN SSO I'll let you know, but hopefully the default should be ok.

Thanks a lot

[dciangot]

So, I have added the OIDC identities to my account (*), now I should be ready to authenticate as soon as this PR gets merged!

So @ericvaandering , let me know what do you think. It looks safe enough to be included already IMO.

(*)

# rucio-admin account list-identities dciangot
Identity: /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=dciangot/CN=735100/CN=Diego Ciangottini,	type: X509
Identity: SUB=c4b22a94-6dda-4312-b77f-726813e963ae,ISS=https://cms-auth.web.cern.ch/,	type: OIDC
Identity: SUB=dciangot,ISS=https://auth.cern.ch/auth/realms/cern,	type: OIDC

[vkuznet]

@dciangot I update aps codebase to provide real access token (turns out the token was rawID one), and you can test it using https://cmsweb-test1.cern.ch/token It shows now

scope": "openid profile email",

and other attributes. But please try it out and confirm

[dciangot]

@vkuznet I confirm. The endpoint is returning a "good" token.

Also I have tried the setup on my own (for cms-iam only) and it's working in both x509 and oidc mode. So I'm moving this pr out of draft