From a639e64f2a07d91decd4615fa5a825ebafe68cab Mon Sep 17 00:00:00 2001 From: Hadley Wickham Date: Sat, 26 Aug 2023 14:37:53 -0500 Subject: [PATCH 1/5] Create specific OAuth vignette --- NEWS.md | 3 + _pkgdown.yml | 7 ++ vignettes/articles/oauth.Rmd | 149 +++++++++++++++++++++++++++ vignettes/articles/wrapping-apis.Rmd | 137 ------------------------ 4 files changed, 159 insertions(+), 137 deletions(-) create mode 100644 vignettes/articles/oauth.Rmd diff --git a/NEWS.md b/NEWS.md index 7b6de8e6..cc886cce 100644 --- a/NEWS.md +++ b/NEWS.md @@ -1,5 +1,8 @@ # httr2 (development version) +* New `vignette("oauth")` makes the details of OAuth usage easier to find + (#234). + * `req_perform()` now throws error with class `httr2_failure` if the request fails. And that error now captures the curl error as the parent. diff --git a/_pkgdown.yml b/_pkgdown.yml index 18cc3d9d..a5700a35 100644 --- a/_pkgdown.yml +++ b/_pkgdown.yml @@ -70,3 +70,10 @@ reference: contents: - starts_with("oauth_") - starts_with("jwt_") + +articles: +- title: Using httr2 + navbar: ~ + contents: + - articles/wrapping-apis + - articles/oauth diff --git a/vignettes/articles/oauth.Rmd b/vignettes/articles/oauth.Rmd new file mode 100644 index 00000000..75232b0f --- /dev/null +++ b/vignettes/articles/oauth.Rmd @@ -0,0 +1,149 @@ +--- +title: "OAuth" +--- + +```{r, include = FALSE} +knitr::opts_chunk$set( + collapse = TRUE, + comment = "#>" +) +``` + +```{r setup} +library(httr2) +``` + +If the API provides access to a website where the user already has an account (think Twitter, Instagram, Facebook, Google, GitHub, etc), it's likely to use OAuth to allow you to authorise on behalf of the user. +OAuth[^1] is an authorisation framework that's designed so that you don't have to share your username and password with an app; instead the app asks for permission to use your account. +You've almost certainly used this before on the web; it's used in most cases where one website wants to use another website on your behalf. + +[^1]: Here I'll only talk about OAuth 2.0 which is the only version in common use today. + OAuth 1.0 is largely only of historical interest. + +OAuth is a broad framework that has many many many different variants which makes it hard to provide generalisable advice. +The following advice draws on my experience working with a number of OAuth using APIs, but don't be surprised if you need to do something slightly different for the API you're working with. + +## Clients + +The first step in working with any OAuth API is to create a client. +This involves you registering for a developer account on the API's website and creating a new OAuth app. +The process varies from API to API, but at the end of it you'll get a client id and in most cases a client secret. + +(You'll definitely need this for testing your package, and you'll probably also baked it into your package for the convenience of your users. Bundling the app is user friendly, but not always possible, particularly if rate limits are enforced on a per-app rather than per-user basis. You should always provide some way for the user to provide their own app.) + +If the API provides a way to authenticate your app without the client secret, you should leave it out of your package. +But in most cases, you'll need to include the secret in the package. +You can use `obfuscate()` to hide the secret; this is not bulletproof[^2], but in most cases it'll be easier to create a new client than try and steal yours. +Additionally, it's unusual for an OAuth client to be able to do anything in its own right, so even if someone does steal your secret there's not much harm they can do with it. + +[^2]: It uses `secret_encrypt()` with a special encryption key that's bundled with httr2. + +To obfuscate a secret, call `obfuscate()`: + +```{r} +obfuscate("secret") +``` + +Then use the client id from the website along with the obfuscated secret to create a client. +The following code shows a GitHub OAuth app that I created specifically for this vignette: + +```{r} +client <- oauth_client( + id = "28acfec0674bb3da9f38", + secret = obfuscated("J9iiGmyelHltyxqrHXW41ZZPZamyUNxSX1_uKnvPeinhhxET_7FfUs2X0LLKotXY2bpgOMoHRCo"), + token_url = "https://github.com/login/oauth/access_token", + name = "hadley-oauth-test" +) +``` + +You need to figure out the `token_url` from the [documentation](https://docs.github.com/en/developers/apps/building-oauth-apps/authorizing-oauth-apps). +I wish I could give good advice about how to find it 😞. + +Note that if you print the client the secret is automatically redacted: + +```{r} +client +``` + +## Flows + +Once you have a client you need to use it with a **flow** in order to get a token. +OAuth provides a number of different "flows", the most common is the "authorisation code" flow, which is implemented by `req_oauth_auth_code()`. +You can try it out by running this code: + +```{r, eval = FALSE} +token <- oauth_flow_auth_code(client, auth_url = "https://github.com/login/oauth/authorize") +``` + +This flow can't be used inside a vignette because it's designed specifically for interactive use: it will open a webpage on GitHub that requires you to interactively confirm it's OK for this app to use your GitHub account. + +Other flows provide different ways of getting the token: + +- `req_oauth_client_credentials()` is used to allow the client to perform actions on its own behalf (instead of on behalf of some other user). + This is typically need if you want to support **service accounts**, which are used in non-interactive environments. + +- `req_oauth_device()` uses the "device" flow which is designed for devices like TVs that don't have an easy way to enter data. + It also works well from the console. + +- `req_oauth_bearer_jwt()` uses a JWT signed by a private key. + +- `req_oauth_password()` exchanges a user name and password for an access token. + +- `req_oauth_refresh()` works directly with a refresh token that you already have. + It's useful for testing. + +There's one historically important OAuth flow that httr2 doesn't support: the implicit grant flow. +This is now [mostly deprecated](https://developer.okta.com/blog/2019/05/01/is-the-oauth-implicit-flow-dead) and was never a particularly good fit for native applications because it relies on a technique for returning the access token that only works inside a web browser. + +When wrapping an API, you'll need to carefully read the documentation to figure out which flows are available. +Typically you'll want to use the auth code flow, but if it's not available you'll need to carefully consider the others. +An additional wrinkle is that many APIs don't implement the flow in exactly the same way as the spec. +If your initial attempt doesn't work, you're going to need to do some sleuthing. +This is going to be painful, but unfortunately there's no way around it. +I recommend using `with_verbosity()` so you can see exactly what httr2 is sending to the server. +You'll then need to carefully compare this to the API documentation and play "spot the difference". + +## Tokens + +The point of a flow is to get a token. +You can use `req_auth_bearer_token()` to authorise a request with the access token stored inside the token object: + +```{r, eval = FALSE} +request("https://api.github.com/user") %>% + req_auth_bearer_token(token$access_token) %>% + req_perform() %>% + resp_body_json() %>% + .$name +#> [1] "Hadley Wickham" +``` + +However, in most cases you won't want to do this, but instead allow httr2 to manage the whole process, by switching from `oauth_flow_{name}` to `req_oauth_{name}`: + +```{r, eval = FALSE} +request("https://api.github.com/user") %>% + req_oauth_auth_code(client, auth_url = "https://github.com/login/oauth/authorize") %>% + req_perform() %>% + resp_body_json() +``` + +This is important because most APIs provide only a short-lived access token that needs to be regularly refreshed using a longer-lived refresh token. +httr2 will automatically refresh the token if it's expired (i.e. its expiry date is in the past) or if the request errors with a 401 and there's an `invalid_token` error in the `WWW-authenticate` header. + +## Caching + +By default, `req_oauth_auth_code()` and friends will cache the token in memory, so that multiple requests in the same session all use the same token. +In some cases, you may want to save the token so that it's automatically used across sessions. +This is easy to do (just set `cache_disk = TRUE` in `req_oauth_auth_code()`) but you need to carefully consider the consequences of saving the user's credentials on disk. + +httr2 does the best it can to save these credentials securely. +They are stored in a local cache directory (`rappdirs::user_cache_dir("httr2"))` that should only be accessible to the current user, and are encrypted so they will be hard for any package other than httr2 to read. +However, there's no way to prevent other R code from using httr2 to access them, so if you do choose to cache tokens, you should inform the user and give them the ability to opt-out. + +You can see which clients have cached tokens by looking in the cache directory used by httr2: + +```{r} +dir(rappdirs::user_cache_dir("httr2"), recursive = TRUE) +``` + +httr2 automatically deletes any cached tokens that are older than 30 days whenever it's loaded. +This means that you'll need to re-auth at least once a month, but prevents tokens for hanging around on disk long after you've forgotten you created them. diff --git a/vignettes/articles/wrapping-apis.Rmd b/vignettes/articles/wrapping-apis.Rmd index cc4d4cb0..b4931080 100644 --- a/vignettes/articles/wrapping-apis.Rmd +++ b/vignettes/articles/wrapping-apis.Rmd @@ -748,140 +748,3 @@ req <- req_gist(token) %>% req %>% req_dry_run() req %>% req_perform() ``` - -## OAuth - -If the API provides access to a website where the user already has an account (think Twitter, Instagram, Facebook, Google, GitHub, etc), it's likely to use OAuth to allow you to authorise on behalf of the user. -OAuth[^2] is an authorisation framework that's designed so that you don't have to share your username and password with an app; instead the app asks for permission to use your account. -You've almost certainly used this before on the web; it's used in most cases where one website wants to use another website on your behalf. - -[^2]: Here I'll only talk about OAuth 2.0 which is the only version in common use today. - OAuth 1.0 is largely only of historical interest. - -OAuth is a broad framework that has many many many different variants which makes it hard to provide generalisable advice. -The following advice draws on my experience working with a number of OAuth using APIs, but don't be surprised if you need to do something slightly different for the API you're working with. - -### Clients - -The first step in working with any OAuth API is to create a client. -This involves you registering for a developer account on the API's website and creating a new OAuth app. -The process varies from API to API, but at the end of it you'll get a client id and in most cases a client secret. - -(You'll definitely need this for testing your package, and you'll probably also baked it into your package for the convenience of your users. Bundling the app is user friendly, but not always possible, particularly if rate limits are enforced on a per-app rather than per-user basis. You should always provide some way for the user to provide their own app.) - -If the API provides a way to authenticate your app without the client secret, you should leave it out of your package. -But in most cases, you'll need to include the secret in the package. -You can use `obfuscate()` to hide the secret; this is not bulletproof[^3], but in most cases it'll be easier to create a new client than try and steal yours. -Additionally, it's unusual for an OAuth client to be able to do anything in its own right, so even if someone does steal your secret there's not much harm they can do with it. - -[^3]: It uses `secret_encrypt()` with a special encryption key that's bundled with httr2. - -To obfuscate a secret, call `obfuscate()`: - -```{r} -obfuscate("secret") -``` - -Then use the client id from the website along with the obfuscated secret to create a client. -The following code shows a GitHub OAuth app that I created specifically for this vignette: - -```{r} -client <- oauth_client( - id = "28acfec0674bb3da9f38", - secret = obfuscated("J9iiGmyelHltyxqrHXW41ZZPZamyUNxSX1_uKnvPeinhhxET_7FfUs2X0LLKotXY2bpgOMoHRCo"), - token_url = "https://github.com/login/oauth/access_token", - name = "hadley-oauth-test" -) -``` - -You need to figure out the `token_url` from the [documentation](https://docs.github.com/en/developers/apps/building-oauth-apps/authorizing-oauth-apps). -I wish I could give good advice about how to find it 😞. - -Note that if you print the client the secret is automatically redacted: - -```{r} -client -``` - -### Flows - -Once you have a client you need to use it with a **flow** in order to get a token. -OAuth provides a number of different "flows", the most common is the "authorisation code" flow, which is implemented by `req_oauth_auth_code()`. -You can try it out by running this code: - -```{r, eval = FALSE} -token <- oauth_flow_auth_code(client, auth_url = "https://github.com/login/oauth/authorize") -``` - -This flow can't be used inside a vignette because it's designed specifically for interactive use: it will open a webpage on GitHub that requires you to interactively confirm it's OK for this app to use your GitHub account. - -Other flows provide different ways of getting the token: - -- `req_oauth_client_credentials()` is used to allow the client to perform actions on its own behalf (instead of on behalf of some other user). - This is typically need if you want to support **service accounts**, which are used in non-interactive environments. - -- `req_oauth_device()` uses the "device" flow which is designed for devices like TVs that don't have an easy way to enter data. - It also works well from the console. - -- `req_oauth_bearer_jwt()` uses a JWT signed by a private key. - -- `req_oauth_password()` exchanges a user name and password for an access token. - -- `req_oauth_refresh()` works directly with a refresh token that you already have. - It's useful for testing. - -There's one historically important OAuth flow that httr2 doesn't support: the implicit grant flow. -This is now [mostly deprecated](https://developer.okta.com/blog/2019/05/01/is-the-oauth-implicit-flow-dead) and was never a particularly good fit for native applications because it relies on a technique for returning the access token that only works inside a web browser. - -When wrapping an API, you'll need to carefully read the documentation to figure out which flows are available. -Typically you'll want to use the auth code flow, but if it's not available you'll need to carefully consider the others. -An additional wrinkle is that many APIs don't implement the flow in exactly the same way as the spec. -If your initial attempt doesn't work, you're going to need to do some sleuthing. -This is going to be painful, but unfortunately there's no way around it. -I recommend using `with_verbosity()` so you can see exactly what httr2 is sending to the server. -You'll then need to carefully compare this to the API documentation and play "spot the difference". - -### Tokens - -The point of a flow is to get a token. -You can use `req_auth_bearer_token()` to authorise a request with the access token stored inside the token object: - -```{r, eval = FALSE} -request("https://api.github.com/user") %>% - req_auth_bearer_token(token$access_token) %>% - req_perform() %>% - resp_body_json() %>% - .$name -#> [1] "Hadley Wickham" -``` - -However, in most cases you won't want to do this, but instead allow httr2 to manage the whole process, by switching from `oauth_flow_{name}` to `req_oauth_{name}`: - -```{r, eval = FALSE} -request("https://api.github.com/user") %>% - req_oauth_auth_code(client, auth_url = "https://github.com/login/oauth/authorize") %>% - req_perform() %>% - resp_body_json() -``` - -This is important because most APIs provide only a short-lived access token that needs to be regularly refreshed using a longer-lived refresh token. -httr2 will automatically refresh the token if it's expired (i.e. its expiry date is in the past) or if the request errors with a 401 and there's an `invalid_token` error in the `WWW-authenticate` header. - -### Caching - -By default, `req_oauth_auth_code()` and friends will cache the token in memory, so that multiple requests in the same session all use the same token. -In some cases, you may want to save the token so that it's automatically used across sessions. -This is easy to do (just set `cache_disk = TRUE` in `req_oauth_auth_code()`) but you need to carefully consider the consequences of saving the user's credentials on disk. - -httr2 does the best it can to save these credentials securely. -They are stored in a local cache directory (`rappdirs::user_cache_dir("httr2"))` that should only be accessible to the current user, and are encrypted so they will be hard for any package other than httr2 to read. -However, there's no way to prevent other R code from using httr2 to access them, so if you do choose to cache tokens, you should inform the user and give them the ability to opt-out. - -You can see which clients have cached tokens by looking in the cache directory used by httr2: - -```{r} -dir(rappdirs::user_cache_dir("httr2"), recursive = TRUE) -``` - -httr2 automatically deletes any cached tokens that are older than 30 days whenever it's loaded. -This means that you'll need to re-auth at least once a month, but prevents tokens for hanging around on disk long after you've forgotten you created them. From f3e0e3c8245c83f57f8fc3fde914f67ddd7dfbd0 Mon Sep 17 00:00:00 2001 From: Hadley Wickham Date: Sat, 26 Aug 2023 14:38:12 -0500 Subject: [PATCH 2/5] Rename cache path helper for req_cache() --- R/req-cache.R | 10 +++++----- tests/testthat/test-req-cache.R | 2 +- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/R/req-cache.R b/R/req-cache.R index 1fff6983..87a3d8b9 100644 --- a/R/req-cache.R +++ b/R/req-cache.R @@ -53,7 +53,7 @@ req_cache <- function(req, path, use_on_error = FALSE, debug = FALSE) { # Do I need to worry about hash collisions? # No - even if the user stores a billion urls, the probably of a collision # is ~ 1e-20: https://preshing.com/20110504/hash-collision-probabilities/ -cache_path <- function(req, ext = ".rds") { +req_cache_path <- function(req, ext = ".rds") { file.path(req$policies$cache_path, paste0(hash(req$url), ext)) } cache_use_on_error <- function(req) { @@ -69,13 +69,13 @@ cache_exists <- function(req) { if (!req_policy_exists(req, "cache_path")) { FALSE } else { - file.exists(cache_path(req)) + file.exists(req_cache_path(req)) } } # Callers responsibility to check that cache exists cache_get <- function(req) { - path <- cache_path(req) + path <- req_cache_path(req) touch(path) readRDS(path) @@ -83,12 +83,12 @@ cache_get <- function(req) { cache_set <- function(req, resp) { if (is_path(resp$body)) { - body_path <- cache_path(req, ".body") + body_path <- req_cache_path(req, ".body") file.copy(resp$body, body_path, overwrite = TRUE) resp$body <- new_path(body_path) } - saveRDS(resp, cache_path(req, ".rds")) + saveRDS(resp, req_cache_path(req, ".rds")) invisible() } diff --git a/tests/testthat/test-req-cache.R b/tests/testthat/test-req-cache.R index 45b92430..669a02bb 100644 --- a/tests/testthat/test-req-cache.R +++ b/tests/testthat/test-req-cache.R @@ -145,7 +145,7 @@ test_that("handles responses with files", { cache_set(req, resp) # File should be copied in cache directory, and response body updated - body_path <- cache_path(req, ".body") + body_path <- req_cache_path(req, ".body") expect_equal(readLines(body_path), "Hi there") expect_equal(cache_get(req)$body, new_path(body_path)) From 37dd23e6915df7599f00cc8cdc613b26801799bf Mon Sep 17 00:00:00 2001 From: Hadley Wickham Date: Sat, 26 Aug 2023 14:51:58 -0500 Subject: [PATCH 3/5] Introduce new cache_path() function + env var --- NAMESPACE | 1 + NEWS.md | 3 +++ R/oauth-flow-auth-code.R | 4 ++-- R/oauth.R | 17 +++++++++++++++++ man/cache_path.Rd | 14 ++++++++++++++ man/req_oauth_auth_code.Rd | 4 ++-- man/req_oauth_device.Rd | 4 ++-- man/req_oauth_password.Rd | 4 ++-- tests/testthat/test-oauth.R | 7 +++++++ vignettes/articles/oauth.Rmd | 4 ++-- 10 files changed, 52 insertions(+), 10 deletions(-) create mode 100644 man/cache_path.Rd diff --git a/NAMESPACE b/NAMESPACE index 43884c9e..0e25958e 100644 --- a/NAMESPACE +++ b/NAMESPACE @@ -13,6 +13,7 @@ S3method(print,httr2_token) S3method(print,httr2_url) S3method(str,httr2_obfuscated) export("%>%") +export(cache_path) export(curl_help) export(curl_translate) export(example_url) diff --git a/NEWS.md b/NEWS.md index cc886cce..4a28b513 100644 --- a/NEWS.md +++ b/NEWS.md @@ -1,5 +1,8 @@ # httr2 (development version) +* New `cache_path()` that returns path to httr2's cache directory. Additionally, + you can now change the cache location by setting the `HTTR2_CACHE` env var. + * New `vignette("oauth")` makes the details of OAuth usage easier to find (#234). diff --git a/R/oauth-flow-auth-code.R b/R/oauth-flow-auth-code.R index 411f8ff5..983e8bb4 100644 --- a/R/oauth-flow-auth-code.R +++ b/R/oauth-flow-auth-code.R @@ -29,8 +29,8 @@ #' @inheritParams req_perform #' @param cache_disk Should the access token be cached on disk? This reduces #' the number of times that you need to re-authenticate at the cost of -#' storing access credentials on disk. Cached tokens are encrypted and -#' automatically deleted 30 days after creation. +#' storing access credentials on disk. Cached tokens are encrypted, +#' automatically deleted 30 days after creation, and stored in [cache_path()]. #' @param cache_key If you want to cache multiple tokens per app, use this #' key to disambiguate them. #' @returns A modified HTTP [request]. diff --git a/R/oauth.R b/R/oauth.R index 2adf2d7e..0d32687c 100644 --- a/R/oauth.R +++ b/R/oauth.R @@ -116,3 +116,20 @@ cache_disk_prune <- function(days = 30, path = rappdirs::user_cache_dir("httr2") old <- mtime < (Sys.time() - days * 86400) unlink(files[old]) } + +#' httr2 cache location +#' +#' When opted-in to, httr2 caches OAuth tokens in this directory. By default, +#' it uses a OS-standard cache directory, but, if needed, you can override the +#' location by setting the `HTTR2_CACHE` env var. +#' +#' @export +#' @keywords internal +cache_path <- function() { + path <- Sys.getenv("HTTR2_CACHE") + if (path != "") { + return(path) + } + + rappdirs::user_cache_dir("httr2") +} diff --git a/man/cache_path.Rd b/man/cache_path.Rd new file mode 100644 index 00000000..395970a4 --- /dev/null +++ b/man/cache_path.Rd @@ -0,0 +1,14 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/oauth.R +\name{cache_path} +\alias{cache_path} +\title{httr2 cache location} +\usage{ +cache_path() +} +\description{ +When opted-in to, httr2 caches OAuth tokens in this directory. By default, +it uses a OS-standard cache directory, but, if needed, you can override the +location by setting the \code{HTTR2_CACHE} env var. +} +\keyword{internal} diff --git a/man/req_oauth_auth_code.Rd b/man/req_oauth_auth_code.Rd index b0f4bdaa..26f7c96f 100644 --- a/man/req_oauth_auth_code.Rd +++ b/man/req_oauth_auth_code.Rd @@ -29,8 +29,8 @@ the documentation.} \item{cache_disk}{Should the access token be cached on disk? This reduces the number of times that you need to re-authenticate at the cost of -storing access credentials on disk. Cached tokens are encrypted and -automatically deleted 30 days after creation.} +storing access credentials on disk. Cached tokens are encrypted, +automatically deleted 30 days after creation, and stored in \code{\link[=cache_path]{cache_path()}}.} \item{cache_key}{If you want to cache multiple tokens per app, use this key to disambiguate them.} diff --git a/man/req_oauth_device.Rd b/man/req_oauth_device.Rd index cf7a4079..8222864a 100644 --- a/man/req_oauth_device.Rd +++ b/man/req_oauth_device.Rd @@ -21,8 +21,8 @@ req_oauth_device( \item{cache_disk}{Should the access token be cached on disk? This reduces the number of times that you need to re-authenticate at the cost of -storing access credentials on disk. Cached tokens are encrypted and -automatically deleted 30 days after creation.} +storing access credentials on disk. Cached tokens are encrypted, +automatically deleted 30 days after creation, and stored in \code{\link[=cache_path]{cache_path()}}.} \item{cache_key}{If you want to cache multiple tokens per app, use this key to disambiguate them.} diff --git a/man/req_oauth_password.Rd b/man/req_oauth_password.Rd index 7bc6b842..8ade91d3 100644 --- a/man/req_oauth_password.Rd +++ b/man/req_oauth_password.Rd @@ -28,8 +28,8 @@ interactively.} \item{cache_disk}{Should the access token be cached on disk? This reduces the number of times that you need to re-authenticate at the cost of -storing access credentials on disk. Cached tokens are encrypted and -automatically deleted 30 days after creation.} +storing access credentials on disk. Cached tokens are encrypted, +automatically deleted 30 days after creation, and stored in \code{\link[=cache_path]{cache_path()}}.} \item{scope}{Scopes to be requested from the resource owner.} diff --git a/tests/testthat/test-oauth.R b/tests/testthat/test-oauth.R index d7f8dd7c..03a6ece6 100644 --- a/tests/testthat/test-oauth.R +++ b/tests/testthat/test-oauth.R @@ -55,3 +55,10 @@ test_that("can prune old files", { cache_disk_prune(2, path) expect_equal(dir(path), "a-token.rds") }) + +# cache_path -------------------------------------------------------------- + +test_that("can override path with env var", { + withr::local_envvar("HTTR2_CACHE" = "/tmp") + expect_equal(cache_path(), "/tmp") +}) diff --git a/vignettes/articles/oauth.Rmd b/vignettes/articles/oauth.Rmd index 75232b0f..034f9c22 100644 --- a/vignettes/articles/oauth.Rmd +++ b/vignettes/articles/oauth.Rmd @@ -136,13 +136,13 @@ In some cases, you may want to save the token so that it's automatically used ac This is easy to do (just set `cache_disk = TRUE` in `req_oauth_auth_code()`) but you need to carefully consider the consequences of saving the user's credentials on disk. httr2 does the best it can to save these credentials securely. -They are stored in a local cache directory (`rappdirs::user_cache_dir("httr2"))` that should only be accessible to the current user, and are encrypted so they will be hard for any package other than httr2 to read. +They are stored in a local cache directory (`cache_path())` that should only be accessible to the current user, and are encrypted so they will be hard for any package other than httr2 to read. However, there's no way to prevent other R code from using httr2 to access them, so if you do choose to cache tokens, you should inform the user and give them the ability to opt-out. You can see which clients have cached tokens by looking in the cache directory used by httr2: ```{r} -dir(rappdirs::user_cache_dir("httr2"), recursive = TRUE) +dir(cache_path(), recursive = TRUE) ``` httr2 automatically deletes any cached tokens that are older than 30 days whenever it's loaded. From b8805bcd0d75a59a508f01627c6d22b434166800 Mon Sep 17 00:00:00 2001 From: Hadley Wickham Date: Sat, 26 Aug 2023 14:58:43 -0500 Subject: [PATCH 4/5] Link to vignette from docs --- R/oauth-flow-auth-code.R | 2 ++ R/oauth-flow-client-credentials.R | 2 ++ R/oauth-flow-device.R | 2 ++ R/oauth-flow-jwt.R | 2 ++ R/oauth-flow-password.R | 2 ++ R/oauth-flow-refresh.R | 2 ++ man/req_oauth_auth_code.Rd | 2 ++ man/req_oauth_bearer_jwt.Rd | 2 ++ man/req_oauth_client_credentials.Rd | 2 ++ man/req_oauth_device.Rd | 2 ++ man/req_oauth_password.Rd | 2 ++ man/req_oauth_refresh.Rd | 2 ++ 12 files changed, 24 insertions(+) diff --git a/R/oauth-flow-auth-code.R b/R/oauth-flow-auth-code.R index 983e8bb4..a8f416cc 100644 --- a/R/oauth-flow-auth-code.R +++ b/R/oauth-flow-auth-code.R @@ -6,6 +6,8 @@ #' The token is automatically cached (either in memory or on disk) to minimise #' the number of times the flow is performed. #' +#' Learn more about the overall flow in `vignette("oauth")`. +#' #' # Security considerations #' #' The authorization code flow is used for both web applications and native diff --git a/R/oauth-flow-client-credentials.R b/R/oauth-flow-client-credentials.R index b24b7ff8..914dcce3 100644 --- a/R/oauth-flow-client-credentials.R +++ b/R/oauth-flow-client-credentials.R @@ -5,6 +5,8 @@ #' which is then used to authentication the request with [req_auth_bearer_token()]. #' The token is cached in memory. #' +#' Learn more about the overall flow in `vignette("oauth")`. +#' #' @export #' @inheritParams req_perform #' @inheritParams oauth_flow_client_credentials diff --git a/R/oauth-flow-device.R b/R/oauth-flow-device.R index 62f3cdca..7d46f4e7 100644 --- a/R/oauth-flow-device.R +++ b/R/oauth-flow-device.R @@ -6,6 +6,8 @@ #' The token is automatically cached (either in memory or on disk) to minimise #' the number of times the flow is performed. #' +#' Learn more about the overall flow in `vignette("oauth")`. +#' #' @export #' @inheritParams oauth_flow_password #' @inheritParams req_oauth_auth_code diff --git a/R/oauth-flow-jwt.R b/R/oauth-flow-jwt.R index 5e8ee781..8a79f668 100644 --- a/R/oauth-flow-jwt.R +++ b/R/oauth-flow-jwt.R @@ -5,6 +5,8 @@ #' used to authenticate the request with [req_auth_bearer_token()]. #' The token is cached in memory. #' +#' Learn more about the overall flow in `vignette("oauth")`. +#' #' @export #' @inheritParams req_perform #' @inheritParams oauth_flow_bearer_jwt diff --git a/R/oauth-flow-password.R b/R/oauth-flow-password.R index 79d0f508..27e7e131 100644 --- a/R/oauth-flow-password.R +++ b/R/oauth-flow-password.R @@ -7,6 +7,8 @@ #' or on disk); the password is used once to get the token and is then #' discarded. #' +#' Learn more about the overall flow in `vignette("oauth")`. +#' #' @export #' @inheritParams oauth_flow_password #' @inheritParams req_oauth_auth_code diff --git a/R/oauth-flow-refresh.R b/R/oauth-flow-refresh.R index aeabe0b8..b5a5859f 100644 --- a/R/oauth-flow-refresh.R +++ b/R/oauth-flow-refresh.R @@ -12,6 +12,8 @@ #' token. If this happens, `oauth_flow_refresh()` will warn, and you'll have to #' update your stored refresh token. #' +#' Learn more about the overall flow in `vignette("oauth")`. +#' #' @export #' @inheritParams req_perform #' @inheritParams oauth_flow_refresh diff --git a/man/req_oauth_auth_code.Rd b/man/req_oauth_auth_code.Rd index 26f7c96f..6bd6611f 100644 --- a/man/req_oauth_auth_code.Rd +++ b/man/req_oauth_auth_code.Rd @@ -62,6 +62,8 @@ This uses \code{\link[=oauth_flow_auth_code]{oauth_flow_auth_code()}} to generat then used to authentication the request with \code{\link[=req_auth_bearer_token]{req_auth_bearer_token()}}. The token is automatically cached (either in memory or on disk) to minimise the number of times the flow is performed. + +Learn more about the overall flow in \code{vignette("oauth")}. } \section{Security considerations}{ The authorization code flow is used for both web applications and native diff --git a/man/req_oauth_bearer_jwt.Rd b/man/req_oauth_bearer_jwt.Rd index 5497ab4b..8c2cd2c3 100644 --- a/man/req_oauth_bearer_jwt.Rd +++ b/man/req_oauth_bearer_jwt.Rd @@ -42,6 +42,8 @@ A modified HTTP \link{request}. This uses \code{\link[=oauth_flow_bearer_jwt]{oauth_flow_bearer_jwt()}} to generate an access token which is then used to authenticate the request with \code{\link[=req_auth_bearer_token]{req_auth_bearer_token()}}. The token is cached in memory. + +Learn more about the overall flow in \code{vignette("oauth")}. } \examples{ client <- oauth_client("example", "https://example.com/get_token") diff --git a/man/req_oauth_client_credentials.Rd b/man/req_oauth_client_credentials.Rd index 80536602..b6ff9159 100644 --- a/man/req_oauth_client_credentials.Rd +++ b/man/req_oauth_client_credentials.Rd @@ -23,6 +23,8 @@ A modified HTTP \link{request}. This uses \code{\link[=oauth_flow_client_credentials]{oauth_flow_client_credentials()}} to generate an access token, which is then used to authentication the request with \code{\link[=req_auth_bearer_token]{req_auth_bearer_token()}}. The token is cached in memory. + +Learn more about the overall flow in \code{vignette("oauth")}. } \examples{ client <- oauth_client("example", "https://example.com/get_token") diff --git a/man/req_oauth_device.Rd b/man/req_oauth_device.Rd index 8222864a..bf5a9a2a 100644 --- a/man/req_oauth_device.Rd +++ b/man/req_oauth_device.Rd @@ -42,6 +42,8 @@ This uses \code{\link[=oauth_flow_device]{oauth_flow_device()}} to generate an a then used to authentication the request with \code{\link[=req_auth_bearer_token]{req_auth_bearer_token()}}. The token is automatically cached (either in memory or on disk) to minimise the number of times the flow is performed. + +Learn more about the overall flow in \code{vignette("oauth")}. } \examples{ client <- oauth_client("example", "https://example.com/get_token") diff --git a/man/req_oauth_password.Rd b/man/req_oauth_password.Rd index 8ade91d3..9efc8b33 100644 --- a/man/req_oauth_password.Rd +++ b/man/req_oauth_password.Rd @@ -45,6 +45,8 @@ then used to authentication the request with \code{\link[=req_auth_bearer_token] The token, not the password is automatically cached (either in memory or on disk); the password is used once to get the token and is then discarded. + +Learn more about the overall flow in \code{vignette("oauth")}. } \examples{ client <- oauth_client("example", "https://example.com/get_token") diff --git a/man/req_oauth_refresh.Rd b/man/req_oauth_refresh.Rd index 61f102dd..1f40a79d 100644 --- a/man/req_oauth_refresh.Rd +++ b/man/req_oauth_refresh.Rd @@ -41,6 +41,8 @@ variable for future use in automated tests. When requesting an access token, the server may also return a new refresh token. If this happens, \code{oauth_flow_refresh()} will warn, and you'll have to update your stored refresh token. + +Learn more about the overall flow in \code{vignette("oauth")}. } \examples{ client <- oauth_client("example", "https://example.com/get_token") From cae3d4e03fc7c11896dbc82d265d3e07a8c5fd43 Mon Sep 17 00:00:00 2001 From: Hadley Wickham Date: Thu, 31 Aug 2023 15:44:10 -0500 Subject: [PATCH 5/5] Rename to oauth_cache_path() And actually use it :| --- NAMESPACE | 2 +- NEWS.md | 5 +++-- R/oauth-flow-auth-code.R | 3 ++- R/oauth.R | 13 ++++++------- man/{cache_path.Rd => oauth_cache_path.Rd} | 11 +++++------ man/req_oauth_auth_code.Rd | 3 ++- man/req_oauth_device.Rd | 3 ++- man/req_oauth_password.Rd | 3 ++- tests/testthat/test-oauth.R | 4 ++-- vignettes/articles/oauth.Rmd | 4 ++-- 10 files changed, 27 insertions(+), 24 deletions(-) rename man/{cache_path.Rd => oauth_cache_path.Rd} (62%) diff --git a/NAMESPACE b/NAMESPACE index 0e25958e..5c211b63 100644 --- a/NAMESPACE +++ b/NAMESPACE @@ -13,7 +13,6 @@ S3method(print,httr2_token) S3method(print,httr2_url) S3method(str,httr2_obfuscated) export("%>%") -export(cache_path) export(curl_help) export(curl_translate) export(example_url) @@ -24,6 +23,7 @@ export(last_request) export(last_response) export(local_mock) export(multi_req_perform) +export(oauth_cache_path) export(oauth_client) export(oauth_client_req_auth) export(oauth_client_req_auth_body) diff --git a/NEWS.md b/NEWS.md index 4a28b513..219808bd 100644 --- a/NEWS.md +++ b/NEWS.md @@ -1,7 +1,8 @@ # httr2 (development version) -* New `cache_path()` that returns path to httr2's cache directory. Additionally, - you can now change the cache location by setting the `HTTR2_CACHE` env var. +* New `oauth_cache_path()` returns the path that httr2 uses for caching OAuth + tokens. Additionally, you can now change the cache location by setting the + `HTTR2_OAUTH_CACHE` env var. * New `vignette("oauth")` makes the details of OAuth usage easier to find (#234). diff --git a/R/oauth-flow-auth-code.R b/R/oauth-flow-auth-code.R index a8f416cc..aa91e82c 100644 --- a/R/oauth-flow-auth-code.R +++ b/R/oauth-flow-auth-code.R @@ -32,7 +32,8 @@ #' @param cache_disk Should the access token be cached on disk? This reduces #' the number of times that you need to re-authenticate at the cost of #' storing access credentials on disk. Cached tokens are encrypted, -#' automatically deleted 30 days after creation, and stored in [cache_path()]. +#' automatically deleted 30 days after creation, and stored in +#' [oauth_cache_path()]. #' @param cache_key If you want to cache multiple tokens per app, use this #' key to disambiguate them. #' @returns A modified HTTP [request]. diff --git a/R/oauth.R b/R/oauth.R index 0d32687c..2012f443 100644 --- a/R/oauth.R +++ b/R/oauth.R @@ -97,7 +97,7 @@ cache_mem <- function(client, key) { ) } cache_disk <- function(client, key) { - app_path <- file.path(rappdirs::user_cache_dir("httr2"), client$name) + app_path <- file.path(oauth_cache_path(), client$name) dir.create(app_path, showWarnings = FALSE, recursive = TRUE) path <- file.path(app_path, paste0(hash(key), "-token.rds.enc")) @@ -109,7 +109,7 @@ cache_disk <- function(client, key) { } # Update req_oauth_auth_code() docs if change default from 30 -cache_disk_prune <- function(days = 30, path = rappdirs::user_cache_dir("httr2")) { +cache_disk_prune <- function(days = 30, path = oauth_cache_path()) { files <- dir(path, recursive = TRUE, full.names = TRUE, pattern = "-token\\.rds$") mtime <- file.mtime(files) @@ -117,16 +117,15 @@ cache_disk_prune <- function(days = 30, path = rappdirs::user_cache_dir("httr2") unlink(files[old]) } -#' httr2 cache location +#' httr2 OAuth cache location #' #' When opted-in to, httr2 caches OAuth tokens in this directory. By default, #' it uses a OS-standard cache directory, but, if needed, you can override the -#' location by setting the `HTTR2_CACHE` env var. +#' location by setting the `HTTR2_OAUTH_CACHE` env var. #' #' @export -#' @keywords internal -cache_path <- function() { - path <- Sys.getenv("HTTR2_CACHE") +oauth_cache_path <- function() { + path <- Sys.getenv("HTTR2_OAUTH_CACHE") if (path != "") { return(path) } diff --git a/man/cache_path.Rd b/man/oauth_cache_path.Rd similarity index 62% rename from man/cache_path.Rd rename to man/oauth_cache_path.Rd index 395970a4..87559962 100644 --- a/man/cache_path.Rd +++ b/man/oauth_cache_path.Rd @@ -1,14 +1,13 @@ % Generated by roxygen2: do not edit by hand % Please edit documentation in R/oauth.R -\name{cache_path} -\alias{cache_path} -\title{httr2 cache location} +\name{oauth_cache_path} +\alias{oauth_cache_path} +\title{httr2 OAuth cache location} \usage{ -cache_path() +oauth_cache_path() } \description{ When opted-in to, httr2 caches OAuth tokens in this directory. By default, it uses a OS-standard cache directory, but, if needed, you can override the -location by setting the \code{HTTR2_CACHE} env var. +location by setting the \code{HTTR2_OAUTH_CACHE} env var. } -\keyword{internal} diff --git a/man/req_oauth_auth_code.Rd b/man/req_oauth_auth_code.Rd index 6bd6611f..cac20ebb 100644 --- a/man/req_oauth_auth_code.Rd +++ b/man/req_oauth_auth_code.Rd @@ -30,7 +30,8 @@ the documentation.} \item{cache_disk}{Should the access token be cached on disk? This reduces the number of times that you need to re-authenticate at the cost of storing access credentials on disk. Cached tokens are encrypted, -automatically deleted 30 days after creation, and stored in \code{\link[=cache_path]{cache_path()}}.} +automatically deleted 30 days after creation, and stored in +\code{\link[=oauth_cache_path]{oauth_cache_path()}}.} \item{cache_key}{If you want to cache multiple tokens per app, use this key to disambiguate them.} diff --git a/man/req_oauth_device.Rd b/man/req_oauth_device.Rd index bf5a9a2a..59eadefe 100644 --- a/man/req_oauth_device.Rd +++ b/man/req_oauth_device.Rd @@ -22,7 +22,8 @@ req_oauth_device( \item{cache_disk}{Should the access token be cached on disk? This reduces the number of times that you need to re-authenticate at the cost of storing access credentials on disk. Cached tokens are encrypted, -automatically deleted 30 days after creation, and stored in \code{\link[=cache_path]{cache_path()}}.} +automatically deleted 30 days after creation, and stored in +\code{\link[=oauth_cache_path]{oauth_cache_path()}}.} \item{cache_key}{If you want to cache multiple tokens per app, use this key to disambiguate them.} diff --git a/man/req_oauth_password.Rd b/man/req_oauth_password.Rd index 9efc8b33..388afb4c 100644 --- a/man/req_oauth_password.Rd +++ b/man/req_oauth_password.Rd @@ -29,7 +29,8 @@ interactively.} \item{cache_disk}{Should the access token be cached on disk? This reduces the number of times that you need to re-authenticate at the cost of storing access credentials on disk. Cached tokens are encrypted, -automatically deleted 30 days after creation, and stored in \code{\link[=cache_path]{cache_path()}}.} +automatically deleted 30 days after creation, and stored in +\code{\link[=oauth_cache_path]{oauth_cache_path()}}.} \item{scope}{Scopes to be requested from the resource owner.} diff --git a/tests/testthat/test-oauth.R b/tests/testthat/test-oauth.R index 03a6ece6..99eebf5c 100644 --- a/tests/testthat/test-oauth.R +++ b/tests/testthat/test-oauth.R @@ -59,6 +59,6 @@ test_that("can prune old files", { # cache_path -------------------------------------------------------------- test_that("can override path with env var", { - withr::local_envvar("HTTR2_CACHE" = "/tmp") - expect_equal(cache_path(), "/tmp") + withr::local_envvar("HTTR2_OAUTH_CACHE" = "/tmp") + expect_equal(oauth_cache_path(), "/tmp") }) diff --git a/vignettes/articles/oauth.Rmd b/vignettes/articles/oauth.Rmd index 034f9c22..b9cb6c4f 100644 --- a/vignettes/articles/oauth.Rmd +++ b/vignettes/articles/oauth.Rmd @@ -136,13 +136,13 @@ In some cases, you may want to save the token so that it's automatically used ac This is easy to do (just set `cache_disk = TRUE` in `req_oauth_auth_code()`) but you need to carefully consider the consequences of saving the user's credentials on disk. httr2 does the best it can to save these credentials securely. -They are stored in a local cache directory (`cache_path())` that should only be accessible to the current user, and are encrypted so they will be hard for any package other than httr2 to read. +They are stored in a local cache directory (`oauth_cache_path())` that should only be accessible to the current user, and are encrypted so they will be hard for any package other than httr2 to read. However, there's no way to prevent other R code from using httr2 to access them, so if you do choose to cache tokens, you should inform the user and give them the ability to opt-out. You can see which clients have cached tokens by looking in the cache directory used by httr2: ```{r} -dir(cache_path(), recursive = TRUE) +dir(oauth_cache_path(), recursive = TRUE) ``` httr2 automatically deletes any cached tokens that are older than 30 days whenever it's loaded.