ElixirRetry

Simple Elixir macros for linear retry, exponential backoff and wait with composable delays.

Installation

Add retry to your list of dependencies in mix.exs:

  def deps do
    [{:retry, "~> 0.18"}]
  end

Ensure retry is started before your application:

  def application do
    [applications: [:retry]]
  end

Documentation

Check out the API reference for the latest documentation.

Features

Retrying

The retry([with: _,] do: _, after: _, else: _) macro provides a way to retry a block of code on failure with a variety of delay and give up behaviors. By default, the execution of a block is considered a failure if it returns :error, {:error, _} or raises a runtime error.

An optional list of atoms can be specified in :atoms if you need to retry anything other than :error or {:error, _}, e.g. retry([with: _, atoms: [:not_ok]], do: _, after: _, else: _).

Similarly, an optional list of exceptions can be specified in :rescue_only if you need to retry anything other than RuntimeError, e.g. retry([with: _, rescue_only: [CustomError]], do: _, after: _, else: _).

The after block evaluates only when the do block returns a valid value before timeout. On the other hand, the else block evaluates only when the do block remains erroneous after timeout. Both are optional. By default, the else clause will return the last erroneous value or re-raise the last exception. The default after clause will simply return the last successful value.

Example -- constant backoff

result = retry with: constant_backoff(100) |> Stream.take(10) do
  ExternalApi.do_something # fails if other system is down
after
  result -> result
else
  error -> error
end

This example retries every 100 milliseconds and gives up after 10 attempts.

Example -- linear backoff

result = retry with: linear_backoff(10, 2) |> cap(1_000) |> Stream.take(10) do
  ExternalApi.do_something # fails if other system is down
after
  result -> result
else
  error -> error
end

This example increases the delay linearly with each retry, starting with 10 milliseconds, caps the delay at 1 second and gives up after 10 attempts.

Example -- exponential backoff

result = retry with: exponential_backoff() |> randomize |> expiry(10_000), rescue_only: [TimeoutError] do
  ExternalApi.do_something # fails if other system is down
after
  result -> result
else
  error -> error
end

Example -- optional clauses

result = retry with: constant_backoff(100) |> Stream.take(10) do
  ExternalApi.do_something # fails if other system is down
end

This example is equivalent to:

result = retry with: constant_backoff(100) |> Stream.take(10) do
  ExternalApi.do_something # fails if other system is down
after
  result -> result
else
  {e, stacktrace} when is_exception(e) -> reraise e, stacktrace
  e -> e
end

Example -- retry annotation

use Retry.Annotation

@retry with: constant_backoff(100) |> Stream.take(10)
def some_func(arg) do
  ExternalApi.do_something # fails if other system is down
end

This example shows how you can annotate a function to retry every 100 milliseconds and gives up after 10 attempts.

Delay streams

The with: option of retry accepts any Stream that yields integers. These integers will be interpreted as the amount of time to delay before retrying a failed operation. When the stream is exhausted retry will give up, returning the last value of the block.

Example

result = retry with: Stream.cycle([500]) do
  ExternalApi.do_something # fails if other system is down
after
  result -> result
else
  error -> error  
end

This will retry failures forever, waiting 0.5 seconds between attempts.

Retry.DelayStreams provides a set of fully composable helper functions for building useful delay behaviors such as the ones in previous examples. See the Retry.DelayStreams module docs for full details and addition behavior not covered here. For convenience these functions are imported by use Retry so you can, usually, use them without prefixing them with the module name.

Waiting

Similar to retry(with: _, do: _), the wait(delay_stream, do: _, after: _, else: _) macro provides a way to wait for a block of code to be truthy with a variety of delay and give up behaviors. The execution of a block is considered a failure if it returns false or nil.

wait constant_backoff(100) |> expiry(1_000) do
  we_there_yet?
after
  _ ->
    {:ok, "We have arrived!"}
else
  _ ->
    {:error, "We're still on our way :("}
end

This example retries every 100 milliseconds and expires after 1 second.

The after block evaluates only when the do block returns a truthy value. On the other hand, the else block evaluates only when the do block remains falsy after timeout. Both are optional. By default, a success value will be returned as {:ok, value} and an erroneous value will be returned as {:error, value}.

Pretty nifty for those pesky asynchronous tests and building more reliable systems in general!