Isolated Mode - Support non-AzureWebJobsStorage storage

[lanni-energinet]

Our organization follows azure functions recommendations from https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-azure-storage-provider#storage-account-selection

If not specified, the default AzureWebJobsStorage storage account is used. For performance-sensitive workloads, however, configuring a non-default storage account is recommended. Durable Functions uses Azure Storage heavily, and using a dedicated storage account isolates Durable Functions storage usage from the internal usage by the Azure Functions host.

When using Storage Account as provider type, this is done by configuring host.json in the following way

{
  "extensions": {
    "durableTask": {
      "storageProvider": {
        "connectionStringName": "MyStorageAccountAppSetting"
      }
    }
  }
}

The issue:

When DurableTaskClient is parameter bound in DfmGetOrchestrationsFunction(Orchestrations.cs), the correct connectionString and storage type for MyStorageAccountAppSetting is resolved, as this is done through DTFx and durableClient.ListAllInstances(...) successfully queries the storage.

Later steps uses Globals.GetFullConnectionStringEnvVariableName(...) which has an deficient implementation of connectionString resolution as it either returns AzureWebJobsStorage, which differs from what DurableTaskClient has resolved, or returns a DFM_ALTERNATIVE_CONNECTION_STRING which is currently unsupported in "Isolated" mode.

Due to isolation requirements it is not feasible for us to deploy in "standalone" mode

Can you advise on how to move forward?

[scale-tone]

Hi @lanni-energinet , just to clarify: what exact scenario(s) do not work in your setup?
Are you able to see the (correct) list of instances?
Do you see instance details/history?
Are you getting any errors?

[lanni-energinet]

@scale-tone

Yes the list of instances are mostly correct but the column customStatus is populated with the value null due to the TableClient querying the wrong storae. Clicking on a instance results in a System.NullReferenceException

[scale-tone]

That's not good indeed, and needs to be fixed.
Thanks for reporting, @lanni-energinet !

[lanni-energinet]

@scale-tone

Thank you for your quick response.

I know its not an optimal fix, but would it be possible to change Globals.GetFullConnectionStringEnvVariableName to the following implementation in a beta release. I am quite confident that this implementation would fix the issue for now, the more permanent solution would be to remove connName, and figure out a mechanism to retrieve the connection from DurableTaskClient

        public static string GetFullConnectionStringEnvVariableName(string connName)
        {
            string hostJsonFileName = GetHostJsonPath();
            dynamic hostJson = JObject.Parse(File.ReadAllText(hostJsonFileName));

            return hostJson?.extensions?.durableTask?.storageProvider?.connectionStringName?.ToString() ?? EnvVariableNames.AzureWebJobsStorage;
        }

[scale-tone]

Pushed a fix in form of beta packages:
https://www.nuget.org/packages/DurableFunctionsMonitor.DotNetBackend/6.5.2-beta1
https://www.nuget.org/packages/DurableFunctionsMonitor.DotNetIsolated/6.5.2-beta1

@lanni-energinet , can I ask you to check if it works for you?

[lanni-energinet]

@scale-tone

We just checked and 6.5.2-beta1 works for us.

Thank you for the quick reponse

[lanni-energinet]

@scale-tone

Can you give a timeframe for a new "stable" release, our organization follows a policy of avoiding prerelease packages