Targeting Net8.0 (Isolated) and using Durable Functions Orchestration within an Azure Functions project causes NuGet to restore old vulnerable packages

[mikejohnstonPremierinc]

Description

NuGet is restoring non-targeted old versions of packages for this specific azure functions project.

For example, if I add <PackageReference Include="Newtonsoft.Json" Version="13.0.3" /> to my .csproj file, NuGet should ONLY restore 13.0.3.

Instead, it is restoring 9.0.1, 10.0.1, 10.0.2, 11.0.2 13.0.1

Cause
I originally thought the cause was bug 13943. 13943 bug was first seen in the new version of the NuGet Dependency Resolver that came packaged with the new version of Visual Studio.

I implemented the workaround for 13943. The workaround is to use the legacy resolver, but my issue persisted. Old packages were still being restored.

I then downgraded Visual Studio to earlier versions, and my issue still persisted.

I posted a bug on the NuGet repository myself, as shown Here. But a developer told me that the issue is not caused by a bug within the NuGet code, but instead it is a bug within the azure functions code, as explained Here.

Specifically:

What's happening in Azure Functions apps is that the SDK is generating a .csproj on the fly in the obj folder:

<Project Sdk="Microsoft.NET.Sdk">
    <PropertyGroup>
        <TargetFramework>net6.0</TargetFramework>
        <Configuration>Release</Configuration>
        <AssemblyName>Microsoft.Azure.Functions.Worker.Extensions</AssemblyName>
        <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
    </PropertyGroup>

    <ItemGroup>
        <PackageReference Include="Microsoft.NETCore.Targets" Version="3.0.0" PrivateAssets="all" />
        <PackageReference Include="Microsoft.NET.Sdk.Functions" Version="4.3.0" />
        <PackageReference Include="Microsoft.Azure.WebJobs.Extensions.DurableTask" Version="2.13.1" />
    </ItemGroup>

    <Target Name="_VerifyTargetFramework" BeforeTargets="Build">
        <!-- It is possible to override our TFM via global properties. This can lead to successful builds, but runtime errors due to incompatible dependencies being brought in. -->
        <Error Condition="'$(TargetFramework)' != 'net6.0'" Text="The target framework '$(TargetFramework)' must be 'net6.0'. Verify if target framework has been overridden by a global property." />
    </Target>
</Project>

Its then restoring the project itself:

This generated project contains a transitive reference to a different Newtonsoft.Json which is why NuGet is installing it.

I'm not sure exactly how this SDK works but you can see if there's an existing issue or file a new one at https://github.com/Azure/azure-functions-dotnet-worker/issues

Workarounds
There are no known workarounds

Steps to reproduce

1. Open Visual Studio, choose to create a new project
2. Search for Azure Functions. Choose it and click Next
3. In the Additional Information window, choose .NET 8.0 Isolated (Long Term Support) and Durable Functions Orchestration
   
4. Click Create
5. Double-click the csproj file and see that nuget restore adds a few packages
   
6. Build your project
7. Open File Explorer, look inside the .nuget packages newtonsoft folder. i.e. C:\Users\username\.nuget\packages\newtonsoft.json. You will notice that NuGet downloaded versions 9.0.1, 10.0.1, 10.0.2, 11.0.2 13.0.1
8. Delete ALL versions/folders from C:\Users\username\.nuget\packages\newtonsoft.json. The folder should now be empty.
9. Add this XML to your .csproj: <PackageReference Include="Newtonsoft.Json" Version="13.0.3" />
10. Adding the XML above should force NuGet to download ONLY 13.0.3, for any and all transitive dependencies.
11. Rebuild the project and notice that it does not download only 13.0.3. It incorrectly downloads all prior versions again

[jviau]

@mikejohnstonPremierinc have you tried using latest durable extension 1.2.1? https://www.nuget.org/packages/Microsoft.Azure.Functions.Worker.Extensions.DurableTask

[jviau]

Transferred to azure-functions-durable-extension to track this. With the current SDK the resolution is for the worker extension owners to release versions that bring in a WebJobs extension with transitive CVE's addressed.

[mikejohnstonPremierinc]

@mikejohnstonPremierinc have you tried using latest durable extension 1.2.1? https://www.nuget.org/packages/Microsoft.Azure.Functions.Worker.Extensions.DurableTask

@jviau I have not. Can you tell me the exact steps to try, then I will reply if it worked? Thanks!

[jviau]

@mikejohnstonPremierinc update the line <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.DurableTask" Version="..." /> to have a version of 1.2.1

[mikejohnstonPremierinc]

@jviau I have now tried this, and this did not work.

[mikejohnstonPremierinc]

@jviau I have a direct ticket with the Product Group team from Microsoft. Below is what they said. If this is the case, then how are vulnerable dependencies automatically addressed in a situation like this?

Azure Functions does not customize the restore behavior of .NET by default.

Customizations` of the inner build are not currently supported

[jviau]

7. Open File Explorer, look inside the .nuget packages newtonsoft folder. i.e. C:\Users\username.nuget\packages\newtonsoft.json. You will notice that NuGet downloaded versions 9.0.1, 10.0.1, 10.0.2, 11.0.2 13.0.1

Is this your root issue, that these other packages were downloaded? This behavior is how nuget is designed. It is not owned by the durable nor functions team. Nuget will first download all packages in the transitive closure, but it will ultimately only include one copy of that in your final build output.

[mikejohnstonPremierinc]

@jviau This is not how NuGet is designed. Please read this stating that when a user adds a direct dependency to the .csproj file, it should NOT download any other version. For other projects, this is what I see occurring and there is no issue with other projects. For this specific Functions project, this is NOT what I am seeing

[jviau]

As I have stated many times, this is NOT how NuGet is designed. I even shared an example with MANY screenshots illustrating this is the case. This example showed CLEARLY that any other App (The App was a simple Console App in the example) does not behave as you say:

You are not comparing the same scenarios. Your console app declares only a single Newtonsoft.Json package. Your function app has a large package closure, where different packages have their own dependency chain. These various dependency chains may point to differing versions of Newtonsoft.Json. The default nuget resolving will download all of these packages and cache them locally. Then unification is performed, reducing these conflicting asks of Newtonsoft.Json versions down to a single version (which in this case is most likely 13.0.1).

The linked issue mentions using <RestoreUseLegacyDependencyResolver>true</RestoreUseLegacyDependencyResolver> - configuring the WorkerExtension.csproj is not supported today. This PR aims to add support for that, but until then your issue in question is not supported and also not considered a bug by our team as this behavior is controlled by nuget, not us.

@jviau This is not how NuGet is designed. Please read NuGet/NuGet.Client#6188 stating that when a user adds a direct dependency to the .csproj file, it should NOT download any other version. For other projects, this is what I see occurring and there is no issue with other projects. For this specific Functions project, this is NOT what I am seeing

There is a second project in play with dotnet isolated function apps: WorkerExtensions.csproj. Which is used to build the extension bundle payload for the host. This is generated, restored, and built at compile time so we can package its output with your app. Today this project is not customizable and its contents are determined by the Worker.Extensions you have. This is where the multiple Newtonsoft.Json downloads come from.

[mikejohnstonPremierinc]

@jviau I see that the PR is about to be merged to main. Once this is merged and deployed, how exactly do I take advantage of this so that older versions of NewtonSoft are not downloaded?

[jviau]

@mikejohnstonPremierinc I'll include more samples /docs when that PR is checked. But the gist is you will add an msbuild property to your app and then use a command we provide to generate the initial WorkerExtensions.csproj on disk. You can then manually change that project as needed, such as adding a direct reference to Newtonsoft.Json with the version you want.

[mikejohnstonPremierinc]

@jviau Can you link me to the sample and docs so I can implement the direct reference to Newtonsoft.Json within WorkerExtensions.csproj?

[mikejohnstonPremierinc]

@jviau Can you see my last reply?

@jviau Can you link me to the sample and docs so I can implement the direct reference to Newtonsoft.Json within WorkerExtensions.csproj?