dotnet publish --no-restore fails during docker build even if the packages are already restored in a web app with dependent class library

[akhanalcs]

Describe the bug

dotnet publish --no-restore fails during docker build even if the packages are already restored. The app is a .NET 8 web api with a dependent class library.

To Reproduce

All the steps in reproducing this issue are documented in this minimalistic repro project:

https://github.com/akhanalcs/docker-with-classlib

In nutshell, the issue is that it's not possible to create a docker image without restoring the packages twice, once during dotnet restore and other during dotnet publish. If you use --no-restore flag in dotnet publish even AFTER doing dotnet restore, the image build fails with this error:

 > [build 9/9] RUN dotnet publish --no-restore -o /app:
2.203 MSBuild version 17.8.3+195e7f5a3 for .NET
3.378 /usr/share/dotnet/sdk/8.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(266,5): error NETSDK1064: Package Microsoft.AspNetCore.OpenApi, version 8.0.0 was not found. It might have been deleted since NuGet restore. Otherwise, NuGet restore might have only partially completed, which might have been due to maximum path length restrictions. [/source/MyCoolTestApp.API/MyCoolTestApp.API.csproj]

ERROR: failed to solve: process "/bin/sh -c dotnet publish --no-restore -o /app" did not complete successfully: exit code: 1

Exceptions (if any)

Further technical details

• Include the output of dotnet --info

  .NET SDK:
     Version:           8.0.100
     Commit:            57efcf1350
     Workload version:  8.0.100-manifests.d43d43dc
  
  Runtime Environment:
     OS Name:     Mac OS X
     OS Version:  14.1
     OS Platform: Darwin
     RID:         osx-x64
     Base Path:   /usr/local/share/dotnet/sdk/8.0.100/
  
  .NET workloads installed:
   Workload version: 8.0.100-manifests.d43d43dc
  There are no installed workloads to display.
  
  Host:
     Version:      8.0.0
     Architecture: x64
     Commit:       5535e31a71
  
  .NET SDKs installed:
     7.0.308 [/usr/local/share/dotnet/sdk]
     8.0.100 [/usr/local/share/dotnet/sdk]
  
  .NET runtimes installed:
     Microsoft.AspNetCore.App 7.0.11 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
     Microsoft.AspNetCore.App 8.0.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
     Microsoft.NETCore.App 7.0.11 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
     Microsoft.NETCore.App 8.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  
  Other architectures found:
     None
  
  Environment variables:
     Not set
  
  global.json file:
     Not found
  
  Learn more:
     https://aka.ms/dotnet/info
  
  Download .NET:
     https://aka.ms/dotnet/download
  

• The IDE (VS / VS Code/ VS4Mac) you're running on, and its version
  JetBrains Rider 2023.2.3

[TrongBYM]

Hello!

Just wanted to chime in and mention that I also struggle with the exact same problem, and am observing the exact same behaviour, e.g. removing --no-restore avoids the problem.

My case differs in that my project are using .NET 7!

I have the following Dockerfile, where the ASPNET project Example.Service depends on my class library Example.Data. I also use some AWS specific packages in my scenario, but removing them does not change the outcome:

FROM mcr.microsoft.com/dotnet/sdk:7.0-bullseye-slim AS build

WORKDIR /src
COPY ["nuget.config", "./"]
COPY ["Example.Service/Example.Service.csproj", "Example.Service/"]
COPY ["Example.Data/Example.Data.csproj", "Example.Data/"]

RUN dotnet restore "Example.Service/Example.Service.csproj"

COPY ["Example.Service/", "Example.Service/"]
COPY ["Example.Data/", "Example.Data/"]

FROM build AS publish
WORKDIR /src/Example.Service
RUN dotnet publish "Example.Service.csproj" \
	--configuration release \
	--no-restore \
	--output /app/publish

FROM mcr.microsoft.com/dotnet/aspnet:7.0-bullseye-slim AS runtime
WORKDIR /app
COPY --from=publish /app/publish .
EXPOSE 80
ENTRYPOINT ["dotnet", "Example.Service.dll"]

Building with this Dockerfile yields me the following error message:

0.677 MSBuild version 17.7.4+3ebbd7c49 for .NET
2.175 /usr/share/dotnet/sdk/7.0.404/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(266,5): error NETSDK1064: Package AWSSDK.AppConfigData, version 3.7.0.23 was not found. It might have been deleted since NuGet restore. Otherwise, NuGet restore might have only partially completed, which might have been due to maximum path length restrictions. [/src/Example.Service/Example.Service.csproj]

My dockerfile is inspired from:

• https://github.com/dotnet/dotnet-docker/blob/main/documentation/scenarios/nuget-credentials.md
• https://github.com/dotnet/dotnet-docker/blob/main/samples/complexapp/Dockerfile

From my dotnet --info:

.NET SDK:
 Version:           8.0.100
 Commit:            57efcf1350
 Workload version:  8.0.100-manifests.8d38d0cc

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19045
 OS Platform: Windows
 RID:         win-x64
 Base Path:   C:\Program Files\dotnet\sdk\8.0.100\

.NET workloads installed:
 Workload version: 8.0.100-manifests.8d38d0cc
There are no installed workloads to display.

Host:
  Version:      8.0.0
  Architecture: x64
  Commit:       5535e31a71

.NET SDKs installed:
  8.0.100 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.25 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.14 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 8.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.25 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.14 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 8.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 6.0.25 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 7.0.14 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 8.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
  x86   [C:\Program Files (x86)\dotnet]
    registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

[bencoveney]

I have the same issue. This is my dockerfile:

FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:8.0 AS build
ARG TARGETARCH
WORKDIR /source

COPY *.csproj .
RUN dotnet restore -a $TARGETARCH

COPY . .
RUN dotnet publish -c Release -a $TARGETARCH --no-restore -o /app

FROM mcr.microsoft.com/dotnet/aspnet:8.0
WORKDIR /app
COPY --from=build /app .
ENV ASPNETCORE_HTTP_PORTS=80
EXPOSE 80
USER $APP_UID
ENTRYPOINT ["./api"]

This is the error I see:

 => ERROR [build 6/6] RUN dotnet publish -c Release -a amd64 --no-restore -o /app                                  1.0s
------
 > [build 6/6] RUN dotnet publish -c Release -a amd64 --no-restore -o /app:
0.690 MSBuild version 17.8.3+195e7f5a3 for .NET
1.002 /usr/share/dotnet/sdk/8.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(266,5): error NETSDK1064: Package Microsoft.AspNetCore.OpenApi, version 8.0.0 was not found. It might have been deleted since NuGet restore. Otherwise, NuGet restore might have only partially completed, which might have been due to maximum path length restrictions. [/source/api.csproj]
------
Dockerfile:9
--------------------
   7 |     
   8 |     COPY . .
   9 | >>> RUN dotnet publish -c Release -a $TARGETARCH --no-restore -o /app
  10 |     
  11 |     FROM mcr.microsoft.com/dotnet/aspnet:8.0
--------------------
ERROR: failed to solve: process "/bin/sh -c dotnet publish -c Release -a $TARGETARCH --no-restore -o /app" did not complete successfully: exit code: 1

Here's the output of dotnet --info:

$ dotnet --info
.NET SDK:
 Version:           8.0.100
 Commit:            57efcf1350
 Workload version:  8.0.100-manifests.6c33ef20

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  20.04
 OS Platform: Linux
 RID:         linux-x64
 Base Path:   /usr/share/dotnet/sdk/8.0.100/

.NET workloads installed:
 Workload version: 8.0.100-manifests.6c33ef20
There are no installed workloads to display.

Host:
  Version:      8.0.0
  Architecture: x64
  Commit:       5535e31a71

.NET SDKs installed:
  6.0.404 [/usr/share/dotnet/sdk]
  7.0.101 [/usr/share/dotnet/sdk]
  8.0.100 [/usr/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.12 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.1 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 8.0.0 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.12 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.1 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 8.0.0 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
  None

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:

I'm running on Windows 10 with WSL and Docker Desktop.

I build using the following docker command:

$ docker build --no-cache -t ${PWD##*/}-api:dev ./api

My project at the moment is a simple hello world project based on the sample here with some simple Swagger boilerplate.

The same commit was working fine but has begun failing on my host machine due to the aforementioned issue. It builds fine in Github Actions. It builds fine when I remove the --no-restore option. I have had inconsistent results after clearing nuget/docker caches, but it fails more than it works.

[markbeazley]

I just had this exact issue, after a lot of fiddling round I finailly got it working by adding this .dockerignore to my project root with this Dockerfile

FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build
WORKDIR /source

COPY *.sln .
COPY src/MySiteApi/*.csproj ./src/MySiteApi/
COPY src/MySiteDb/*.csproj ./src/MySiteDb/
RUN dotnet restore

COPY src/MySiteApi/. ./src/MySiteApi/
COPY src/MySiteDb/. ./src/MySiteDb/
WORKDIR /source/src/MySiteApi
RUN dotnet publish -c release -o /app --no-restore

FROM mcr.microsoft.com/dotnet/aspnet:8.0
WORKDIR /app
COPY --from=build /app ./
ENTRYPOINT ["dotnet", "MySiteApi.dll"]

Don't think it was one of the other things I tried, so posting this in case it helps anyone else.

[jamiewinder]

I'm seeing the same thing. In my case, I'm hoping to restore the packages outside of Docker to avoid complications with the Azure credentials provider not being available in the container.

If I do add and configure the Azure credentials provider in the container manually then I'm able to restore as you'd expect, but then a subsequent build with --no-restore still fails despite the packages having just been restored.

[jamiewinder]

Just to update, so I've reproduced this without a Docker build be involved at all. This basically seems to be an issue with using the .NET SDK on Linux. The dotnet restore command doesn't seem to reliably restore everything that dotnet build needs.

With my affected project, the following steps reproduce the issue:

• docker run -it --rm -v ${pwd}:/app -w /app mcr.microsoft.com/dotnet/sdk:8.0
• dotnet restore
• dotnet build --no-restore

i.e. run the restore and build within the container. Doing the same on Windows - i.e. step 2 and 3 - doesn't have the issue.

[Seabizkit]

HI ms
This has been open since FEB,

could you put some attention here please, the amount of fiddling to get a build to work is ... dying for attention.