Can no longer build Swift-Crypto, from Swift-Certificates (X509) on Mac OS with the latest version (3.4.0)

[Deblanko]

New Regression Checklist

• I'm using the latest version of Swift Crypto (master branch)
• I read the Contribution Guidelines
• I searched for existing GitHub issues

Regression Information

• Breaking version: 3.4.0
• Last working version: 3.3.0

Regression Description

Every since the release of 3.4.0 our automated builds are failing with errors like this:

.../SourcePackages/checkouts/swift-crypto/Package.swift: error: Signing for "swift-crypto_CCryptoBoringSSLShims" requires a development team. Select a development team in the Signing & Capabilities editor. (in target 'swift-crypto_CCryptoBoringSSLShims' from project 'swift-crypto')
.../SourcePackages/checkouts/swift-crypto/Package.swift: error: Signing for "swift-crypto__CryptoExtras" requires a development team. Select a development team in the Signing & Capabilities editor. (in target 'swift-crypto__CryptoExtras' from project 'swift-crypto')
etc.

We have code signing set to manual (but I have tried automatic too)

It is being included from Swift-Certificates so the default is to include the latest:
.package(url: "https://github.com/apple/swift-crypto.git", "2.5.0" ..< "4.0.0"),

This should not need to be signed until the main app has included the code and needs to be signed (like it did prior to 3.4.0)

I can workaround by getting a local copy of Swift-Cetrtificates and forcing the package to be < "3.4.0" but that does not seem a long term solution.

Your thoughts and comments are welcomed.

Swift Crypto version/commit hash

bc1c292

Environment

Stand in the root of the repo and run ./scripts/environment.sh and paste the output below

Swift version: Apple Swift version 5.10 (swiftlang-5.10.0.13 clang-1500.3.9.4)
Target: x86_64-apple-macosx14.0
Unix version: Darwin IVNT-C02FX0X1MD6R 23.4.0 Darwin Kernel Version 23.4.0: Fri Mar 15 00:11:05 PDT 2024
 root:xnu-10063.101.17~1/RELEASE_X86_64 x86_64
macOS version: 14.4.1

Xcode-select path: '/Applications/Xcode.app/Contents/Developer

Xcode: Xcode 15.3 (Build version 15E204a)

[Lukasa]

#226 is the most likely source of your issues. Can you please use Feedback Assistant to report the problem? This shouldn't be causing breakages. What version of Xcode are you using?

[Deblanko]

Build machine is Xcode 15.0.1 but tried 15.4 as well I will use feedback assistant tomorrow first thing Thanks Sent from my iPhone XVI (Beta) On 20 May 2024, at 17:44, Cory Benfield ***@***.***> wrote:  #226<#226> is the most likely source of your issues. Can you please use Feedback Assistant to report the problem? This shouldn't be causing breakages. What version of Xcode are you using? — Reply to this email directly, view it on GitHub<#231 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAI5OKY4Z4KFPV7IKY7G67LZDIR6FAVCNFSM6AAAAABIAAOEEOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRQHAYTMNBTGA>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[Deblanko]

Additional information...
The Swift Certificates package (which includes the Swift Crypto package) is included in a local package that contains local XCFrameworks too.
Regardless I would expect the final product to be code signed and not the individual packages as they are compiled?

Copy of the local Package.swift

// swift-tools-version: 5.7
// The swift-tools-version declares the minimum version of Swift required to build this package.

import PackageDescription

let package = Package(
    name: "IVSecureStoreDaemonLib",
    platforms: [
        .macOS(.v11)
    ],
    products: [
        // Products define the executables and libraries a package produces, and make them visible to other packages.
        .library(
            name: "IVSecureStoreDaemonLib",
            targets: ["IVSecureStoreDaemonLib"]),
    ],
    dependencies: [
        // Dependencies declare other packages that this package depends on.
        // .package(url: /* package url */, from: "1.0.0"),
        .package(url: "https://github.com/apple/swift-certificates.git", .upToNextMajor(from: "1.0.0")),
        .package(url: "https://github.com/apple/swift-docc-plugin", from: "1.0.0"),
    ],
    targets: [
        // Targets are the basic building blocks of a package. A target can define a module or a test suite.
        // Targets can depend on other targets in this package, and on products in packages this package depends on.
        .target(
            name: "IVSecureStoreDaemonLib",
            dependencies: [
                .product(name: "X509", package: "swift-certificates"),
                "IVCommonKit",
                "IVSecureItKit"
            ]
        ),
        .binaryTarget(name: "IVCommonKit", path: "./Frameworks/IVCommonKit.xcframework"),
        .binaryTarget(name: "IVSecureItKit", path: "./Frameworks/IVSecureItKit.xcframework"),
    ]
)

[Lukasa]

Yeah, I'm honestly not sure how this is supposed to work, but my gut instinct is that the addition of the empty privacy manifests is related. What surprises me is that we're not hearing more about it.

Did you get that Feedback Assistant issue opened?

[Lukasa]

Also, can you try two things for me?

1. Add an explicit dependency on swift-crypto in your package.swift, and import it in IVSecureStoreDaemonLib.
2. If that still shows the problem, pin to exactly 89876ab. Check if the issue still manifests.
3. If it doesn't (that is, if step 2 fixed it), try pinning to exactly 2adec61 and see if it manifests again.

[Deblanko]

I filled out the feedback assistant.
Step 1 (using a range less than 3.00) worked
Step 2 (pinned to 89876ab ) worked
Step 3 TBD (I cannot build releases locally since only the build machine has the deployment certificate, so I will try it shortly)

[Deblanko]

Step 3 failed (as expected?)

[Lukasa]

Yeah, that points the finger at step 3. Can you paste the Feedback Assistant number here?

[Deblanko]

https://feedbackassistant.apple.com/feedback/13805266

As stated in the feedback, this fails with xcodebuild but, depending on which certificates are selected it appears to work in Xcode and hence that may be why it has not been reported earlier?

However, like many others, all our release builds are don on DevOps Azure or GitHub actions so xcodebuild, with the correct imported certificates (ie manual code sign) is the default approach for us.

[Lukasa]

@Deblanko can you provide your full xcodebuild invocation?

[Deblanko]

This showed the issue:
xcodebuild -sdk macosx -configuration Release -workspace IVSecureStore.xcworkspace -scheme AllSecureStore clean install DSTROOT=/tmp/Release CONFIGURATION_BUILD_DIR=/tmp/temp_build -destination generic/platform=macOS CODE_SIGN_STYLE=Manual "CODE_SIGN_IDENTITY=Developer ID Application" PROVISIONING_PROFILE= PROVISIONING_PROFILE_SPECIFIER=

I'm not sure how it built a release on automatic (in X-Code) since I thought I had no valid certificates for that. However that would have been a build and not an install. (install just lets me extract various artefacts from the build in known locations)

[Lukasa]

Can you try setting DEVELOPMENT_TEAM? This is probably set in your Xcode project, and seems to be missing from that invocation.

[Deblanko]

Thank you, that appears to solve the issue.
The YAML https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/xcode-v5?view=azure-pipelines
states "#teamId: # string. Optional. Use when signingOption = auto. Team ID."
So we were not auto signing and it was not used?
Setting it explicitly as a command line argument appears to fix the error.
No idea why it worked without it.
I am happy to close this now, unless you have further comments/observations?

[Lukasa]

I think the reason this used to work is that you previously didn't get any bundles. When we added a resource to this project, that started to produce a bundle, which needed to be signed. I have no idea why that flag was needed, that would be a question for the xcode team. But I'm glad you solved it!