python3Packages.pyobjc-core: init at 10.3.1 #336801

ferrine · 2024-08-23T15:21:51Z

Description of changes

Things done

Add a 👍 reaction to pull requests you find important.

ferrine · 2024-08-24T22:14:44Z

Hi, can anyone please review?

pkgs/development/python-modules/pyobjc-core/default.nix

SuperSandro2000 · 2024-08-25T13:32:12Z

pkgs/development/python-modules/pyobjc-core/default.nix

+  #
+  # ZZZ: Sorted out the issues with tests and put them in prePatch
+  checkPhase = ''
+    python3 setup.py test


That is deprecated and we removed already all usages of running tests like that. Please use pytest.

I did not manage to do that, looking into the actual code, the author of the package did some extra stuff to bootstrap unittest

Can we use pytestCheckHook and pytestFlagsArray = [ "PyObjCTest" ]?

that's what I tried and it was failing with

ERROR PyObjCTest/test_voidpointer.py::TestCase - AttributeError: 'TestCase' object has no attribute 'runTest'. Did you mean:... ERROR PyObjCTest/test_voidpointer.py::TestVoidPointer - AttributeError: 'TestVoidPointer' object has no attribute 'runTest'. Did yo... ERROR PyObjCTest/test_weakref.py::TestCase - AttributeError: 'TestCase' object has no attribute 'runTest'. Did you mean:... ERROR PyObjCTest/test_weakref.py::TestWeakrefs - AttributeError: 'TestWeakrefs' object has no attribute 'runTest'. Did you m... ERROR PyObjCTest/test_weakref.py::TestObjCWeakRef - AttributeError: 'TestObjCWeakRef' object has no attribute 'runTest'. Did yo... !!!!!!!!!!!!!!!!!! Interrupted: 438 errors during collection !!!!!!!!!!!!!!!!!!! ====================== 11 warnings, 438 errors in 20.36s =======================

unittestCheckHook is promising, trying to make it work as well

niether unittestCheckHook is easy to make working

With unittestCheck Hook errors are different and seem to be related to how tests are set up in the library

https://github.com/ronaldoussoren/pyobjc/blob/master/pyobjc-core/setup.py#L216-L329

I've updated the PR with all the comments taken into account, but for running tests with all the guidelines.

pkgs/development/python-modules/pyobjc-core/default.nix

adrian-gierakowski · 2024-09-12T08:30:06Z

would be great to have this merged

ferrine · 2024-09-12T20:50:22Z

Any more issues there?

pkgs/development/python-modules/pyobjc-core/default.nix

SuperSandro2000 · 2024-09-24T08:05:34Z

pkgs/development/python-modules/pyobjc-core/default.nix

@@ -0,0 +1,117 @@
+# inpired by https://github.com/input-output-hk/lace/blob/0fd166666cc454171811b307050d0815452bea82/nix/lace-blockchain-services/internal/any-darwin.nix#L339


Suggested change

# inpired by https://github.com/input-output-hk/lace/blob/0fd166666cc454171811b307050d0815452bea82/nix/lace-blockchain-services/internal/any-darwin.nix#L339

SuperSandro2000 · 2024-09-24T08:07:05Z

pkgs/development/python-modules/pyobjc-core/default.nix

+    done
+    # impurities:
+    ( grep -RF '/usr/bin/xcrun' || true ; ) | cut -d: -f1 | while IFS= read -r file ; do
+      sed -r "s+/usr/bin/xcrun+$(${lib.getExe pkgs.which} xcrun)+g" -i "$file"


We want to use which instead of pkgs.which

SuperSandro2000 · 2024-09-24T08:07:44Z

pkgs/development/python-modules/pyobjc-core/default.nix

+  checkPhase = ''
+    # TODO: This library does not follow standard testing with pytest
+    # and implemented its own test runner bootstrapping unittest
+    python3 setup.py test


That command is deprecated. Can we use pytest?

I tried, it just fails with internal errors, not running any tests. In the source code testing is managed by this code and has additional bootstrapping in setup.py

SuperSandro2000 · 2024-09-24T08:08:03Z

pkgs/development/python-modules/pyobjc-core/default.nix

+let
+  appleSDK = darwin.apple_sdk_11_0;
+  inherit (appleSDK) MacOSX-SDK objc4;
+  darwinFrameworks = appleSDK.frameworks;


Can we reduce the shorthands?

frankplow · 2024-10-23T17:34:08Z

It looks like the oustanding requested changes, with the exception of the checkPhase as explained by the contributor, were addressed in c0ea0f0. Is there anything else blocking this being merged?

sarahec · 2024-11-06T01:00:12Z

@ferrine Are you on Matrix? I've been chatting with @emilazy about this PR there and there's now a much easier way to do it. Your work just fell into a crack between the old way and the new way.

There's a lot of interest in getting this done.

ferrine · 2024-11-10T09:07:10Z

Not yet on matrix, let me join

sarahec · 2024-11-10T17:33:18Z

This is about to become much simpler. A build of darwin.libffi is coming: #354108

The rest of the build becomes easier. You can remove the darwin import and all of the BuildInputs (and the xcodebuild reference); these are all provided automatically for Darwin builds now.

I can take a look at the unusual test setup and see if I can fit it into the standard model. The same for replacing the substitutions with patches (it's much clearer to read then).

reckenrode · 2024-11-11T14:44:26Z

I was waiting on the checks to finish before merging #354108. It’s merged now, so you should be able to use it for pyobjc.

sarahec · 2024-11-17T19:09:54Z

@ferrine could you rebase this over master, please? It's the one thing I can't do for you directly.

ferrine · 2024-11-19T17:48:08Z

@ferrine could you rebase this over master, please? It's the one thing I can't do for you directly.

Did that today and got two errors in tests, will push soon

ferrine · 2024-11-19T17:54:39Z

These new errors appeared

error: builder for '/nix/store/vp41w5311igaw6rjz98qriwk3nadfzda-python3.12-pyobjc-core-10.3.1.drv' failed with exit code 1;
       last 25 log lines:
       > test_cannot_create (PyObjCTest.test_varlist.TestVarlistVarious.test_cannot_create) ... ok
       >
       > ======================================================================
       > FAIL: testSignatureCount (PyObjCTest.test_splitsig.SplitSignatureTest.testSignatureCount) (cls='NSObject', sel=b'_ns:setObservationTrackingDictionary:')
       > ----------------------------------------------------------------------
       > Traceback (most recent call last):
       >   File "/private/tmp/nix-build-python3.12-pyobjc-core-10.3.1.drv-0/pyobjc_core-10.3.1/PyObjCTest/test_splitsig.py", line 174, in testSignatureCount
       >     self.assertEqual(
       > AssertionError: 1 != 2 : _ns:setObservationTrackingDictionary: [1:2] (b'v', b'@', b':', b'@') <objective-c class NSObject at 0x1f93a0330>
       >
       > ======================================================================
       > FAIL: testSignatureCount (PyObjCTest.test_splitsig.SplitSignatureTest.testSignatureCount) (cls='NSObject', sel=b'isNSURL::')
       > ----------------------------------------------------------------------
       > Traceback (most recent call last):
       >   File "/private/tmp/nix-build-python3.12-pyobjc-core-10.3.1.drv-0/pyobjc_core-10.3.1/PyObjCTest/test_splitsig.py", line 174, in testSignatureCount
       >     self.assertEqual(
       > AssertionError: 0 != 2 : isNSURL:: [0:2] (b'B', b'@', b':') <objective-c class NSObject at 0x1f93a0330>
       >
       > ----------------------------------------------------------------------
       > Ran 3786 tests in 475.284s
       >
       > FAILED (failures=2, skipped=4, expected failures=3)
       > SUMMARY: {'count': 3786, 'fails': 2, 'errors': 0, 'xfails': 3, 'xpass': 0, 'skip': 4}
       > error: some tests failed
       > hello
       For full logs, run 'nix log /nix/store/vp41w5311igaw6rjz98qriwk3nadfzda-python3.12-pyobjc-core-10.3.1.drv'.

emilazy · 2024-11-19T17:57:05Z

pkgs/development/python-modules/pyobjc-core/default.nix

+    # TODO: Make patch for setup.py
+    # ignore the manual include flag for ffi, appears that it needs a very specific ffi from sdk (needs confirmation)
+    substituteInPlace setup.py --replace-fail '"-I/usr/include/ffi"' '#"-I/usr/include/ffi"'
+    # make os.path.exists that can spoil objc4 return True
+    substituteInPlace setup.py --replace-fail 'os.path.join(self.sdk_root, "usr/include/objc/runtime.h")' '"/"'
+
+    # Turn off clang’s Link Time Optimization, or else we can’t recognize (and link) Objective C .o’s:
+    sed -r 's/"-flto=[^"]+",//g' -i setup.py
+    # Fix some test code:
+    grep -RF '"sw_vers"' | cut -d: -f1 | while IFS= read -r file ; do
+      sed -r "s+"sw_vers"+"/usr/bin/sw_vers"+g" -i "$file"
+    done


Not sure about the test failures, but all of this should now be unnecessary.

I agree the SDK and LTO stuff should be unnecessary. LTO was fixed with the ld64 update in #307880.

The sw_vers stuff might still be?

emilazy · 2024-11-19T17:57:34Z

pkgs/development/python-modules/pyobjc-core/default.nix

+    # Fixes impurities in the package and fixes darwin min version
+    # TODO: Propose a patch that fixes it in a better way
+    # Force it to target our ‘darwinMinVersion’, it’s not recognized correctly:
+    grep -RF -- '-DPyObjC_BUILD_RELEASE=%02d%02d' | cut -d: -f1 | while IFS= read -r file ; do
+      sed -r '/-DPyObjC_BUILD_RELEASE=%02d%02d/{s/%02d%02d/${
+        lib.concatMapStrings (lib.fixedWidthString 2 "0") (
+          lib.splitString "." stdenv.targetPlatform.darwinMinVersion
+        )
+      }/;n;d;}' -i "$file"
+    done


This is also likely unnecessary.

darwinMinVersion probably shouldn’t be used even if it were. Build scripts should get the minimum version from MACOSX_DEPLOYMENT_TARGET.

domenkozar · 2024-11-20T12:45:59Z

This is one of the last things to merge cachix/devenv-nixpkgs#9, how can I help?

emilazy · 2024-11-20T14:10:52Z

I believe @sarahec was working on this. I’ll ask if there’s anything I can do to help the process, since this blocks a lot of things.

Edit: Though looking at the current state of the PR it seems like it should probably be good to merge with a little clean‐up and figuring out what to do with the failing tests (maybe just disable them if we’re already disabling a bunch?)

sarahec · 2024-11-20T19:24:25Z

I believe @sarahec was working on this

Apologies. I spent yesterday getting an updated ID and passport and have been away from the keyboard.

sarahec · 2024-11-20T22:13:04Z

I have this working and just need to figure out how to send the changes

sarahec · 2024-11-21T00:42:02Z

Ugh. There's a bunch of patches missing. I may have to open a fresh PR and link back to this one.

ferrine · 2024-11-21T21:33:45Z

I'm happy if you help with this

sarahec · 2024-11-22T05:23:58Z

I finished the patch. Unfortunately, I had to open a new PR (#358063)

It couldn't be done here in suggestions, nor could I push code to this PR.

ferrine · 2024-11-29T18:01:59Z

No problems, the progress you did is huge step forward anyway

ferrine requested a review from natsukium as a code owner August 23, 2024 15:21

github-actions bot added the 6.topic: python label Aug 23, 2024

ferrine force-pushed the new/pyobjc-core branch 4 times, most recently from e7490ab to c628cdb Compare August 24, 2024 13:51

ofborg bot added 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 1-10 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux labels Aug 24, 2024

This was referenced Aug 25, 2024

Python on Darwin: pyobj tracking issue #105156

Closed

Enabling pyobjc on python3Packages #101360

Open

ferrine force-pushed the new/pyobjc-core branch 2 times, most recently from 3879755 to 8c5df9a Compare August 25, 2024 10:21

SuperSandro2000 reviewed Aug 25, 2024

View reviewed changes

ferrine force-pushed the new/pyobjc-core branch from 8c5df9a to 807e3a4 Compare September 12, 2024 14:02

natsukium requested changes Sep 13, 2024

View reviewed changes

ferrine force-pushed the new/pyobjc-core branch 2 times, most recently from 5931b48 to 986077d Compare September 13, 2024 15:02

ofborg bot added the 11.by: package-maintainer This PR was created by the maintainer of the package it changes label Sep 13, 2024

ferrine requested review from natsukium and SuperSandro2000 September 18, 2024 12:26

SuperSandro2000 reviewed Sep 24, 2024

View reviewed changes

ferrine force-pushed the new/pyobjc-core branch from 986077d to c0ea0f0 Compare September 27, 2024 15:48

thecaralice mentioned this pull request Oct 17, 2024

Build failure: python3Packages.jaraco-path on darwin #346003

Open

vinnymeller mentioned this pull request Oct 24, 2024

jetbrains-mono: build from source #349612

Draft

13 tasks

python3Packages.pyobjc-core: init at 10.3.1

2ddb668

ferrine force-pushed the new/pyobjc-core branch from c0ea0f0 to 2ddb668 Compare November 19, 2024 17:54

emilazy reviewed Nov 19, 2024

View reviewed changes

sandydoo mentioned this pull request Nov 19, 2024

track bump progress cachix/devenv-nixpkgs#9

Merged

10 tasks

sarahec mentioned this pull request Nov 22, 2024

python312Packages.pyobjc-core: init at 10.3.1 #358063

Draft

11 tasks

domenkozar closed this Nov 22, 2024

ferrine deleted the new/pyobjc-core branch November 29, 2024 18:02

		@@ -0,0 +1,117 @@
		# inpired by https://github.com/input-output-hk/lace/blob/0fd166666cc454171811b307050d0815452bea82/nix/lace-blockchain-services/internal/any-darwin.nix#L339

python3Packages.pyobjc-core: init at 10.3.1 #336801

python3Packages.pyobjc-core: init at 10.3.1 #336801

Conversation

ferrine commented Aug 23, 2024 • edited Loading

Description of changes

Things done

ferrine commented Aug 24, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adrian-gierakowski commented Sep 12, 2024

ferrine commented Sep 12, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

frankplow commented Oct 23, 2024 • edited Loading

sarahec commented Nov 6, 2024 • edited Loading

ferrine commented Nov 10, 2024

sarahec commented Nov 10, 2024

reckenrode commented Nov 11, 2024 • edited Loading

sarahec commented Nov 17, 2024

ferrine commented Nov 19, 2024

ferrine commented Nov 19, 2024

Choose a reason for hiding this comment

reckenrode Nov 20, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

domenkozar commented Nov 20, 2024

emilazy commented Nov 20, 2024 • edited Loading

sarahec commented Nov 20, 2024

sarahec commented Nov 20, 2024

sarahec commented Nov 21, 2024

ferrine commented Nov 21, 2024

sarahec commented Nov 22, 2024

ferrine commented Nov 29, 2024

ferrine commented Aug 23, 2024 •

edited

Loading

frankplow commented Oct 23, 2024 •

edited

Loading

sarahec commented Nov 6, 2024 •

edited

Loading

reckenrode commented Nov 11, 2024 •

edited

Loading

reckenrode Nov 20, 2024 •

edited

Loading

emilazy commented Nov 20, 2024 •

edited

Loading