macos 13.4: python @3.12.3 quit unexpectedly using zsh @5.9 CLI yubikey-manager @5.4.0_0

[infosecCLI]

• YubiKey Manager (ykman) version:yubikey-manager @5.4.0_0
• How was it installed?:macports
• Operating system and version:macos 13.4
• YubiKey model and version:5cnfc, 5cnano
• Bug description summary:
  trying to use CLI and various commands crash python ...

Steps to reproduce

ykman list
YubiKey 5C Nano (5.4.3) [OTP+FIDO+CCID] Serial: 00000000
zsh: abort      ykman list

ykman -h  
Usage: ykman [OPTIONS] COMMAND [ARGS]...

  Configure your YubiKey via the command line.

  Examples:

    List connected YubiKeys, only output serial number:
    $ ykman list --serials

    Show information about YubiKey with serial number 123456:
    $ ykman --device 123456 info

Options:
  -d, --device SERIAL             specify which YubiKey to interact with by serial number
  -r, --reader NAME               specify a YubiKey by smart card reader name (can't be used with --device or list)
  -l, --log-level [ERROR|WARNING|INFO|DEBUG|TRAFFIC]
                                  enable logging at given verbosity level
  --log-file FILE                 write log to FILE instead of printing to stderr (requires --log-level)
  --diagnose                      show diagnostics information useful for troubleshooting
  -v, --version                   show version information about the app
  --full-help                     show --help output, including hidden commands
  -h, --help                      show this message and exit

Commands:
  info     show general information
  list     list connected YubiKeys
  script   run a python script
  config   configure the YubiKey, enable or disable applications
  fido     manage the FIDO applications
  hsmauth  manage the YubiHSM Auth application
  oath     manage the OATH application
  openpgp  manage the OpenPGP application
  otp      manage the YubiOTP application
  piv      manage the PIV application
zsh: abort      ykman -h

ykman fido
~
~
~
zsh: abort      ykman fido

[Please explain what you did when the bug appeared, and if and how you have been
able to reproduce it.]
hit the return key after typing in a command

Expected result
i.e. list connected YubiKeys

[What did you expect to happen when you did the above?]
i,e, list connected YubiKeys

Actual results and logs

-------------------------------------
Translated Report (Full Report Below)
-------------------------------------

Process:               Python [2912]
Path:                  /opt/local/Library/Frameworks/Python.framework/Versions/3.12/Resources/Python.app/Contents/MacOS/Python
Identifier:            org.python.python
Version:               3.12.3 (3.12.3)
Code Type:             X86-64 (Native)
Parent Process:        zsh [649]
Responsible:           Terminal [508]
User ID:               502

Date/Time:             2024-05-21 20:49:10.2300 -0400
OS Version:            macOS 13.4 (22F66)
Report Version:        12
Bridge OS Version:     7.5 (20P5058)
Anonymous UUID:        63896CFE-17F7-6160-956A-E3BF952E0BFC


Time Awake Since Boot: 3900 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000

Application Specific Information:
abort() called


Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	    0x7ff8153641f2 __pthread_kill + 10
1   libsystem_pthread.dylib       	    0x7ff81539bee6 pthread_kill + 263
2   libsystem_c.dylib             	    0x7ff8152c2b45 abort + 123
3   libffi.8.dylib                	       0x10b04072e dlfree + 2991
4   _ctypes.cpython-312-darwin.so 	       0x10b1075d8 CThunkObject_dealloc + 41
5   Python                        	       0x10b1d6996 dictkeys_decref + 151
6   Python                        	       0x10b1d98f0 dict_dealloc + 247
7   _ctypes.cpython-312-darwin.so 	       0x10b100cda PyCData_clear + 93
8   _ctypes.cpython-312-darwin.so 	       0x10b101f01 PyCFuncPtr_clear + 353
9   _ctypes.cpython-312-darwin.so 	       0x10b1014e3 PyCFuncPtr_dealloc + 14
10  Python                        	       0x10b208e99 subtype_dealloc + 798
11  Python                        	       0x10b1d5dc5 insertdict + 562
12  Python                        	       0x10b1e90d3 _PyModule_ClearDict + 450
13  Python                        	       0x10b2e03eb finalize_modules + 1579
14  Python                        	       0x10b2dfaa2 Py_FinalizeEx + 188
15  Python                        	       0x10b2e123f Py_Exit + 13
16  Python                        	       0x10b2e87b7 handle_system_exit + 35
17  Python                        	       0x10b2e7d42 _PyErr_PrintEx + 41
18  Python                        	       0x10b2e6f69 _PyRun_SimpleFileObject + 497
19  Python                        	       0x10b2e6b4f _PyRun_AnyFileObject + 148
20  Python                        	       0x10b30af89 pymain_run_file_obj + 226
21  Python                        	       0x10b30ac3e pymain_run_file + 89
22  Python                        	       0x10b30a497 Py_RunMain + 971
23  Python                        	       0x10b30a601 Py_BytesMain + 42
24  dyld                          	    0x7ff81504241f start + 1903

Thread 1:
0   libsystem_pthread.dylib       	    0x7ff815397bb0 start_wqthread + 0


Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000000000000  rbx: 0x00007ff858a78640  rcx: 0x00007ff7b54d6d98  rdx: 0x0000000000000000
  rdi: 0x0000000000000103  rsi: 0x0000000000000006  rbp: 0x00007ff7b54d6dc0  rsp: 0x00007ff7b54d6d98
   r8: 0x000000000000f0f5   r9: 0x0000000000000070  r10: 0x0000000000000000  r11: 0x0000000000000246
  r12: 0x0000000000000103  r13: 0x000000010b636e18  r14: 0x0000000000000006  r15: 0x0000000000000016
  rip: 0x00007ff8153641f2  rfl: 0x0000000000000246  cr2: 0x000000010bbb6b00
  
Logical CPU:     0
Error Code:      0x02000148 
Trap Number:     133


Binary Images:
       0x10aa28000 -        0x10aa2bfff org.python.python (3.12.3) <20529671-5ae4-338d-a754-266e1d1bcced> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/Resources/Python.app/Contents/MacOS/Python
       0x10b136000 -        0x10b465fff org.python.python (3.12.3, (c) 2001-2023 Python Software Foundation.) <3a9a9e0b-0f50-3652-88c4-e69b4ff52898> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/Python
       0x10aa95000 -        0x10aaacfff libintl.8.dylib (*) <231c6cca-44ad-3ae7-a080-7aaf4a4e9d05> /opt/local/lib/libintl.8.dylib
       0x10abd1000 -        0x10acd4fff libiconv.2.dylib (*) <53f687f6-88a6-3f39-92b2-8a1ae884ab0d> /opt/local/lib/libiconv.2.dylib
       0x10afff000 -        0x10b006fff _struct.cpython-312-darwin.so (*) <a4e60d78-8186-34a6-8d46-a6ea73cbe977> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_struct.cpython-312-darwin.so
       0x10b017000 -        0x10b01afff _opcode.cpython-312-darwin.so (*) <1a7de035-4ffc-3f31-8dc9-5db5ea8b6ff1> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_opcode.cpython-312-darwin.so
       0x10b027000 -        0x10b02efff binascii.cpython-312-darwin.so (*) <20a989f0-17c1-3c8b-bec5-b4fd6b66eb0b> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/binascii.cpython-312-darwin.so
       0x10b055000 -        0x10b068fff libz.1.3.1.dylib (*) <571be436-8134-30de-a89a-8c76f30ee87e> /opt/local/lib/libz.1.3.1.dylib
       0x10bb32000 -        0x10bd69fff _rust.cpython-312-darwin.so (*) <c4811aec-4826-3081-bbd7-b9435823c114> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/cryptography/hazmat/bindings/_rust.cpython-312-darwin.so
       0x10b8c3000 -        0x10b94efff libssl.3.dylib (*) <352d2dd5-1f5e-3d07-b727-af2b2bc6b237> /opt/local/libexec/*/libssl.3.dylib
       0x10c2a4000 -        0x10c5cbfff libcrypto.3.dylib (*) <6acbf71e-08d0-349c-b9be-116919add356> /opt/local/libexec/*/libcrypto.3.dylib
       0x10b0a4000 -        0x10b0bffff _cffi_backend.cpython-312-darwin.so (*) <076c02b1-f190-3766-83a4-288fb425705d> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/_cffi_backend.cpython-312-darwin.so
       0x10b03b000 -        0x10b042fff libffi.8.dylib (*) <7b54ca2c-c7d6-32d9-9724-aa64821dbfdd> /opt/local/lib/libffi.8.dylib
       0x10b0dc000 -        0x10b0ebfff legacy.dylib (*) <66ae68b0-5e8d-3c37-b5a3-7c6bad8078d7> /opt/local/libexec/*/legacy.dylib
       0x10b0fc000 -        0x10b10ffff _ctypes.cpython-312-darwin.so (*) <56800c92-fd87-3eac-9535-2920eea5b43d> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_ctypes.cpython-312-darwin.so
       0x10b071000 -        0x10b078fff zlib.cpython-312-darwin.so (*) <ae1105f9-7f2f-3d68-b519-31770399d918> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/zlib.cpython-312-darwin.so
       0x10b089000 -        0x10b08cfff _bz2.cpython-312-darwin.so (*) <fc2b1703-b6cb-35cd-b0a7-feddd9b8a66f> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_bz2.cpython-312-darwin.so
       0x10b819000 -        0x10b82cfff libbz2.1.0.8.dylib (*) <c596b77c-a43e-3592-85ec-94083337d7c2> /opt/local/lib/libbz2.1.0.8.dylib
       0x10b7fa000 -        0x10b801fff _lzma.cpython-312-darwin.so (*) <ff2de14a-1d46-3cde-81f6-8ff29846b6cd> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_lzma.cpython-312-darwin.so
       0x10b866000 -        0x10b889fff liblzma.5.dylib (*) <a0946c11-d8f4-32e7-9c51-c193306bf183> /opt/local/lib/liblzma.5.dylib
       0x10b839000 -        0x10b83cfff fcntl.cpython-312-darwin.so (*) <942e6bc4-4792-3f66-b8db-ebcba4a1bd52> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/fcntl.cpython-312-darwin.so
       0x10b849000 -        0x10b84cfff _posixsubprocess.cpython-312-darwin.so (*) <b2deec7b-550d-336c-a96e-c12be0308053> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_posixsubprocess.cpython-312-darwin.so
       0x10b89a000 -        0x10b8a1fff select.cpython-312-darwin.so (*) <1077d79e-4592-327c-ac53-d1b724128ea1> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/select.cpython-312-darwin.so
       0x10baa8000 -        0x10bab3fff math.cpython-312-darwin.so (*) <c7fbf012-dd95-33e3-8a89-d388cc65e101> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/math.cpython-312-darwin.so
       0x10b8ae000 -        0x10b8b1fff _heapq.cpython-312-darwin.so (*) <5849ec7f-320c-366e-9b20-ef9cfb50ac02> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_heapq.cpython-312-darwin.so
       0x10ba8f000 -        0x10ba92fff _queue.cpython-312-darwin.so (*) <6bf73981-c31d-3a7d-808a-2d60a8c01b61> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_queue.cpython-312-darwin.so
       0x10bac4000 -        0x10bacbfff _hashlib.cpython-312-darwin.so (*) <47fa71dc-5980-3abb-8f41-a8f2a3748731> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_hashlib.cpython-312-darwin.so
       0x10badc000 -        0x10bae3fff _blake2.cpython-312-darwin.so (*) <359214fb-9ad9-3cfa-bf66-ff59fd2c11f3> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_blake2.cpython-312-darwin.so
       0x10bb0f000 -        0x10bb1efff _scard.cpython-312-darwin.so (*) <af3b1d56-7a34-377d-8d89-b94b993f301d> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/smartcard/scard/_scard.cpython-312-darwin.so
       0x10c08b000 -        0x10c09afff _datetime.cpython-312-darwin.so (*) <2d1515fc-0e9e-36e8-9da7-8765bbe3c994> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_datetime.cpython-312-darwin.so
       0x10baf0000 -        0x10baf3fff _bisect.cpython-312-darwin.so (*) <79735aec-0a7d-3b75-9eac-81252ba346ae> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_bisect.cpython-312-darwin.so
       0x10c06a000 -        0x10c06dfff _random.cpython-312-darwin.so (*) <3304821c-3731-3290-a47f-6b7b3a06c9ee> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_random.cpython-312-darwin.so
       0x10c0c7000 -        0x10c0d2fff _sha2.cpython-312-darwin.so (*) <a1e0238b-8c37-3dbd-9b35-607f25a04663> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_sha2.cpython-312-darwin.so
       0x10c0fe000 -        0x10c10dfff _socket.cpython-312-darwin.so (*) <fa9f5d6d-5dd3-3b55-8b07-9fe9c158dec7> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_socket.cpython-312-darwin.so
       0x10c0af000 -        0x10c0b6fff array.cpython-312-darwin.so (*) <98030b4a-1409-390f-adbc-d26f63b1563c> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/array.cpython-312-darwin.so
       0x10c0df000 -        0x10c0e6fff _json.cpython-312-darwin.so (*) <2cb32054-925f-302c-8dbb-3bef8c892a62> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_json.cpython-312-darwin.so
       0x10c251000 -        0x10c268fff _ssl.cpython-312-darwin.so (*) <940fc70a-aab0-389b-a4c6-991f9e39fb1b> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_ssl.cpython-312-darwin.so
       0x10c07a000 -        0x10c07dfff _scproxy.cpython-312-darwin.so (*) <724d852a-8116-3fec-9631-832e060be0eb> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_scproxy.cpython-312-darwin.so
       0x10c21e000 -        0x10c221fff grp.cpython-312-darwin.so (*) <224475a6-b051-3cfa-aaee-d7b264b34651> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/grp.cpython-312-darwin.so
       0x10c22e000 -        0x10c235fff _csv.cpython-312-darwin.so (*) <f87ed128-dab4-3be3-8314-004669fb84dc> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_csv.cpython-312-darwin.so
       0x10cd7f000 -        0x10cd96fff com.apple.iokit.IOHIDLib (2.0.0) <50595de3-5fdd-3ac7-a510-72b6e76719e7> /System/Library/Extensions/IOHIDFamily.kext/Contents/PlugIns/IOHIDLib.plugin/Contents/MacOS/IOHIDLib
    0x7ff81535c000 -     0x7ff815395ff7 libsystem_kernel.dylib (*) <84597863-2c70-3a4a-b0f1-84b39d731209> /usr/lib/system/libsystem_kernel.dylib
    0x7ff815396000 -     0x7ff8153a1fff libsystem_pthread.dylib (*) <8cb0b396-011f-31f2-8915-94cde1abae84> /usr/lib/system/libsystem_pthread.dylib
    0x7ff815243000 -     0x7ff8152caff7 libsystem_c.dylib (*) <5ca0d0cf-fcdd-3180-af7b-f137fe1e7aca> /usr/lib/system/libsystem_c.dylib
    0x7ff81503c000 -     0x7ff8150d45cf dyld (*) <9e98a840-a3ac-31c1-ab97-829af9bd6864> /usr/lib/dyld

External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0

VM Region Summary:
ReadOnly portion of Libraries: Total=302.4M resident=0K(0%) swapped_out_or_unallocated=302.4M(100%)
Writable regions: Total=1.2G written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=1.2G(100%)

                                VIRTUAL   REGION 
REGION TYPE                        SIZE    COUNT (non-coalesced) 
===========                     =======  ======= 
Activity Tracing                   256K        1 
Kernel Alloc Once                    8K        1 
MALLOC                           243.2M       33 
MALLOC guard page                   24K        6 
MALLOC_MEDIUM (reserved)         600.0M        5         reserved VM address space (unallocated)
MALLOC_NANO (reserved)           384.0M        1         reserved VM address space (unallocated)
STACK GUARD                          8K        2 
Stack                             16.5M        2 
VM_ALLOCATE                       14.0M       17 
__DATA                            5131K      178 
__DATA_CONST                      7285K      139 
__DATA_DIRTY                       339K       57 
__LINKEDIT                       175.3M       42 
__OBJC_RO                         66.3M        1 
__OBJC_RW                         2013K        2 
__TEXT                           127.1M      192 
dyld private memory                260K        2 
mapped file                         52K        1 
shared memory                       20K        3 
===========                     =======  ======= 
TOTAL                              1.6G      685 
TOTAL, minus reserved VM space   657.4M      685 

[What actually happened?]
zsh: abort ykman -h

Other info

[Anything else you would like to add?]
thank you for looking into this

[infosecCLI]

Please note, I uninstalled the port of yubikey-manager.
I installed yubikey-manager-5.4.0-mac.pkg.

I did not have the same problem. There was no "quit unexpectedly."

The commands I mentioned previously did not abort.

Thank you.

[amake]

I am a maintainer of the MacPorts port.

This is almost certainly a MacPorts-specific problem, though I have been unsuccessful in debugging it.

[amake]

I note that installing yubikey_manager via pip instead of via MacPorts still has the same problem, but only if using the MacPorts python3:

% /opt/local/bin/python3 -m venv mpenv
% . ./mpenv/bin/activate
% pip install yubikey_manager
% ./mpenv/bin/ykman -h
# ...
zsh: abort      ykman -h

% /usr/bin/python3 -m venv applenv
% . ./applenv/bin/activate
% pip install yubikey_manager
% ./applenv/bin/ykman -h
# no abort

The MacPorts Python crashes even when built without the +lto or +optimizations variants enabled.

[amake]

I pared down ykman/_cli/__main__.py until ykman -h stopped aborting, and I found that I had to get rid of basically everything, including imports from './util'. So I think the proximate cause is a side effect somewhere else in the program, perhaps a resource that is not cleaned up appropriately.

I'm at a loss as to how to debug this further.

[dainnilsson]

Based on the presence of ctypes in the trace I suspect the problem is related to the HID communication layer. Can you try running the following command to see if that alone triggers the abort?

python3 -c "import ykman.hid.macos"

[amake]

Based on the presence of ctypes in the trace I suspect the problem is related to the HID communication layer. Can you try running the following command to see if that alone triggers the abort?

python3 -c "import ykman.hid.macos"

Yes, that does trigger the abort!

[amake]

Oh I should note that the problem only seems to happen on x86_64, not on arm.