MPF-MC in Developer Editable Mode Failing - gstreamer-1.0

[atummons]

In 0.55, I had editable versions of MPF and MPF-MC for development; however, it now is not working with 0.56. MPF works, but MPF-MC does not. It gives the following error complaining about gstreamer-1.0, whereas the dependency is 0.3.3 through kivy. Not sure what is wrong here.

C:\Users\atummons\Desktop\mpf\mpf-mc>pip3 install mpf-mc --pre
Requirement already satisfied: mpf-mc in c:\users\atummons\appdata\local\programs\python\python39\lib\site-packages (0.56.1)
Requirement already satisfied: kivy-deps.sdl2==0.4.5 in c:\users\atummons\appdata\local\programs\python\python39\lib\site-packages (from mpf-mc) (0.4.5)
Requirement already satisfied: Pygments==2.14.0 in c:\users\atummons\appdata\local\programs\python\python39\lib\site-packages (from mpf-mc) (2.14.0)
Requirement already satisfied: kivy-deps.glew==0.3.1 in c:\users\atummons\appdata\local\programs\python\python39\lib\site-packages (from mpf-mc) (0.3.1)
Requirement already satisfied: kivy==2.1.0 in c:\users\atummons\appdata\local\programs\python\python39\lib\site-packages (from mpf-mc) (2.1.0)
Requirement already satisfied: ruamel.yaml==0.15.42 in c:\users\atummons\appdata\local\programs\python\python39\lib\site-packages (from mpf-mc) (0.15.42)
Requirement already satisfied: psutil==5.9.4 in c:\users\atummons\appdata\local\programs\python\python39\lib\site-packages (from mpf-mc) (5.9.4)
Requirement already satisfied: mpf>=0.56.0 in c:\users\atummons\appdata\local\programs\python\python39\lib\site-packages (from mpf-mc) (0.56.0)
Requirement already satisfied: ffpyplayer==4.3.5 in c:\users\atummons\appdata\local\programs\python\python39\lib\site-packages (from mpf-mc) (4.3.5)
Requirement already satisfied: kivy-deps.gstreamer==0.3.3 in c:\users\atummons\appdata\local\programs\python\python39\lib\site-packages (from mpf-mc) (0.3.3)
Requirement already satisfied: kivy-deps.angle~=0.3.2 in c:\users\atummons\appdata\local\programs\python\python39\lib\site-packages (from kivy==2.1.0->mpf-mc) (0.3.3)
Requirement already satisfied: pypiwin32 in c:\users\atummons\appdata\local\programs\python\python39\lib\site-packages (from kivy==2.1.0->mpf-mc) (223)
Requirement already satisfied: Kivy-Garden>=0.1.4 in c:\users\atummons\appdata\local\programs\python\python39\lib\site-packages (from kivy==2.1.0->mpf-mc) (0.1.5)
Requirement already satisfied: docutils in c:\users\atummons\appdata\local\programs\python\python39\lib\site-packages (from kivy==2.1.0->mpf-mc) (0.19)
Requirement already satisfied: pyserial==3.5 in c:\users\atummons\appdata\local\programs\python\python39\lib\site-packages (from mpf>=0.56.0->mpf-mc) (3.5)
Requirement already satisfied: sortedcontainers==2.3.0 in c:\users\atummons\appdata\local\programs\python\python39\lib\site-packages (from mpf>=0.56.0->mpf-mc) (2.3.0)
Requirement already satisfied: pyserial-asyncio==0.4 in c:\users\atummons\appdata\local\programs\python\python39\lib\site-packages (from mpf>=0.56.0->mpf-mc) (0.4)
Requirement already satisfied: packaging in c:\users\atummons\appdata\roaming\python\python39\site-packages (from mpf>=0.56.0->mpf-mc) (23.0)
Requirement already satisfied: requests in c:\users\atummons\appdata\local\programs\python\python39\lib\site-packages (from Kivy-Garden>=0.1.4->kivy==2.1.0->mpf-mc) (2.28.2)
Requirement already satisfied: pywin32>=223 in c:\users\atummons\appdata\local\programs\python\python39\lib\site-packages (from pypiwin32->kivy==2.1.0->mpf-mc) (305)
Requirement already satisfied: charset-normalizer<4,>=2 in c:\users\atummons\appdata\local\programs\python\python39\lib\site-packages (from requests->Kivy-Garden>=0.1.4->kivy==2.1.0->mpf-mc) (3.0.1)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\atummons\appdata\local\programs\python\python39\lib\site-packages (from requests->Kivy-Garden>=0.1.4->kivy==2.1.0->mpf-mc) (2022.12.7)
Requirement already satisfied: idna<4,>=2.5 in c:\users\atummons\appdata\local\programs\python\python39\lib\site-packages (from requests->Kivy-Garden>=0.1.4->kivy==2.1.0->mpf-mc) (3.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\users\atummons\appdata\local\programs\python\python39\lib\site-packages (from requests->Kivy-Garden>=0.1.4->kivy==2.1.0->mpf-mc) (1.26.14)

C:\Users\atummons\Desktop\mpf\mpf-mc>pip3 install -e .
Obtaining file:///C:/Users/atummons/Desktop/mpf/mpf-mc
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [17 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "C:\Users\atummons\Desktop\mpf\mpf-mc\setup.py", line 354, in <module>
          raise RuntimeError('GStreamer not found and is required to build MPF-MC')
      RuntimeError: GStreamer not found and is required to build MPF-MC
      Using setuptools

      Skipping Cython build (using .c files)
      WARNING: A problem occurred while running pkg-config --libs --cflags gstreamer-1.0 (code 1)

      b"'pkg-config' is not recognized as an internal or external command,\r\noperable program or batch file.\r\n"

      WARNING: A problem occurred while running pkg-config --libs --cflags gstreamer-1.0 (code 1)

      b"'pkg-config' is not recognized as an internal or external command,\r\noperable program or batch file.\r\n"

      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[qcapen]

In order to build the MC on Windows, you'll also need to install the kivy-deps dev packages (be sure to install the matching version numbers as the runtime packages):
kivy-deps.gstreamer-dev 0.3.3
kivy-deps.angle-dev 0.3.2
kivy-deps.sdl2-dev 0.4.5
kivy-deps.glew-dev 0.3.1

[atummons]

Quinn, Thanks for that. That got me over hurdle number one.

Now onto the next one, the different packages are requiring different version of psutil and ruamel between MPF, MPF-MC, and MPF Monitor. Thus when putting each of the three into editable mode, they break each other. MPF-MC and MPF Monitor appear to work fine. But then when getting MPF into editable mode it throws the following.

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
mpf-mc 0.56.0.dev1 requires psutil==5.7.3, but you have psutil 5.9.4 which is incompatible.
mpf-mc 0.56.0.dev1 requires ruamel.yaml==0.15.100, but you have ruamel-yaml 0.15.42 which is incompatible.
mpf-monitor 0.56.0.dev2 requires ruamel.yaml==0.15.100, but you have ruamel-yaml 0.15.42 which is incompatible.

Should they all be the same thing? Or should I be making MPF editable, then manually installing the correct version, then making MPF and MPF Monitor editable?

The only reason that I really care is because now I can't launch Monitor on my computer, it goes to try and find MPF in a folder called Monitor and yells it can't find the config. This worked prior to me getting MPC-MC installed in editable using your guidance above.

[atummons]

So I started all the way over. Uninstalled everything (Python included). I can install by the docs, and it all works.

I set mpf to editable with pip3 install -e . and then when I try to run the game I get
ModuleNotFoundError: No module named 'mpfmc'
So editable mpf breaks mpf-mc. I have then tried to go into the mpf-mc folder and make it editable (succeeds, but then I get an error in the error log and thus can't start my game.

Traceback (most recent call last):
  File "C:\Users\atummons\Desktop\mpf\mpf-mc\mpfmc\commands\mc.py", line 225, in __init__
    MpfMc(options=vars(args),
  File "C:\Users\atummons\Desktop\mpf\mpf-mc\mpfmc\core\mc.py", line 151, in __init__
    create_config_collections(self, self.machine_config['mpf-mc']['config_collections'])
  File "C:\Users\atummons\Desktop\mpf\mpf-mc\mpfmc\core\config_collection.py", line 82, in create_config_collections
    imported_module = import_module(module)
  File "C:\Users\atummons\AppData\Local\Programs\Python\Python39\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "C:\Users\atummons\Desktop\mpf\mpf-mc\mpfmc\config_collections\sound_loop_set.py", line 1, in <module>
    from mpfmc.core.audio.audio_exception import AudioException
  File "C:\Users\atummons\Desktop\mpf\mpf-mc\mpfmc\core\audio\__init__.py", line 7, in <module>
    from mpfmc.core.audio.audio_interface import AudioInterface
ImportError: DLL load failed while importing audio_interface: The specified module could not be found.

I am following the Contributing to MPF document, which I last updated. So I know that it is right for how it was in 0.55. @toomanybrians anything need to change with that structure after going to pipx with 0.56? It works up until I put mpf in editable mode, then everything breaks for various reasons.

[atummons]

I have completely wiped everything, and restarted about 5 times to no avail. I am now getting this error about Cython, which repeats for every call in any .pxd file

Error compiling Cython file:
      ------------------------------------------------------------
      ...
      cimport cpython.pycapsule as pycapsule

      from math import pow
      import logging

      from mpfmc.core.audio.sdl2 cimport *
      ^
      ------------------------------------------------------------

      mpfmc\core\audio\audio_interface.pyx:18:0: 'mpfmc\core\audio\sdl2.pxd' not found

Ultimately it leads to this (sorry for the long post, but I think the main thing is that its erroring out and tells me I can try legacy behavior. I do not know what that's exactly asking me to do. My big question is has anyone gotten this working on 0.56 on a Windows machine? Because I am missing something blatantly obvious or some direction is missing along the way and it is tripping up my steps to get this up and running. I can get it in user mode, but not editable mode. The second I install MPF in editable, everything breaks.

"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -D_THREAD_SAFE -IC:\Users\atummons\AppData\Local\Temp\pip-build-env-ewd_yhyf\overlay\Include\SDL2 -IC:\Users\atummons\AppData\Local\Temp\pip-build-env-ewd_yhyf\overlay\Include -IC:\Users\atummons\AppData\Local\Programs\Python\Python39\include -IC:\Users\atummons\AppData\Local\Programs\Python\Python39\Include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt" /Tcmpfmc/core/audio\sound_file.c /FoC:\Users\atummons\AppData\Local\Temp\tmpi5z_08dc.build-temp\Release\mpfmc/core/audio\sound_file.obj
      sound_file.c
      c:\users\atummons\appdata\local\programs\python\python39\include\pyconfig.h(200): fatal error C1083: Cannot open include file: 'basetsd.h': No such file or directory
      Traceback (most recent call last):
        File "C:\Users\atummons\AppData\Local\Temp\pip-build-env-ewd_yhyf\overlay\Lib\site-packages\setuptools\_distutils\_msvccompiler.py", line 419, in compile
          self.spawn(args)
        File "C:\Users\atummons\AppData\Local\Temp\pip-build-env-ewd_yhyf\overlay\Lib\site-packages\setuptools\_distutils\_msvccompiler.py", line 517, in spawn
          return super().spawn(cmd, env=env)
        File "C:\Users\atummons\AppData\Local\Temp\pip-build-env-ewd_yhyf\overlay\Lib\site-packages\setuptools\_distutils\ccompiler.py", line 1041, in spawn
          spawn(cmd, dry_run=self.dry_run, **kwargs)
        File "C:\Users\atummons\AppData\Local\Temp\pip-build-env-ewd_yhyf\overlay\Lib\site-packages\setuptools\_distutils\spawn.py", line 70, in spawn
          raise DistutilsExecError(
      distutils.errors.DistutilsExecError: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\cl.exe' failed with exit code 2

      During handling of the above exception, another exception occurred:

      Traceback (most recent call last):
        File "C:\Users\atummons\AppData\Local\Temp\pip-build-env-ewd_yhyf\overlay\Lib\site-packages\setuptools\command\editable_wheel.py", line 140, in run
          self._create_wheel_file(bdist_wheel)
        File "C:\Users\atummons\AppData\Local\Temp\pip-build-env-ewd_yhyf\overlay\Lib\site-packages\setuptools\command\editable_wheel.py", line 330, in _create_wheel_file
          files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
        File "C:\Users\atummons\AppData\Local\Temp\pip-build-env-ewd_yhyf\overlay\Lib\site-packages\setuptools\command\editable_wheel.py", line 261, in _run_build_commands
          self._run_build_subcommands()
        File "C:\Users\atummons\AppData\Local\Temp\pip-build-env-ewd_yhyf\overlay\Lib\site-packages\setuptools\command\editable_wheel.py", line 288, in _run_build_subcommands
          self.run_command(name)
        File "C:\Users\atummons\AppData\Local\Temp\pip-build-env-ewd_yhyf\overlay\Lib\site-packages\setuptools\_distutils\cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "C:\Users\atummons\AppData\Local\Temp\pip-build-env-ewd_yhyf\overlay\Lib\site-packages\setuptools\dist.py", line 1213, in run_command
          super().run_command(command)
        File "C:\Users\atummons\AppData\Local\Temp\pip-build-env-ewd_yhyf\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
          cmd_obj.run()
        File "C:\Users\atummons\AppData\Local\Temp\pip-build-env-ewd_yhyf\overlay\Lib\site-packages\setuptools\command\build_ext.py", line 84, in run
          _build_ext.run(self)
        File "C:\Users\atummons\AppData\Local\Temp\pip-build-env-ewd_yhyf\overlay\Lib\site-packages\Cython\Distutils\old_build_ext.py", line 186, in run
          _build_ext.build_ext.run(self)
        File "C:\Users\atummons\AppData\Local\Temp\pip-build-env-ewd_yhyf\overlay\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 345, in run
          self.build_extensions()
        File "C:\Users\atummons\AppData\Local\Temp\pip-build-env-ewd_yhyf\overlay\Lib\site-packages\Cython\Distutils\old_build_ext.py", line 195, in build_extensions
          _build_ext.build_ext.build_extensions(self)
        File "C:\Users\atummons\AppData\Local\Temp\pip-build-env-ewd_yhyf\overlay\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 467, in build_extensions
          self._build_extensions_serial()
        File "C:\Users\atummons\AppData\Local\Temp\pip-build-env-ewd_yhyf\overlay\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 493, in _build_extensions_serial
          self.build_extension(ext)
        File "C:\Users\atummons\AppData\Local\Temp\pip-build-env-ewd_yhyf\overlay\Lib\site-packages\setuptools\command\build_ext.py", line 246, in build_extension
          _build_ext.build_extension(self, ext)
        File "C:\Users\atummons\AppData\Local\Temp\pip-build-env-ewd_yhyf\overlay\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 548, in build_extension
          objects = self.compiler.compile(
        File "C:\Users\atummons\AppData\Local\Temp\pip-build-env-ewd_yhyf\overlay\Lib\site-packages\setuptools\_distutils\_msvccompiler.py", line 421, in compile
          raise CompileError(msg)
      distutils.errors.CompileError: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\cl.exe' failed with exit code 2
      error: Support for editable installs via PEP 660 was recently introduced
      in `setuptools`. If you are seeing this error, please report to:

      https://github.com/pypa/setuptools/issues

      Meanwhile you can try the legacy behavior by setting an
      environment variable and trying to install again:

      SETUPTOOLS_ENABLE_FEATURES="legacy-editable

[qcapen]

I have been successfully building MPF-MC on Windows, but I am not installing it in editable mode. Instead I have the source project cloned in a local directory and I install it locally into my virtual env by running setup.py install (that builds it and install it). Not quite the same as installing in editable mode and not quite as convenient if you are modifying python files, but for modifying the cython source it works for me.

Quinn

[atummons]

Quinn,

Thanks for that. I am just following the instructions here: https://docs.missionpinball.org/en/stable/about/contributing_to_mpf.html

Can you walk me through the steps you do for that? I am planning to modify some python files there, but right now I can't get it to work at all so my changes are 0% of getting anything done.

I have my fork synced down to my machine, which I can access through PyCharm. I installed via pipx per Brian's new instructions and injected mpf-mc and monitor into that. Then from my forked repo of MPF set it editable mode, then ... to get mpfmc working. Right now when trying to run mpf both -X I get ModuleNotFoundError: No module named 'mpfmc'

[atummons]

So I got things moved one step forward. After uninstalling the VS 2019 build tools and downloading 2022 it is now getting through install (installed MSVC 14.0, 10SDK (newest version), which is the same as I had before. Now I have MPF and Monitor running as expected. MC still throwing a fit about some of the Cython/C stuff it appears.

When I open it up in PyCharm there are red squiggles under libc, cimport, cpython. So something appears to be missing with that. Can I just inject into my venv for that? Any ideas?

Traceback (most recent call last):
  File "C:\Users\atummons\Desktop\mpf\mpf-mc\mpfmc\commands\mc.py", line 225, in __init__
    MpfMc(options=vars(args),
  File "C:\Users\atummons\Desktop\mpf\mpf-mc\mpfmc\core\mc.py", line 151, in __init__
    create_config_collections(self, self.machine_config['mpf-mc']['config_collections'])
  File "C:\Users\atummons\Desktop\mpf\mpf-mc\mpfmc\core\config_collection.py", line 82, in create_config_collections
    imported_module = import_module(module)
  File "C:\Users\atummons\AppData\Local\Programs\Python\Python39\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "C:\Users\atummons\Desktop\mpf\mpf-mc\mpfmc\config_collections\sound_loop_set.py", line 1, in <module>
    from mpfmc.core.audio.audio_exception import AudioException
  File "C:\Users\atummons\Desktop\mpf\mpf-mc\mpfmc\core\audio\__init__.py", line 7, in <module>
    from mpfmc.core.audio.audio_interface import AudioInterface
ImportError: DLL load failed while importing audio_interface: The specified module could not be found.

[atummons]

Sorry to bombard with my continued saga here, but this is really frustrating!

I am in my local repo of mpf-mc: C:\Users\atummons\Desktop\mpf\mpf-mc

When I run pip3 install mpf-mc --pre it works as expected.

When I run pip3 install -e ., pip3 install mpf-mc --pre -e ., or pip3 install . it fails, and i get the same error about import errors. I can find bunches of articles about editable install issues with new setuptools, so I downgraded that to an older version and still have the same issue.

Does anyone have any thoughts? Anyone want to be a guinea pig and follow the directions here to see if it works for them? https://docs.missionpinball.org/en/stable/about/contributing_to_mpf.html

I can get MPF to work in editable, but the second I try to get MPF-MC in editable mode, it crashes with the error in the previous post.