Processing Jumpers and Loops, Plus Processing Wires inside the Table as Edges

[tobiasfalk]

Here I started to implement Jumpers, and a new way of drawing Wires inside the Wire table.

• Way to implement Jumpers/Interconnects #350
• Proposal for new way to draw Wires in the Wire box #352 6610f51

Currently, only the Black circles are done and nothing else.
See ex15.png
Ps. And now even the right branch

Edit:
Ex15:

Ex16:

[tobiasfalk]

Now everything from #350 is implemented.

[tobiasfalk]

Syntax:

connectors:
  X1:
    type: Molex KK 254 # more information
    subtype: female
    pinlabels: [GND, VCC, RX, TX, GND, VCC, GND] # pincount is implicit in pinout
    internal_shorts: [[1, 5, 7], [2, 6]]
    internal_shorts_color: [PK, RD]
  X2:
    type: Molex KK 254
    subtype: female
    pinlabels: [GND, VCC, RX, TX, GND, VCC, GND]
    internal_shorts: [[1, 5, 7], [2, 6]]

[tobiasfalk]

New Syntax:

connectors:
  X1:
    type: Molex KK 254 # more information
    subtype: female
    pinlabels: [GND, VCC, RX, TX, GND, VCC, GND] # pincount is implicit in pinout
    shorts:
      SH1:
        pins: [1, 5, 7]
        color: PK
        manufacturer: WireViz
        mpn: 42XCD42A5
        description: shortPart
      SH2:
        pins: [2, 6]
        color: RD
        manufacturer: WireViz
        mpn: 42XCD42A5
        description: shortPart
        length: 42

[tobiasfalk]

[tobiasfalk]

ex16:

[kvid]

I'm sorry that I've not yet had time to play with this PR, but it seems to work as proof of concept for the #350 discussions.

• I suggest that you activate "convert to draft" while this is work-in-progress (I do forget that with my PRs also sometimes).
• Be aware that your latest syntax expansion probably will need coordinaton with changes in Large scale refactoring #251 that is first in line for merge into dev.
• The way you want to combine internal shorts and loops, might also need a discussion in Way to implement Jumpers/Interconnects #350 because I fear it will be too complicated for the simplest use cases.

[tobiasfalk]

I mainly have problems with the bom and how I add the shorts to the bom

[kvid]

I mainly have problems with the bom and how I add the shorts to the bom

Have you considered making the additional components list be a union of shorts and explicitly specified entries, instead of having a separate list for shorts? Then adding to BOM could be handled equally as well.

[tobiasfalk]

@kvid @martinrieder could you look at the new syntax in ex15 & 16?

[martinrieder]

This needs a bit of rework. Comments added.

[martinrieder]

I find qty confusing for unit mm. How about length as an alias?

[tobiasfalk]

Yes, but Additional Components works with qty

[martinrieder]

Yes, just leave it, as changing this is not in scope.

[martinrieder]

Using a list here is fine. Does not match the syntax description though.

[tobiasfalk]

Syntax description is yet to be updated.

[martinrieder]

numShorts could be the return value of the function.

[tobiasfalk]

Do not completely understand what you mean

[tobiasfalk]

@martinrieder ?

[martinrieder]

You had the qty_multiplier choice extended for shorted, so I thought that the function get_qty_multiplier might return numShorts as an integer in that case. The meaning has changed now with references.

[martinrieder]

This could be avoided, see below.

[tobiasfalk]

Do not completely understand what you mean

[tobiasfalk]

@martinrieder ?

[martinrieder]

Sorry, the order of comments got mixed up and my statement was possibly misleading.

[tobiasfalk]

Overall question is if the syntax is acceptable?
If so Checks and other things can be done.

[martinrieder]

@tobiasfalk wrote:

Overall question is if the syntax is acceptable?

I would prefer a solution that does not require using loops or shorts in two different locations. Alternatively, I suggest to use a reference key instead for both, since designators have to be unique anyways. It will also avoid the situation like in ex15 where loops and shorts got unintentionally mixed up.

Overall, I would say that this implementation is quite close to what was discussed in #224 (comment). The only difference is that reference was implied, but implementing that is a little more tricky. Still, I think that we should clarify the outcome of #224 before merging this PR.

Please also have a look at #288, which also implements some loop handling and might cause some merge issues. What I need for my daily work is having alphanumeric pin names and pinlabels matching to work also on loops. I am currently using a workaround for this.

ex15

connectors:
  X1:
    #[...] 
    pinlabels: [GND, VCC, RX, TX, GND, VCC, GND]
    shorts:
      - SH1: [1, 5, 7]  
      - SH2: [2, 6]
    additional_components:
      - reference: SH1 # discussed in #224
        color: PK
        #[...] 
        
      - reference: SH2
        color: RD
        #[...] 

  X2:
    #[...] 
    pinlabels: [GND, VCC, RX, TX, GND, VCC, GND]
    shorts:
      - SH1: [1, 5, 7] # same designator as above... 
      - SH2: [2, 6]   # does this imply the same attributes? 

EDIT: Cross-referring to #224 (comment) for additional details.

[martinrieder]

@tobiasfalk I fixed it by adding the line

os.environ['GVPRPATH'] = str(Path(__file__).parent)

[tobiasfalk]

@martinrieder to where exactly?

[martinrieder]

Just before the call to gvpr in the if branch with find_executable in harness.py.

[tobiasfalk]

@martinrieder could you test it again?

[tobiasfalk]

Some commits back, do not know exactly which one, I implemented #352 and it works without a problem for me

[martinrieder]

• Proposal for new way to draw Wires in the Wire box #352
• 6610f51

Then I propose you edit the description of the PR to include this and the other issues from #369 (comment) as well.

[tobiasfalk]

If everything is now Ok, then I would now wait for #251
Is there a Time frame for when it will be merged? Since the last update on it was in April(2 months ago)

[tobiasfalk]

Oh yes I need to clean up the code a bit, I know that

[martinrieder]

@tobiasfalk: While working on another machine I ran into a dependency issue for find_executable:

C:\Users\Martin\Git\WireViz\src\wireviz>python build_examples.py
Traceback (most recent call last):
  File "C:\Users\Martin\Git\WireViz\src\wireviz\build_examples.py", line 14, in <module>
    from wireviz import APP_NAME, __version__, wireviz
  File "C:\Users\Martin\Git\WireViz\src\wireviz\wireviz.py", line 14, in <module>
    from wireviz.Harness import Harness
  File "C:\Users\Martin\Git\WireViz\src\wireviz\Harness.py", line 12, in <module>
    from distutils.spawn import find_executable
ModuleNotFoundError: No module named 'distutils'

It was a fresh install of Python 3.12.4, see the detailed log below:

Details

C:\Users\Martin\Git\WireViz>pip install -e .
Obtaining file:///C:/Users/Martin/Git/WireViz
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Requirement already satisfied: click in c:\users\martin\appdata\local\programs\python\python312\lib\site-packages (from wireviz==0.5.dev0) (8.1.7)
Requirement already satisfied: pyyaml in c:\users\martin\appdata\local\programs\python\python312\lib\site-packages (from wireviz==0.5.dev0) (6.0.1)
Requirement already satisfied: pillow in c:\users\martin\appdata\local\programs\python\python312\lib\site-packages (from wireviz==0.5.dev0) (10.4.0)
Requirement already satisfied: graphviz in c:\users\martin\appdata\local\programs\python\python312\lib\site-packages (from wireviz==0.5.dev0) (0.20.3)
Requirement already satisfied: colorama in c:\users\martin\appdata\local\programs\python\python312\lib\site-packages (from click->wireviz==0.5.dev0) (0.4.6)
Building wheels for collected packages: wireviz
  Building editable for wireviz (pyproject.toml) ... done
  Created wheel for wireviz: filename=wireviz-0.5.dev0-0.editable-py3-none-any.whl size=16699 sha256=aa3cf29677b77974ad34bc45a6258d9f5487b32f06b5b91fef9c7af69ca17f22
  Stored in directory: C:\Users\Martin\AppData\Local\Temp\pip-ephem-wheel-cache-01soqxb3\wheels\9c\81\24\1a1dd1dee9702950ff7aec4857f0e0d6d2ce93b0c328529227
Successfully built wireviz
Installing collected packages: wireviz
  Attempting uninstall: wireviz
    Found existing installation: wireviz 0.4.1.dev0
    Uninstalling wireviz-0.4.1.dev0:
      Successfully uninstalled wireviz-0.4.1.dev0
Successfully installed wireviz-0.5.dev0

C:\Users\Martin\Git\WireViz>pip list
Package  Version  Editable project location
-------- -------- ---------------------------
click    8.1.7
colorama 0.4.6
graphviz 0.20.3
pillow   10.4.0
pip      24.1.2
PyYAML   6.0.1
wireviz  0.5.dev0 C:\Users\Martin\Git\WireViz

C:\Users\Martin\Git\WireViz>wireviz --version
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\Martin\AppData\Local\Programs\Python\Python312\Scripts\wireviz.exe\__main__.py", line 4, in <module>
  File "C:\Users\Martin\Git\WireViz\src\wireviz\wv_cli.py", line 12, in <module>
    import wireviz.wireviz as wv
  File "C:\Users\Martin\Git\WireViz\src\wireviz\wireviz.py", line 14, in <module>
    from wireviz.Harness import Harness
  File "C:\Users\Martin\Git\WireViz\src\wireviz\Harness.py", line 12, in <module>
    from distutils.spawn import find_executable
ModuleNotFoundError: No module named 'distutils'

During some research, I found out that distutils was deprecated in Python 3.10 already:

https://docs.python.org/3.12/whatsnew/3.12.html#distutils

Remove the distutils package. It was deprecated in Python 3.10 by PEP 632 “Deprecate distutils module”. For projects still using distutils and cannot be updated to something else, the setuptools project can be installed: it still provides distutils. (Contributed by Victor Stinner in gh-92584.)

Doing pip install setuptools fixes the issue, though it is just a workaround. Do you want to add this to requirements.txt or rather find a replacement for the function?

[tobiasfalk]

@martinrieder
When I have time I will find a replacement. Did the notice give a list of replacements?

[martinrieder]

Did the notice give a list of replacements?

https://peps.python.org/pep-0632/

For these modules or functions, use the standard library module shown:

• distutils.fancy_getopt — use the argparse module
• distutils.spawn.find_executable — use the shutil.which function
• distutils.spawn.spawn — use the subprocess.run function
• distutils.sysconfig — use the sysconfig module
• distutils.util.get_platform — use the platform module

[martinrieder]

I realized that setuptools is already listed in requirements.txt, but not in setup.py. Not sure why they are not aligned, because I found out that this behavior has been discussed in #172 earlier.

[tobiasfalk]

I still want to look if there is a current way to do it since it is already depricated.

[tobiasfalk]

@martinrieder in #412 I have now replaced distutils.spawn.find_executable with shutil.which. Could you test it again?

[SnowMB]

Hey,

@tobiasfalk I am loosely following the progress on this project and just saw the awesome results you achieved with the jumpers. I digged a little bit into the implementation and got really confused about all the changes of the dev branch and different merge requests.

Does #412 replace this one (#369)?

I wanted to give banded wires #109 and twisted wires #3 another go and I've seen that your solution is quite similar to what I did back in the day. Using dot to layout the graph and edit it before generating the output with neato -n2 to keep the node and edge positions.

I see however that you used gvpr scripts. The syntax in this scripts is very foreign to me. And I found this comment by an graphviz developer:

We could retire gvpr due to its insufficiencies. Maybe someone would implement something in the same spirit as gvpr using python.

https://forum.graphviz.org/t/im-trying-to-use-gvpr-is-that-a-mistake/1825/6

I think you could achieve the same with using a different graphviz package. I just tried pydot again wich received updates in the meantime and is actively maintained. Switching out the graphviz package with pydot for this project is fairly easy (about 30 lines in the 0.4.1 release, I started here before looking into the dev/branch but i figure it shouldn't be much harder). And the manipulation of the graph after layout could be implemented in python instead of gpvr. Should help with maintainability of the project!

I will be looking into you branch and will try to work on #103 and #3 ontop of it.

Thanks for your great contribution to this project! 👍

And thanks for all the other maintainers and contributors.

[tobiasfalk]

@SnowMB

Does #412 replace this one (#369)?

Yes, since #412 is based on #251(Large scale refactoring), which is hopeful merged soon™

I wanted to give banded wires #109 and twisted wires #3 another go and I've seen that your solution is quite similar to what I did back in the day. Using dot to layout the graph and edit it before generating the output with neato -n2 to keep the node and edge positions.

For #109, as said there, I think that a pure GrapViz solution is not relay possible but by going over SVG and then convert the SVG to the other wanted formats it could be possible (#423, wanted to hear the opinion on this idea and actually did not have the time to start on it, or had other thing that I wanted to do).
To Twisted Pars, I think that this would help with implementing them, but I have no real idea how.

I see however that you used gvpr scripts. The syntax in this scripts is very foreign to me. And I found this comment by an graphviz developer:

We could retire gvpr due to its insufficiencies. Maybe someone would implement something in the same spirit as gvpr using python.

https://forum.graphviz.org/t/im-trying-to-use-gvpr-is-that-a-mistake/1825/6

I used gvpr, since this is what they have given me in the GraphViz forum, see https://forum.graphviz.org/t/straitening-one-line-throu-a-table/2196 and https://forum.graphviz.org/t/way-of-drawing-a-black-circle-inside-a-table-field/2273/12 , it is also quite foreign for me, also why I link to the forum posts at the top of the script.

I think you could achieve the same with using a different graphviz package. I just tried pydot again wich received updates in the meantime and is actively maintained. Switching out the graphviz package with pydot for this project is fairly easy (about 30 lines in the 0.4.1 release, I started here before looking into the dev/branch but i figure it shouldn't be much harder). And the manipulation of the graph after layout could be implemented in python instead of gpvr. Should help with maintainability of the project!

I will be looking into you branch and will try to work on #103 and #3 ontop of it.

Seams interesting, and feel free to change as much as you like, I personally would say that you should look in to #251, since it changes a bunch of stuff and you most likely would have to redo your work if you base it on the current dev branch.