2nd attempt at: picodrive revisions from irixxxx: major drc work, bugfixing and enhancements for megadrive vdp, and some more

[hizzlekizzle]

attempting to cleanly merge irixxxx's commits without derailing their PR to notaz. See also #124

[irixxxx]

I've pushed another fix for windows (using mingw and wine) and osx. Please try this out. I've done minimal checks for both and know that 32x with DRC is basically working with my setup.

[hizzlekizzle]

I think that got it. At least, I'm able to cross-compile from linux to Windows, so I suspect we're in good shape. Let me get independent verification on that and then I think we're ready to go :)

[bslenul]

32X crashes RA for me on Windows (looks fine on my Linux VM), retroarch_debug.exe doesn't generate a crash log, not sure why. I used WinDbg instead, first time I'm using it so I hope I did it properly :p

(43d4.24b8): Illegal instruction - code c000001d (first chance)
(43d4.24b8): Break instruction exception - code 80000003 (first chance)
*** WARNING: Unable to verify timestamp for D:\programmes\RetroArch\libglib-2.0-0.dll
KERNELBASE!wil::details::DebugBreak+0x2:
00007ff9`e4e50aa2 cc              int     3
0:000> k
 # Child-SP          RetAddr           Call Site
00 00000000`0136ec68 00000000`64a41879 KERNELBASE!wil::details::DebugBreak+0x2
01 00000000`0136ec70 00007ff9`e7ad85b6 libglib_2_0_0!g_vasprintf+0x7c9
02 00000000`0136ef10 00007ff9`e7aca056 ntdll!RtlpCallVectoredHandlers+0x196
03 00000000`0136efb0 00007ff9`e7affe3e ntdll!RtlDispatchException+0x66
04 00000000`0136f1c0 00000000`190a218f ntdll!KiUserExceptionDispatch+0x2e
05 00000000`0136f8e8 00000000`1948203a picodrive_libretro!retro_deinit+0x26e587
06 00000000`0136f8f0 00000000`18f7ac80 picodrive_libretro!retro_deinit+0x64e432
07 00000000`0136f8f8 00000000`00000002 picodrive_libretro!retro_deinit+0x147078
08 00000000`0136f900 00000000`00000000 0x2

And last lines of the console log before the crash:

[libretro INFO] 00003:142: 32X startup
[libretro INFO] 00003:142: drc_cmn_init: 0000000018EE2000, 4194304 bytes: 1
DRC registers created, 15 host regs (4 REG, 3 STATIC, 1 CTX)
[libretro WARN] warning: wanted to map @06000000, got 000000000FC40000
[INFO] [Environ]: SET_GEOMETRY.
Exception code=0xc000001d flags=0x0 at 0x0000000018EE218F

MD, SMS and Sega-CD seems to work fine however.

[irixxxx]

Wow... I'm not sure if I'm interpreting this correctly, but a crash in vasprintf? And unfortunately no good symbols in retroarch :-?
Hmm, I was only able to test the 32 bit version with windows. my wine doesn't like the mingw 64 bit executable. The OSX 64 bit executable works fine though... I'll put up a ReactOS VM and try it there.

[Tatsuya79]

32x crashed here too on win7 x64.

32x crash.txt

retroarch.log

If I force interpreter it stays on a black screen, log console says:
DRC registers created, 15 host regs (4 REG, 3 STATIC, 1 CTX)

[irixxxx]

I can reproduce this both under wine and reactos. Interstingly enough it works fine with a 32 bit retroarch with both. It must be something with the 64 bit code generator, OTOH I don't get this with OSX and x86 linux. The only difference is MS uses a different ABI. I've reviewed that part of the backend to no avail. I can also say that the crash happens in the translation cache, hence there must be something wrong with the x86 backend.
I'll set up a debugging environment with wine and gdb ASAP and hope to see more with that.

[irixxxx]

I think I found the problem - a missing REX for jmp/call in case the register is r8-r15. Now it works for me in wine64. Please pull.

[irixxxx]

BTW, I think this should solve the following reported issues: #122, #91, #87, #84, #34, #32.

[bslenul]

Works for me too now with your latest changes, it makes a little *pop* sound however on launch (not on restart): https://streamable.com/ie1dc7

It's not the end of the world of course :p but still weird, it only happens with 32X.

edit: after more testing with the core available in the core updater, yours is actually better, with the "official" core I get that little sound too (but not in MKII, that's why I thought it was a new issue, my bad!), but it's much worse because it makes a sound even worse each time I open/close RetroArch menu :D
Current "official" core: https://streamable.com/bu7bly
The core from this PR: https://streamable.com/3yt14z

The core crashes with dynarec OFF however (again, only with 32X): crash-200708-211934.log

About the issues you mentionned:

• Tempo: can't hear anything wrong.
• Test Drive II, you can see the other cars:
  
• ToeJam & Earl you can see that the enemy sprite doesn't overlap with the HUD:
  
• Virtua Fighter works fine with dynarec:
  

[irixxxx]

The popping audio comes from a DC offset in the PWM channel - this can even happen if it is not used for output, when it is initialized to a bad value. You should hear it only with 32x or CD games. Tempo is one of the bad examples for this. There is a highpass to filter away DC, but it must adapt to the offset value. That's why you hear a click in the start phase.

Regarding sh2 interpreter, I know why it crashes, but I must ponder a bit how to fix it in a suitably clean way.

[hizzlekizzle]

Are we ready to merge this, then? If so, I can pull the trigger.

[irixxxx]

I think the interpreter bug isn't harmful - it occurs only for platforms where the DRC is available and I presume few people would ever use the interpreter in that case. Anyway, I'm going to fix this today Fixed in my repo.

Platforms I've tested with libretro: Windows (x86+x64 via wine and reactos), OSX (x86_64, 10.14), Linux (x86_64, Ubuntu 16+18).
Platforms I've tested standalone: caanoo+gp2x, Linux (Ubuntu x86_64, aarch64), Linux via qemu (Debian mipsel 32, ppcel 64, riscv 64)
So, that leaves out a lot of the platforms supported by Makefile.libretro on my side. However, I presume some of them can't be used anyway - e.g. I think picodrive isn't running on a PS3 or Wii since that is AFAIK big endian.

You may wonder about that gp2x... it's my testbed for speed regressions. I'll note it immediately if something gets notably slower :-D

[irixxxx]

Great, thank you!

Just a thought occurring to me right now: since version 1.96 there are some commits now... should I rename this 1.97 to indicate this as the libretro version? Or should this possibly be done after more testing?

BTW, a list of additional platforms I presume this might run on:

• should probably run out of the box if it did before (well, before applying murphy's law :-)): pi, switch, 3ds, vita, psp, gcw0
• needs testing: ps2, all ios-devices.
  rg350 isn't in the Makefile, however it should work too (standalone version is available from @gameblabla).

[hizzlekizzle]

@irixxxx I think the version is fine for now. If it causes confusion later we can always change it easily enough.

As for testing other platforms, I guess our users will handle that for us! :P

[hizzlekizzle]

@irixxxx We got a few build errors on the ARM platforms, seems to be related to:

pico/draw_arm.S:12:10: fatal error: pico_int_offs.h: No such file or directory
 #include "pico_int_offs.h"

Any thoughts?

[irixxxx]

Is there a warning from mkoffsets.sh in the build output? If so, there's either no readelf, or the file command detects the compiler has no ELF output format. Which platforms are this? edit: I just realized that my approach might fail on ios devices. I'll come up with something for this. The mkoffset script expects a generic readelf. That's available on all linux build hosts, but not on windows. In that case, there might be a readelf in the toolchain. It might be possible to check for generic readelf, then for toolchain readelf. Notify me if there are other platforms without ELF (apart from windows and osx).

[hizzlekizzle]

Looks like the build failures we have are:

1. emscripten http://paste.libretro.com/215792
2. vita http://paste.libretro.com/215788
3. psp http://paste.libretro.com/215786 (looks like the same issue as vita)
4. 3DS http://paste.libretro.com/215784

[irixxxx]

I've pushed some changes which should fix at least some of the lot.

[hizzlekizzle]

Alright, I pushed those up. I'll let you know how it goes. Thanks!

[irixxxx]

Is there more buildbot output but https://travis-ci.org/github/libretro/picodrive? There's only 2 platforms in it, so I presume there is more.
Anyway, I think I've fixed the type incompatibilities seen for 3ds in that output.

[fjtrujy]

Hello,
I'm coming for making the PS2 platform works again!

So far is not compiling either.
I have made some changes in localhost to make it compiles, but the emulation is totally broken.
Sounds are making weird tones, and video is just showing whole screen with a single solid color.

I could have an idea about the video.
https://github.com/libretro/picodrive/blob/master/platform/libretro/libretro.c#L1641-L1643

@irixxxx in order to speed up the performance (have in mind that PS2 just is a MIPS 300 Mhz) I tried to use CLUT palette colors.
This was not working properly in the original notaz repo, so I did some workarounds, I remember you making changes about the palette colors, maybe my changes are no longer needed?

Thanks

[inactive123]

Hi @irixxxx , do you happen to hang out on Discord too? It could perhaps be productive if you were on there and you and fjtrujy could talk there in realtime about this.

Our Discord channel can be found on our website www.retroarch.com - but here is the direct link -

https://discord.com/invite/27Xxm2h

My handle on there is Autechre.

[irixxxx]

@twinaphex I'm on the discord channel. I've had a look at the failed output... I must say the pico Makefile stuff isn't in best shape. I've tried to repair some of it (hopefully without damaging other stuff), but as far as I'm concerned it's still kind of a band aid and I would like to test this with as much platforms as possible. There's maybe more lurking there.
Is there a possibility to use the buildbot to check for fallout before making this official?

@fjtrujy The 8 bit fast renderer only renders to Pico.est.Draw2FB and there's currently no possibility to set Draw2FB to something other then the default buffer in draw2.c.
Could you try the 8 bit accurate renderer (PDF_8BIT)? That has the ability to use a supplied buffer.
If you need PDF_NONE for speed, I can add some code to PicoDrawSetOutBuf to set Draw2FB in case PDF_NONE was selected. However, my experience with PDF_NONE on the gp2x shows only a very modest gain.

[inactive123]

Wha't s your username on Discord? Try contacting fjtrujy in Direct Message on there.

[irixxxx]

@twinaphex it's (surprise!) irixxxx, just to avoid confusion. It's from old times, unfortunately not really fitting anymore. Whatever, I've sent @fjtrujy a message.

[irixxxx]

Just to say it: I've pretty much addressed those problems visible in the failed builds, apart from the android stuff. For the android build framework I need some help since I don't know much about it.
The last commit will most probably help ameliorate things, but ATM I don't have the SDKs for the failing platforms to ensure it really works. It works fine with platform set to osx, ios, windows, linux and gp2x, though.