Need help for bits2rbt

[MidsummerNight]

Hi, I was trying to use bits2rbt in tools/bits2rbt to convert a 7 series .bit bitstream into a .rbt bitstream for inspection (I assume that's the use of this tool because of its name). I ran it like this:

me@me-machine:~/prjxray/build/tools/bits2rbt$ ./bits2rbt top.bit -o top.rbt
terminate called after throwing an instance of 'std::out_of_range'
  what():  vector::_M_range_check: __n (which is 480732416) >= this->size() (which is 101)
Aborted (core dumped)
me@me-machine:~/prjxray/build/tools/bits2rbt$ 

I am currently debugging the source code to see what's happening, but since this tool lacks any form of documentation, I'm having a very hard time.

First, can anyone please help me get a general understanding of what this tool does? Is it truly for converting .bit files into .rbt files? Did I write the command correctly? And what's the "auxilary file" the --aux flag stands for? Is it mandatory?

Thanks! @tmichalak @rw1nkler !

[tmichalak]

@MidsummerNight this tool is used to convert the .bits and accompanying .aux file into .rbt. In order to generate the .bits and corresponding .aux files another tool called bitread is required. Refer to the test makefile to see an example. The .bits file alone only contains information about the fpga configuration to program the design, but doesn't have other metadata like the part number or bus width which is needed to construct a valid .rbt file. This is what the .aux file is for.

[MidsummerNight]

Thanks! So bitread takes in a .bit and part.yaml then outputs a .bits and .aux file, which are then sent into bits2rbt to be converted into the .rbt file. Basically:

build/tools/bitread -part_file build/Series7/part.yaml -architecture Series7 -y -o build/xc7.bits -aux build/xc7.aux build/Series7/design.bit
build/tools/bits2rbt/bits2rbt -arch Series7 -aux build/xc7.aux -o build/xc7.rbt build/xc7.bits 
diff <(tail -n +8 build/Series7/design.rbt) <(tail -n +8 build/xc7.rbt)

The example design.bit was indeed correctly converted, however when I tried to convert my own .bit file (generated by Vivado for the xc7a50tfgg484-1 which successfully ran on my board so no problem on that side). I did the conversion first with a modified Makefile.tools_test, then by manually running the commands, the results were the same:

build/tools/bitread -part_file build/Series7/part.yaml -architecture Series7 -y -o build/led_test_linux.bits -aux build/a50t_linux.aux build/Series7/led_test_linux.bit
build/tools/bits2rbt/bits2rbt -arch Series7 -aux build/a50t_linux.aux -o build/led_test_linux.rbt build/led_test_linux.bits

Which got me this (took out unrelated info for clarity):

~/prjxray/build$ ls -al
-rw-rw-r--  1 me me  4981484 a50t.aux
-rw-rw-r--  1 me me  4981502 a50t_linux.aux
-rw-rw-r--  1 me me   101500 led_test.bits
-rw-rw-r--  1 me me   100780 led_test_linux.bits
-rw-rw-r--  1 me me 37024857 led_test_linux.rbt
-rw-rw-r--  1 me me 37024851 led_test.rbt
-rw-rw-r--  1 me me    54338 xc7.aux
-rw-rw-r--  1 me me  3197400 xc7.bits
-rw-rw-r--  1 me me 18958699 xc7.rbt

(My files that are denoted by _linux are converted from led_test_linux.bit while the others are from led_test.bit, see below for details)
Compared with the example results of design.bit, my .rbt is nearly twice as large, .aux is several times the example's size, and my .bits is a fraction of xc7.bits. The bit-->rbt flow worked, but clearly the the right way.

Adding to the confusion is that my own .rbt files generated by Vivado has a slightly different size than the examples', which shouldn't be the case:

~/prjxray/build/Series7$ ls -al
-rw-r-----  1 me me  2298099 bram.bit
-rw-r-----  1 me me 18958760 bram.rbt
-rw-rw-r--  1 me me  2298099 design.bit
-rw-rw-r--  1 me me 18958760 design.rbt
-rwxrwx---  1 me me  2192116 led_test.bit
-rwxrwx---  1 me me 18632374 led_test_gold.rbt
-rw-rw-r--  1 me me  2192122 led_test_linux.bit
-rw-rw-r--  1 me me 18084370 led_test_linux_gold.rbt

(led_test_gold.rbt was generated by Vivado 2021.1 on Windows 10 and led_test_linux_gold.rbt by Vivado 2017.2 on Ubuntu 20.04, both from the same source project).

A starting section of the converted .rbt file is identical to Vivado's .rbt file, but things start to go wrong some several thousands/ten-thousands lines later. Do you have any idea of what's happening? Thank you again! @tmichalak

[mithro]

As far as I can see, you haven't actually explained what is actually wrong?

[MidsummerNight]

@mithro My result .rbt file output by the tools is nearly twice as long as the one produced by Vivado (whereas the xc7.rbt generated by the example is identical with the original design.rbt). When trying to program my board (via Vivado) with my .rbt file converted from my .bit file by the tools, Vivado repeatedly reports
ERROR: [Bitstream 40-65] Trying to read past end of bitstream. The bitstream may be corrupted.
(presumably because the .rbt is longer than it should be) followed by

INFO: [Labtools 27-3164] End of startup status: HIGH
ERROR: [Common 17-39] 'program_hw_devices' failed due to earlier errors.

although my design does work on the board despite those errors.

Whatever's happening, the result .rbt I converted from my .bit (my .bit was also generated by Vivado) is drastically different from the .rbt generated by Vivado.

[mithro]

My random guess is that it seems likely that your Vivado generated bit files have stuff not currently supported by prjxray. If Vivado is using something in the design which prjxray can't support then things are likely to go wrong.

prjxray tools are also currently tested only with Vivado 2017.3

[MidsummerNight]

@mithro Wait what 2017.3? I thought it was 2017.2?

The example's design.rbt states it was generated by 2017.2, so presumably the design.bit was also from 2017.2. My .bit was from 2017.2 as well, yet design.rbt was correctly converted whereas my .bit wasn't (at least not 100% correctly). My design is a very simple LED counter, so it shouldn't be caused by prjxray not supporting complex blocks either.

[mithro]

@mithro Wait what 2017.3? I thought it was 2017.2?

You are probably right...

[MidsummerNight]

@mithro Update: For the xc7a50tfgg484-1 part, the Vicado-generated .rbt file has 548010 lines, while the xray-generated .rbt file has 1121966 lines. Line 70 to 574025 of the xray .rbt are excess (though I cannot figure out what they are right now), once I manually delete them, the xray rbt will be the same as the Vivado one and can be written to my board without any issues.

@tmichalak Do you have the source files used to generate the example design.bit and the bram.bit? I would like to check if the source code has anything to do with this. By the way do you have any idea why the tools are adding so many lines? Have you tested these tools on other .bit files before?

[tmichalak]

@MidsummerNight so for testing the bits2rbt flow the basic fuzzer designs were used, .i.e. xc7 and bram. What I can tell from the sizes of your .bit and .rbt files you are missing the PERFRAMECRC setting in your vivado script as here. bits2rbt heavily depends on this setting to correctly calculate the CRC values.

[MidsummerNight]

@tmichalak I see. Thanks, I'll check this tomorrow! Meanwhile for what it's worth, I found that line 26 to 1002445 of the converted .rbt file of xc7a100tcsga324-1 are excessive, and once they're manually removed, the .rbt would load as normal.

[MidsummerNight]

@tmichalak Update: By setting the PERFRAMECRC for Vivado, the Vivado .bit file increased from 3.8MB to 4.0MB, and the .rbt converted from the .bit is now the same as Vivado's .rbt.

Does SymbiFlow has an equivalent CRC setting that can be used during bitstream generation though? My original intent was to convert SymbiFlow-generated .bits into .rbts, but adding set_property BITSTREAM.GENERAL.PERFRAMECRC YES [current_design] into the XDC constraint doesn't work (SymbiFlow seems to just ignore this setting and goes on to produce a normal no-crc .bit file). The .rbt converted from SymibiFlow's .bit (for xc7a100tcsga324-1) file is again nearly twice as large as it should be, able to be loaded onto the board despite error reports, except this time the excessive section seems to be different (after removing line 26 to 1002445, the .rbt can be programed into the board via Vivado without warnings, yet the board has no reaction).

[tmichalak]

@MidsummerNight so the bits2rbt tool was meant for verifying if bitread is reading the Vivado generated bitstream correctly and that the bits files are correct since the fuzzers heavily depend on them. With bits2rbt it is possible to make a roundtrip test.
There is no way of setting PERFRAMECRC in the SymbiFlow bitstream yet. The xc7frames2bit tool which is constructs the bitstream concatenates the frame information into a Packet Type 2, so a packet where the frame addresses are auto incremented and the CRC is at the end. This is also why set_property BITSTREAM.GENERAL.PERFRAMECRC YES is currently ignored in the XDC reader.

[MidsummerNight]

@tmichalak Thank you for the explanation, I think I have a much better understanding on the tool now. One more question though (sorry if this is sounds dumb), what is the "Packet Type" you're talking about? I mean I know Packet Type 2 is a packet where the frame addresses are auto incremented and the CRC is at the end since you said so, but in what context are we using the term "Packet Type"? Are we talking about packet types specific to 7-series bitstreams, or is this referring to general CRC algorithms that's also appliable to other fields like computer networks? I was unable to find any satifying answers on Google and would really appreciated it if you could point out some explanatory links for me.

[tmichalak]

@MidsummerNight I was referring to the packet types specific to 7-series as described in 7-Series FPGAs Configuration User Guide (UG470) p.108. It is how the bitstream is created in SymbiFlow.

[MidsummerNight]

@tmichalak Thank you very much! I'll close this issue now since my initial problems were solved.