Linux Kernel Cannot Print where Bootrom and BBL can

[Kendidi]

Hi,

I tried to load and run Bootrom, BBL and Linux Kernel on FPGA.

Bootrom and BBL can print on UART console without any issue, but the Linux Kernel cannot, even it went pass through a few places in setup.c and init.c where it should have printed something on the console. I think the ariane.dts in bootrom should be OK, otherwise BBL may not be able to print.

I wonder:
1) What may be the issue? Where should I look at?
2) How does the UART/console get initialized normally so that functions like pr_info( ) and printk( ) can print?

Thanks a lot in advance!

[jctullos]

@Kendidi

Did you ever find a fix? I'm at the same issue right now. BBL and Bootloader print just fine, and when it hands over to Linux, nothing gets printed. I figured it has to do with the UART, but still trying to work out how to fix it.

[Kendidi]

@jctullos

Not yet. I am still trying to figure it out. I initially thought early_init_dt_scan( ) takes care of UART initialization. But it appears in a Qemu environment, Linux Kernel can already print to console before early_init_dt_scan( ) is launched. Therefore I think the UART config was either done before MMU is enabled, or before Linux Kernel is launched.

[jctullos]

So if you pass earlycon through bootargs in defconfig, it will setup an early console with the uart. Are you running the Open Piton build or just regular CVA6/Ariane? What FPGA?

I'm running the Open Piton build, that uses ns16550, on a VCU118. I think the uart device isn't getting picked up correctly in the device tree. Right now I'm rebuilding the FPGA bitstream. My initial hunch is that the uart in the device tree passes "ns16550" as the compatible name, and maybe "ns16550a" is what the Linux driver is looking for, even though it should work fine with "ns16550". I'll let you know how it goes.

@zarubaf or @Jbalkind Would either of you have seen this issue before?

[Jbalkind]

@jctullos please be sure you're building from the openpiton branch. Also, have you set earlycon=sbi?

[jctullos]

I didn't on this build, but I'll add it. I didn't see it on the openpiton branch for the linux defconfig, that might need to be added if it's required. I'll try it right now.

[Jbalkind]

At what point in the boot process are you not seeing output? With the existing defconfig built from the openpiton branch it should just print. Does the prebuilt bbl.bin work? https://www.princeton.edu/~cloud/openpiton/os_images/openpiton_ariane_linux_r12.tar.gz

[jctullos]

The prebuilt doesn't work either, I just tried again with the link you posted. Here's the output:

[Jbalkind]

Could you share the start of the output too? How did you prepare the SD?

[jctullos]

SD was prepared the way it shows on the openpiton repo. Here's the start:

[jctullos]

I am also working on another bbl build, it will print the device tree just fine, but hangs when Linux normally takes over. This is in minit.c and after which nothing outputs.

At the jump to payload. I had a debug statement there previously which was the last thing it printed before jumping to the payload.

[Jbalkind]

The output looks reasonable. Partition table is correct, bbl is clearly located in the right place, and system frequency is reported correctly which means the ariane.dts should also be correct. Since the partition table is correct, the SD reading clearly isn't totally broken... Timing was probably met too I'm going to guess?

Looks like you're building from openpiton-dev... Have you done a git submodule update --init --recursive piton/design/chip/tile/ariane by any chance?

I don't really trust bbl to be honest. It has a kernel memory corruption bug which has caused us some strife. You could try opensbi instead? I shared some instructions here previously: https://groups.google.com/g/openpiton/c/8J9IX1VLRHA/m/poid3PVbCgAJ

[jctullos]

I haven't done an update in a few days, I'll check it out.

Also, earlycon=sbi didn't change anything. And I'd like to move to opensbi, but my research is using Keystone and they still use bbl unfortunately.

I've seen this issue in the past, I just can't remember how I fixed it. I had issues with 64 bit memory addressing in the dts file getting parsed correctly. That would have been my next attempt, to change address and size cells to 1 instead of 2. But the openpiton build has high addressing, so I'm assuming that's out unless it can be easily changed.

[Jbalkind]

There shouldn't be any new ariane updates in terms of our submodule pointer, it was more in case you were going from the openpiton branch to openpiton-dev and hadn't got the accompanying changes in doing so.

That's a shame. I'll warn you that linux boot can be flaky with bbl. If you see random stalls and stack traces, it's likely to be bbl's fault. When we switched to opensbi, all of that went away. This was a problem both for standalone ariane and op+ariane. Others had reported the same for rocket I believe, too.

You can move the ariane peripherals to a different location by changing the base addresses in piton/design/xilinx/vcu118/devices_ariane.xml and see if that helps? But we were using bbl for quite a while and I don't remember seeing this issue. That does make me suspect that perhaps not all of the SD woes have passed. May be worth trying our sdctrl_test design on vcu118 and seeing if it passes?

[Kendidi]

@jctullos

I use Xilinx FPGA, bootrom.elf built off from the CVA6 fpga tree and bbl built from ariane-sdk.

[Kendidi]

@Jbalkind and @jctullos

I initially thought bootrom.elf (from cva6/fpga/) + bbl (from ariane-sdk) should be enough for booting Linux on the Xilinx FPFA we built. Is bbl not good enough?
What approach would you recommend? Thanks.

[Kendidi]

@jctullos

Please let me know if you have any success or have found anything with your bbl build. Thanks.

[Kendidi]

@jctullos

So if you pass earlycon through bootargs in defconfig, it will setup an early console with the uart.

Can you please elaborate?

What I have in "\ariane-sdk\buildroot\output\build\linux-ariane-v0.7\arch\riscv\configs\defconfig":

• CONFIG_FIX_EARLYCON_MEM=y
• CONFIG_SERIAL_EARLYCON=y
• CONFIG_SERIAL_EARLYCON_RISCV_SBI=y

So which routine is supposed to setup the console?

[Moschn]

Also, earlycon=sbi didn't change anything. And I'd like to move to opensbi, but my research is using Keystone and they still use bbl unfortunately.

@jctullos I have also noticed no uart output with keystone on stock Ariane (without openPiton). If I remember correctly I solved this with using a newer linux kernel version. I do not know where this comes from though. I would advise you to try a few different kernel versions (I believe I used 5.3).

[Kendidi]

@Moschn

Where in Ariane SDK we can select which Linux Kernel version to use?

[Kendidi]

According to https://github.com/pulp-platform/linux/, Linux 5.1-rc7 appears to be the latest.

[Moschn]

@Kendidi It is not supported to select this. You need to build it manually. I guess you could take a look at our build process and try to adapt it to an upstream kernel.

[jctullos]

thanks @Moschn although I'm using 5.3 right now for Keystone, so there must be something more.

[jctullos]

@Jbalkind

I think you might be right regarding the SD issues. After some random resets, Linux will boot. It's not consistent. So I assume data isn't getting copied over all the time. It hasn't completed a boot to the login prompt yet, right now it's failing on the RPC module, but at least there's output.

[Kendidi]

@jctullos

So what changes did you make so it can print now?

[Kendidi]

@Moschn

Ok. Thanks. After I downloaded Kernel 5.3, how should I modify before compiling it?

[jctullos]

@Kendidi I thought I replied yesterday, sorry!

So I'm using a Linux 5.3 build. And passing console=hvc and earlycon=sbi to bootargs. FYI, Linux boots maybe 1/3rd of the time. And it always lags by a few seconds before it starts printing.

[Kendidi]

Thank you @jctullos!! I will try to find a suitable copy of Linux 5.3 and try it.

[Kendidi]

Besides "console=hvc" and "earlycon=sbi", what else do we need to put into bootargs?

[jctullos]

@Jbalkind

So I believe I've fixed the issue with Linux not showing the console. I remembered that I also had issues with the memory still holding data that wasn't getting cleared. I wrote a small while loop in the bootloader that just clears the memory where BBL is going, and then the sd_copy starts. So far, this has fixed it. There's no more issues between BBL and Linux handover.

One question though, I still have issues with the Linux boot stopping halfway. Looking at the vmlinux system map, it seems to be when clock source switches over. Earlier, I had changed the timebased-frequency value in the DTS to half of the clock frequency, this was only because the Ariane/CVA6 repo had a commit that fixed their DTS to half the system clock.

Does openpiton use another RTC build for the timebased-frequency? I know in the DTS generation script, it's the Sysclock/128. I'm changing it back to the default openpiton setting to see if this fixes Linux booting. But wanted to verify the frequency.

[Kendidi]

@jctullos

I am using Kernel v5.3. I have added the following to ariane.dts but it's still not printing once Kernel has taken over

bootargs = "console=hvc earlycon=sbi";

Is there anything else I have missed?

[Kendidi]

Alright, I have output from Linux now. For the ESP build, add these config options to the ariane defconfig and let me know if it works for you:

CONFIG_HVC_DRIVER=y
CONFIG_SERIAL_EARLYCON_RISCV_SBI=y

CONFIG_CMDLINE="earlyprintk console=hvc earlycon=sbi"

There might have been one other, but I can't remember offhand. But now I have output from Linux again.

Thank you @jctullos !! I will try it out.

[Kendidi]

@jctullos

I've added those 3 config options to "../esp/soft/ariane/linux/arch/riscv/configs/ariane_defconfig". Re-run "make linux". Then used the resulted bbl. But got same result - no printout at Linux land, and stuck at "wfi" in .

Did I miss anything?

[Jbalkind]

Being stuck at wfi to me suggests that your CLINT/PLIC may not be working. Usually the core will sit there waiting to be woken up by some interrupt like a timer or IPI from the CLINT.

Are you using ariane-fpga? Is it up to date? I don't necessarily have much assistance to give in debugging it since I don't use Ariane's FPGA environment, but if you're definitely stuck at wfi I'd probably be looking into interactions with the CLINT.

[Kendidi]

@Jbalkind

Thank you for your advice! Yes I am using FPGA - Xilinx VCU128. It's pretty updated I think. Which version would you recommend? Should we used the latest in repository - "master" or the last release - Ariane 4.2?

[Kendidi]

By the way, since it's a single core setup, who's suppose to wake up Ariane?

[Jbalkind]

Ariane master is probably a good call. There have been a few important changes recently though (mainly the PMPs and 32 bit support) and so I'm not sure what state everything is in.

If it's single core, the core can just be waiting for a timer that it set for itself. It's possible that that wfi is just the OS's idle loop. How did you check the instruction?

[Kendidi]

Thanks @Jbalkind.

I used GDB to run the code. I can see printout from Bootrom and from bbl. But nothing from Linux. I stopped GDB and it showed:

(gdb) c
Continuing.
^C
Program received signal SIGINT, Interrupt.
0xffffffe000000124 in ?? ()
(gdb)

I used riscv64-unknown-elf-objdump to disassemble vmlinux and it shows:
ffffffe00000007a :
......
......
ffffffe000000120: 2aa0206f j ffffffe0000023ca <smp_callin>
ffffffe000000124: 10500073 wfi
ffffffe000000128: bff5 j ffffffe000000124 <relocate+0xaa>

[Jbalkind]

Looks like that corresponds with this (assuming kernel 5.1): https://elixir.bootlin.com/linux/v5.1/source/arch/riscv/kernel/head.S#L162

Weird place to be stuck. Maybe try disabling CONFIG_SMP if you're not using it?

[Kendidi]

Yes, it's 5.1
After CONFIG_SMP is disabled in config file. It still stuck at instruction "wfi". Address of the instructions shifted though.

(gdb) c
Continuing.
^C
Program received signal SIGINT, Interrupt.
0xffffffe0000000e0 in ?? ()
(gdb)



ffffffe0000000de:       0001                    nop
ffffffe0000000e0:       10500073                wfi
ffffffe0000000e4:       bff5                    j       ffffffe0000000e0 <relocate+0x66>

[Jbalkind]

If you look at the comments above where that's put in the trap vector here: https://elixir.bootlin.com/linux/v5.1/source/arch/riscv/kernel/head.S#L112 then you can see it's intended to help debug a particular error case. Can you maybe try and get the trap pc from gdb and see what it was executing before this happened?

Honestly I find this case you're stuck in pretty weird. I would again encourage you to use opensbi if there's any possibility of that for your use case.

[Kendidi]

Not sure how to get the trap PC from GDB since BT doesn't in current environment.

I have considered replacing bbl with opensbi. I just don't know where to start. Is there any documents/website demonstrate a good way to do it? Thanks.

[Kendidi]

With Kernel v5.3 and bbl from ariane-sdk, it stuck in memset() instead.

[Kendidi]

@jctullos

In my situation, the following are required for Kernel to print messages onto the console. Somehow SMP needs to be off for the other changes to be effective.

CONFIG_SMP=n

CONFIG_HVC_DRIVER=y
CONFIG_SERIAL_EARLYCON_RISCV_SBI=y

CONFIG_CMDLINE="earlyprintk console=hvc earlycon=sbi"

Unfortunately, there are still some issues that forbid it to boot all the way. Have you encountered that before? Thanks.

[    0.000000] OF: fdt: Ignoring memory range 0x80000000 - 0x80200000
[    0.000000] Linux version 5.5.0 (root@aie-Machine) (gcc version 8.2.0 (GCC))0
[    0.000000] earlycon: sbi0 at I/O port 0x0 (options '')
[    0.000000] printk: bootconsole [sbi0] enabled
[    0.000000] initrd not found or empty - disabling initrd
[    0.000000] Zone ranges:
[    0.000000]   DMA32    [mem 0x0000000080200000-0x00000000bfffffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080200000-0x00000000bfffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080200000-0x00000000bfffffff]
[    0.000000] software IO TLB: mapped [mem 0xbb1fc000-0xbf1fc000] (64MB)
[    0.000000] elf_hwcap is 0x112d
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 258055
[    0.000000] Kernel command line: earlyprintk console=hvc earlycon=sbi
[    0.000000] Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes)
[    0.000000] Inode-cache hash table entries: 65536 (order: 7, 524288 bytes, l)
[    0.000000] Sorting __ex_table...
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] BUG: Bad page state in process swapper  pfn:80ce4
[    0.000000] page:ffffffe03f02c1e0 refcount:1 mapcount:0 mapping:0000000000000
[    0.000000] raw: 0000000000000000 ffffffe03f02c1e8 ffffffe03f02c1e8 000000000
[    0.000000] raw: 0000000000000000 0000000000000000 00000001ffffffff
[    0.000000] page dumped because: nonzero _refcount
[    0.000000] Modules linked in:
[    0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 5.5.0 #5
[    0.000000] Call Trace:
[    0.000000] [<ffffffe0005f1158>] walk_stackframe+0x0/0xa0
[    0.000000] [<ffffffe0005f133a>] show_stack+0x2a/0x34
[    0.000000] [<ffffffe000938f46>] dump_stack+0x20/0x28
[    0.000000] [<ffffffe00065a8fc>] bad_page+0xe0/0xfe
[    0.000000] [<ffffffe00065a952>] free_pages_check_bad+0x38/0x7a
[    0.000000] [<ffffffe00065af62>] __free_pages_ok+0xa4/0x4c2
[    0.000000] [<ffffffe00065c220>] __free_pages.part.142+0x1a/0x22
[    0.000000] [<ffffffe00065c2e4>] __free_pages_core+0x94/0xa0
[    0.000000] [<ffffffe000006bbe>] memblock_free_pages+0x12/0x1a
[    0.000000] [<ffffffe0000083f8>] memblock_free_all+0x14e/0x1ae
[    0.000000] [<ffffffe0000026a0>] mem_init+0x2a/0x38
[    0.000000] [<ffffffe0000008d2>] start_kernel+0x1f6/0x394
[    0.000000] [<ffffffe0000000bc>] clear_bss_done+0x3c/0x40
[    0.000000] Disabling lock debugging due to kernel taint
[    0.000000] BUG: Bad page state in process swapper  pfn:80ce8
[    0.000000] page:ffffffe03f02c2c0 refcount:1 mapcount:0 mapping:0000000000000
[    0.000000] raw: 0000000000000000 ffffffe03f02c2c8 ffffffe03f02c2c8 000000000
[    0.000000] raw: 0000000000000000 0000000000000000 00000001ffffffff
[    0.000000] page dumped because: nonzero _refcount
......
......

[jctullos]

@Kendidi

Hey sorry I didn't realize the progress you've made! I should have said one of the other configure options I had was SMP off. I've seen it in the past where there's a bug that if only one core is running and SMP is on, it hangs.

What commit are you building the Ariane processor from on the CVA6 repo?

[jctullos]

So here's my suggestion also. If you're using the OpenPiton or ESP build, and you don't need PMP, then stick to whichever Ariane version they have linked. There's an issue with PMP that I believe isn't getting captured up at the top level builds for those two, which was why the ESP folks added an issue to track it.

If you don't need PMP, then keep the Ariane at the same commit as the OpenPiton/ESP builds. Also keep Linux on the 5.1 or 5.3, it says in your boot logs that you're loading Linux 5.5.

The problem with this is that the Linux defconfig in the Ariane SDK is based on Linux 5.1. Same with the riscv-pk (bbl) is older. So there were a bunch of changes that happened to go to 5.5, and I believe riscv-pk had been updated as well. Some of the references in the bbl code, or the encoding.h, get updated with new values that are used in a newer version of Linux. So having 2 different versions of the bbl and Linux can work, but sometimes going too far apart it will mess with it.

If you do need PMP, with the vcu128 you might be out of luck right now. There's a post on the CVA6 issues where a guy was able to get a VCU108 working and he has a repo. You could follow his changes to get your VCU128 going. I did the same for the VCU118 but was having some uart issues. And I got the vc707 working so I've switched over to it.

[Kendidi]

@jctullos

Thank you very much for your advice!

What commit are you building the Ariane processor from on the CVA6 repo?

I have been using Ariane CVA6 commit #453 since Thursday because I think it's prior to PMP support and it was recommended by the ESP folks.

it says in your boot logs that you're loading Linux 5.5.
Yeah, I didn't see any improvement on the issues with 5.1 and 5.3, so I tried 5.5 to see if there'll be difference.

So what setup/config you have now that will allow Kernel to boot successfully? Are you still having SMP turned off and with the other 3 config option set?

[jctullos]

Can you link it here the commit url for CVA6?

Using the default Ariane SDK build (probably worth it to wipe it and start with a fresh repo) the only changes I made were to the Linux defconfig are:

The changes I did make to the Ariane SDK linux_defconfig are:

CONFIG_DEFAULT_HOSTNAME="ariane-fpga"
#CONFIG_CROSS_MEMORY_ATTACH is not set
CONFIG_NAMESPACES=y
CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE="${BR_BINARIES_DIR}/rootfs.cpio"
CONFIG_EMBEDDED=y
CONFIG_SMP=y
CONFIG_HZ_100=y
CONFIG_CMDLINE="earlyprintk console=hvc earlycon=sbi"
CONFIG_MODULES=y
#CONFIG_BLK_DEV_BSG is not set
CONFIG_PARTITION_ADVANCED=y
#CONFIG_COMPACTION is not set
CONFIG_NET=y
CONFIG_PACKET=y
CONFIG_UNIX=y
CONFIG_TLS=y
CONFIG_INET=y
CONFIG_IP_MULTICAST=y
#CONFIG_WIRELESS is not set
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
CONFIG_VIRTIO_BLK=y
CONFIG_EEPROM_AT24=y
CONFIG_NETDEVICES=y
CONFIG_VXLAN=y
#CONFIG_LOWRISC_DIGILENT_100MHZ=y
#CONFIG_XILINX_EMACLITE=y
#CONFIG_MDIO_BCM_UNIMAC=y
#CONFIG_MDIO_BITBANG=y
#CONFIG_MDIO_BUS_MUX_GPIO=y
#CONFIG_MDIO_BUS_MUX_MMIOREG=y
#CONFIG_MDIO_BUS_MUX_MULTIPLEXER=y
#CONFIG_MDIO_GPIO=y
#CONFIG_REALTEK_PHY=y
#CONFIG_WLAN is not set
#CONFIG_INPUT_KEYBOARD is not set
#CONFIG_INPUT_MOUSE is not set
#CONFIG_VT is not set
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SERIAL_OF_PLATFORM=y
CONFIG_SERIAL_UARTLITE=y
CONFIG_HVC_RISCV_SBI=y
CONFIG_VIRTIO_CONSOLE=y
#CONFIG_HW_RANDOM is not set
CONFIG_I2C=y
CONFIG_I2C_OCORES=y
#CONFIG_SPI=y
#CONFIG_SPI_XILINX=y
CONFIG_GPIOLIB=y
CONFIG_GPIO_SYSFS=y
CONFIG_GPIO_XILINX=y
CONFIG_POWER_RESET=y
CONFIG_POWER_RESET_GPIO_RESTART=y
CONFIG_USB=y
CONFIG_USB_XHCI_HCD=y
CONFIG_USB_EHCI_HCD=y
#CONFIG_MMC=y
#CONFIG_MMC_SPI=y
CONFIG_NEW_LEDS=y
CONFIG_LEDS_CLASS=y
CONFIG_LEDS_PWM=y
CONFIG_LEDS_TRIGGERS=y
CONFIG_LEDS_TRIGGER_HEARTBEAT=y
CONFIG_LEDS_TRIGGER_PANIC=y
CONFIG_VIRTIO_MMIO=y
#CONFIG_IOMMU_SUPPORT is not set
CONFIG_PWM=y
CONFIG_SIFIVE_PLIC=y
CONFIG_EXT3_FS=y
#CONFIG_PROC_PAGE_MONITOR is not set
CONFIG_TMPFS=y
#CONFIG_MISC_FILESYSTEMS is not set
CONFIG_NFS_FS=y
CONFIG_NFS_V4=y
CONFIG_NFS_SWAP=y
CONFIG_NFS_V4_1=y
CONFIG_NFS_V4_2=y
CONFIG_CRYPTO_ECHAINIV=y
#CONFIG_CRYPTO_HW is not set
CONFIG_PRINTK_TIME=y
CONFIG_STRIP_ASM_SYMS=y
CONFIG_DEBUG_SECTION_MISMATCH=y
CONFIG_STACKTRACE=y

So all I did was remove the ethernet items they have listed. I even kept the SMP it looks like. But this allowed the VCU118 to build and boot, so it should work for you as well.

[Kendidi]

By the way, I recall if I set the following options in the ESP environment, then Kernel will crash during boot the same way as my Ariane now does.

CONFIG_HVC_DRIVER=y
CONFIG_SERIAL_EARLYCON_RISCV_SBI=y

CONFIG_CMDLINE="earlyprintk console=hvc earlycon=sbi"

Have you experienced that? To me, seems like these options help printing messages on the console, but somehow will generate issues later on on both Ariane and ESP platforms.

[Kendidi]

Can you link it here the commit url for CVA6?

https://github.com/openhwgroup/cva6/pull/453/files

Thank you very much for the config option list! I will try it.

[Kendidi]

@jctullos

No luck so far. I downloaded SDK. Replaced default linux_defconfig with yours and compiled bbl.

Below is the DST file I am using. The clock frequency we are using is 40MHz and time-base frequency is 20MHz. Our design doesn't support ethernet. It shouldn't matter if we keep CONFIG_NET enabled. Right?

/dts-v1/;

/ {
  #address-cells = <2>;
  #size-cells = <2>;
  compatible = "eth,ariane-bare-dev";
  model = "eth,ariane-bare";
  chosen {
    stdout-path = "/soc/uart@10000000:115200";
  };
  cpus {
    #address-cells = <1>;
    #size-cells = <0>;
    timebase-frequency = <20000000>; // 20 MHz
    CPU0: cpu@0 {
      clock-frequency = <40000000>; // 40 MHz
      device_type = "cpu";
      reg = <0>;
      status = "okay";
      compatible = "eth, ariane", "riscv";
      riscv,isa = "rv64imafdc";
      mmu-type = "riscv,sv39";
      tlb-split;
      // HLIC - hart local interrupt controller
      CPU0_intc: interrupt-controller {
        #interrupt-cells = <1>;
        interrupt-controller;
        compatible = "riscv,cpu-intc";
      };
    };
  };
  memory@80000000 {
    device_type = "memory";
    reg = <0x0 0x80000000 0x0 0x40000000>;
  };
//   leds {
//     compatible = "gpio-leds";
//     heartbeat-led {
//       gpios = <&xlnx_gpio 1 0>;
//       linux,default-trigger = "heartbeat";
//       retain-state-suspended;
//     };
//   };
  L26: soc {
    #address-cells = <2>;
    #size-cells = <2>;
    compatible = "eth,ariane-bare-soc", "simple-bus";
    ranges;
    clint@2000000 {
      compatible = "riscv,clint0";
      interrupts-extended = <&CPU0_intc 3 &CPU0_intc 7>;
      reg = <0x0 0x2000000 0x0 0xc0000>;
      reg-names = "control";
    };
    PLIC0: interrupt-controller@c000000 {
      #address-cells = <0>;
      #interrupt-cells = <1>;
      compatible = "riscv,plic0";
      interrupt-controller;
      interrupts-extended = <&CPU0_intc 11 &CPU0_intc 9>;
      reg = <0x0 0xc000000 0x0 0x4000000>;
      riscv,max-priority = <7>;
      riscv,ndev = <30>;
    };
    debug-controller@0 {
      compatible = "riscv,debug-013";
      interrupts-extended = <&CPU0_intc 65535>;
      reg = <0x0 0x0 0x0 0x1000>;
      reg-names = "control";
    };
    uart@10000000 {
      compatible = "ns16750";
      reg = <0x0 0x10000000 0x0 0x1000>;
      clock-frequency = <40000000>;
      current-speed = <115200>;
      interrupt-parent = <&PLIC0>;
      interrupts = <1>;
      reg-shift = <2>; // regs are spaced on 32 bit boundary
      reg-io-width = <4>; // only 32-bit access are supported
    };
    timer@18000000 {
      compatible = "pulp,apb_timer";
      interrupts = <0x00000004 0x00000005 0x00000006 0x00000007>;
      reg = <0x00000000 0x18000000 0x00000000 0x00001000>;
      interrupt-parent = <&PLIC0>;
      reg-names = "control";
    };
//     xps-spi@20000000 {
//       compatible = "xlnx,xps-spi-2.00.b", "xlnx,xps-spi-2.00.a";
//       #address-cells = <1>;
//       #size-cells = <0>;
//       interrupt-parent = <&PLIC0>;
//       interrupts = < 2 2 >;
//       reg = < 0x0 0x20000000 0x0 0x1000 >;
//       xlnx,family = "kintex7";
//       xlnx,fifo-exist = <0x1>;
//       xlnx,num-ss-bits = <0x1>;
//       xlnx,num-transfer-bits = <0x8>;
//       xlnx,sck-ratio = <0x4>;
// 
//       mmc@0 {
//         compatible = "mmc-spi-slot";
//         reg = <0>;
//         spi-max-frequency = <12500000>;
//         voltage-ranges = <3300 3300>;
//         disable-wp;
//       };

      // mmc-slot@0 {
      //   compatible = "fsl,mpc8323rdb-mmc-slot", "mmc-spi-slot";
      //   reg = <0>;  //Chip select 0
      //   spi-max-frequency = <12500000>;
      //   voltage-ranges = <3300 3300>;
      //   //interrupts = < 2 2 >;
      //   //interrupt-parent = <&PLIC0>;
      // };
//     };
//     eth: lowrisc-eth@30000000 {
//       compatible = "lowrisc-eth";
//       device_type = "network";
//       interrupt-parent = <&PLIC0>;
//       interrupts = <3 0>;
//       local-mac-address = [00 18 3e 02 e3 7f]; // This needs to change if more than one GenesysII on a VLAN
//       reg = <0x0 0x30000000 0x0 0x8000>;
//     };
    xlnx_gpio: gpio@40000000 {
      #gpio-cells = <2>;
      compatible = "xlnx,xps-gpio-1.00.a";
      gpio-controller ;
      reg = <0x0 0x40000000 0x0 0x10000 >;
      xlnx,all-inputs = <0x0>;
      xlnx,all-inputs-2 = <0x0>;
      xlnx,dout-default = <0x0>;
      xlnx,dout-default-2 = <0x0>;
      xlnx,gpio-width = <0x8>;
      xlnx,gpio2-width = <0x8>;
      xlnx,interrupt-present = <0x0>;
      xlnx,is-dual = <0x1>;
      xlnx,tri-default = <0xffffffff>;
      xlnx,tri-default-2 = <0xffffffff>;
    };
  };
};

[jctullos]

Which build are you using? OpenPiton or ESP?

[Kendidi]

CVA6 commit #453.
Which one are you using?

[jctullos]

So you customized the CVA6 build for a VCU128? You're not relying on ESP/OpenPiton?

FYI I didn't get my VCU118 working on the CVA6, it builds but the onboard uart wasn't working for me. Could you upload your build to github for the vcu128 for CVA6? If yours works I could take a look at why my uart doesnt.

[Kendidi]

We thought CVA6 is the simplest therefore should be easiest to work with.

I only have info on the device tree. I do not have source code to the build.

[Kendidi]

For ESP builds, which of the following is being using?

/esp/third-party/ariane/bootrom
/esp/third-party/ariane/fpga/src/bootrom
/esp/third-party/ariane/openpiton/bootrom
/esp/soft/ariane/bootrom

[paulmnt]

/esp/soft/ariane/bootrom

If you look at the Make targets in /utils/Makefile, you should be able to navigate all the dependencies. Also, adding the flag -n to any make target will give you the exact command lines used.

e.g.

make soft -n

The other folders exist because they are part of the Ariane's repository.

The ESP-Ariane bootloader /esp/soft/ariane/bootrom is very similar to the original one in /esp/third-party/ariane/fpga/src/bootrom, but we removed code that is unnecessary for ESP and applied minor patches to support the APB UART from GRLIB.

[Kendidi]

Cool! Thank you @paulmnt !!

[jimaandro]

@Jbalkind

Thank you for your advice! Yes I am using FPGA - Xilinx VCU128. It's pretty updated I think. Which version would you recommend? Should we used the latest in repository - "master" or the last release - Ariane 4.2?

Hello @Kendidi
I am trying to do something similar. I want to boot Linux on OpenPiton, using VCU128.
My question is, how did you conncet JTAG in order to use openOCD?
What constraints did you use, and did you change something in verilog files?
And what .cfg file did you used for openOCD?
Also did you change the constraints for UART?
Because UART and JTAG use the same port on VCU128