Cannot build the Quadcore with NVDLALarge

[franktaTian]

Hi,
I am trying to rebuild some HW configs on AMI 1.60 which supports Vivado 2018.03.op (Not the 2018.02)
Here is the how to ：
I download the firesim and firesim-nvdla . Setup up the firesim environment successfully

Under the firesim-nvdla/sim
export DESIGN=FireSimNoNIC
export TARGET_CONFIG=FireSimRocketChipSingleCoreConfig_WithNVDLALarge
export PLATFORM_CONFIG=FireSimDDR3FRFCFSLLC4MBConfig75MHz
make replace-rtl

Copy the generated directory to firesim as follows :
cp -r /home/centos/src/project_data/firesim-nvdla/platforms/f1/aws-fpga/hdk/cl/developer_designs/cl_FireSimNoNIC-FireSimRocketChipSingleCoreConfig_WithNVDLALarge-FireSimDDR3FRFCFSLLC4MBConfig75MHz /home/centos/src/project_data/firesim/platforms/f1/aws-fpga/hdk/cl/developer_designs/

Then build this design under firesim : pass .
Create the load the AFI :passed.
make f1 ： passed

And run simulation successfully!
sudo ./FireSim-f1 +permissive $(sed ':a;N;$!ba;s/\n/ /g' runtime.conf) +macaddr0=00:12:6D:00:00:02 +slotid=0 +niclog0=niclog +trace-start0=0 +trace-end0=-1 +linklatency0=350 +netbw0=100 +profile-interval=-1 +zero-out-dram +shmemportname0=default +permissive-off +prog0=darknet-nvdla-bin +blkdev0=darknet-nvdla.img

But when I try to build another HW configuration ,
the vivado(2018.03.op) can not route successfully.

export DESIGN=FireSimNoNIC
export TARGET_CONFIG=FireSimRocketChipQuadCoreConfig_WithNVDLALarge
export PLATFORM_CONFIG=FireSimDDR3FRFCFSLLC4MBConfig75MHz
make replace-rtl

And when I try to build this design, vivado cannot route it successfully:
>>
Phase 9 Depositing Routes
Phase 9 Depositing Routes | Checksum: 1d4c635ab

Time (s): cpu = 21:24:12 ; elapsed = 09:48:13 . Memory (MB): peak = 25375.250 ; gain = 3578.977 ; free physical = 4768 ; free virtual = 31398
INFO: [Route 35-77] Router completed with failures. Please check the log file for Critical Warnings and run report_route_status for a summary of routing status.
ERROR: [Constraints 18-1000] Routing results verification failed due to partially-conflicted nets (Up to first 10 of violated nets): WRAPPER_INST/CL/firesim_top/top/sim/target/nvdla/u_nvdla/nvdla_top/u_partition_p/u_NV_NVDLA_sdp/u_core/u_c/c_int_5/pipe_p1/p1_skid_pipe_data[0] WRAPPER_INST/CL/firesim_top/top/sim/target/nvdla/u_nvdla/nvdla_top/u_partition_p/u_NV_NVDLA_sdp/u_core/u_c/c_int_5/pipe_p1/p1_skid_pipe_data[12] WRAPPER_INST/CL/firesim_top/top/sim/target/nvdla/u_nvdla/nvdla_top/u_partition_p/u_NV_NVDLA_sdp/u_core/u_c/c_int_5/pipe_p1/p1_skid_pipe_data[32] WRAPPER_INST/CL/firesim_top/top/sim/target/nvdla/u_nvdla/nvdla_top/u_partition_o/u_NV_NVDLA_cvif/u_read/u_eg/lat_fifo1/rq_rd_adr[0] WRAPPER_INST/CL/firesim_top/top/sim/target/nvdla/u_nvdla/nvdla_top/u_partition_o/u_NV_NVDLA_cvif/u_read/u_eg/lat_fifo1/rq_rd_adr[1] WRAPPER_INST/CL/firesim_top/top/sim/target/nvdla/u_nvdla/nvdla_top/u_partition_p/u_NV_NVDLA_sdp/u_rdma/u_erdma/u_eg/u_alu/pipe_p1/beat_cnt_reg[0]rep_2 WRAPPER_INST/CL/firesim_top/top/sim/target/nvdla/u_nvdla/nvdla_top/u_partition_p/u_NV_NVDLA_sdp/u_rdma/u_erdma/u_eg/u_alu/pipe_p1/pipe_skid_out_pd0 WRAPPER_INST/CL/firesim_top/top/sim/target/nvdla/u_nvdla/nvdla_top/u_partition_p/u_NV_NVDLA_sdp/u_core/u_c/c_int_2/pipe_p2/p2_pipe_data_reg_n_0[12] WRAPPER_INST/CL/firesim_top/top/sim/target/nvdla/u_nvdla/nvdla_top/u_partition_p/u_NV_NVDLA_sdp/u_core/u_c/c_int_2/pipe_p2/p2_pipe_data_reg_n_0_[35] WRAPPER_INST/CL/firesim_top/top/sim/target/nvdla/u_nvdla/nvdla_top/u_partition_p/u_NV_NVDLA_sdp/u_core/u_c/c_int_2/pipe_p2/p3_skid_data[15]_i_15__13_n_0

...

Routing Is Done.
INFO: [Common 17-83] Releasing license: Implementation
183 Infos, 12 Warnings, 7 Critical Warnings and 1 Errors encountered.
route_design completed successfully
route_design: Time (s): cpu = 21:36:47 ; elapsed = 09:57:01 . Memory (MB): peak = 25399.258 ; gain = 3602.984 ; free physical = 7102 ; free virtual = 33733
ERROR: route_design failed. Writing route_design checkpoint for debug
/home/centos/src/project_data/firesim/platforms/f1/aws-fpga/hdk/cl/developer_designs/cl_FireSimNoNIC-FireSimRocketChipQuadCoreConfig_WithNVDLALarge-FireSimDDR3FRFCFSLLC4MBConfig75MHz/build/checkpoints/19_11_02-152028.post_route_design_error.dcp
Netlist sorting complete. Time (s): cpu = 00:00:00.25 ; elapsed = 00:00:00.25 . Memory (MB): peak = 25399.258 ; gain = 0.000 ; free physical = 7052 ; free virtual = 33687
Writing placer database...
INFO: [Timing 38-480] Writing timing data to binary archive.
Netlist sorting complete. Time (s): cpu = 00:00:01 ; elapsed = 00:00:00.40 . Memory (MB): peak = 25399.258 ; gain = 0.000 ; free physical = 5914 ; free virtual = 33197
Writing XDEF routing.
Writing XDEF routing logical nets.
Writing XDEF routing special nets.
Write XDEF Complete: Time (s): cpu = 00:03:43 ; elapsed = 00:01:48 . Memory (MB): peak = 25399.258 ; gain = 0.000 ; free physical = 3905 ; free virtual = 33276
INFO: [Common 17-1381] The checkpoint '/home/centos/src/project_data/firesim/platforms/f1/aws-fpga/hdk/cl/developer_designs/cl_FireSimNoNIC-FireSimRocketChipQuadCoreConfig_WithNVDLALarge-FireSimDDR3FRFCFSLLC4MBConfig75MHz/build/checkpoints/19_11_02-152028.post_route_design_error.dcp' has been generated.
write_checkpoint: Time (s): cpu = 00:10:58 ; elapsed = 00:11:30 . Memory (MB): peak = 25399.258 ; gain = 0.000 ; free physical = 6292 ; free virtual = 33654

ERROR: route_design failed.
while executing
"error $errMsg"
invoked from within
"if {[catch $impl_step $errMsg]} {
if {[string match $phase "route_design"]} {
puts "\tERROR: $phase failed. Writing $phase checkpoint f..."
(procedure "impl_step" line 92)
invoked from within
"impl_step route_design $TOP $route_options $route_directive $route_preHookTcl $route_postHookTcl"
invoked from within
"if {$route} {
puts "\nAWS FPGA: ([clock format [clock seconds] -format %T]) - Routing design";
impl_step route_design $TOP $route_options ..."
invoked from within
"if {$implement} {

#######################
\ # Link Design
########################
if {$link} {
\ ####Create in-memory prjoect and set..."
(file "create_dcp_from_cl.tcl" line 225)
INFO: [Common 17-206] Exiting Vivado at Sun Nov 3 07:31:39 2019...

I find you published a AFI as ：
{
"UpdateTime": "2019-03-31T12:38:35.000Z",
"Name": "firesim-quadcore-no-nic-nvdla-ddr3-llc4mb",
"Tags": [],
"PciId": {
"SubsystemVendorId": "0xfedd",
"VendorId": "0x1d0f",
"DeviceId": "0xf000",
"SubsystemId": "0x1d51"
},
"FpgaImageGlobalId": "agfi-0ceed51cae44c4e59",
"Public": true,
"State": {
"Code": "available"
},
"ShellVersion": "0x04261818",
"OwnerId": "785173868334",
"FpgaImageId": "afi-0f3d175134d69618e",
"CreateTime": "2019-03-31T11:36:21.000Z",
"Description": "firesim-buildtriplet:FireSimNoNIC-FireSimRocketChipQuadCoreConfig_WithNVDLALarge-FireSimDDR3FRFCFSLLC4MBConfig75MHz,firesim-deploytriplet:FireSimNoNIC-FireSimRocketChipQuadCoreConfig_WithNVDLALarge-FireSimDDR3FRFCFSLLC4MBConfig75MHz,firesim-commit:2e8b749c031da794b6c768df187119fbf3e10170-dirty"
},
It work.
How did you build it successfully using vivado?

### BTW:
1、 I can also build a HW config and can simulate it( to login ):
FireSimNoNIC-FireSimRocketChipQuadCoreConfig_WithNVDLASmall-FireSimDDR3FRFCFSLLC4MBConfig75MHz

but it hang up when run ./solo.sh .

And 2、 when I tried to build a Rocket Siglecore with NVDLALarge and NIC as follows:
export DESIGN=FireSim
export TARGET_CONFIG=FireSimRocketChipSingleCoreConfig_WithNVDLALarge
export PLATFORM_CONFIG=FireSimDDR3FRFCFSLLC4MBConfig75MHz

when I try to simulate it:
sudo ./FireSim-f1 +permissive $(sed ':a;N;$!ba;s/\n/ /g' runtime.conf) +macaddr0=00:12:6D:00:00:02 +slotid=0 +niclog0=niclog +trace-start0=0 +trace-end0=-1 +linklatency0=350 +netbw0=100 +profile-interval=-1 +zero-out-dram +shmemportname0=default +permissive-off +prog0=darknet-nvdla-bin +blkdev0=darknet-nvdla.img

It hung on the Commencing simulation.
sudo ./FireSim-f1 +permissive $(sed ':a;N;$!ba;s/\n/ /g' runtime.conf) +macaddr0=00:12:6D:00:00:02 +slotid=0 +niclog0=niclog +trace-start0=0 +trace-end0=-1 +linklatency0=6405 +netbw0=200 +profile-interval=-1 +zero-out-dram +shmemportname0=default +permissive-off +prog0=br-base-bin +blkdev0=br-base.img
AFI PCI Vendor ID: 0x1d0f, Device ID 0xf000
Using xdma write queue: /dev/xdma0_h2c_0
Using xdma read queue: /dev/xdma0_c2h_0
UART0 is here (stdin/stdout).
command line for program 0. argc=45:
+permissive +mm_relaxFunctionalModel=0 +mm_openPagePolicy=1 +mm_backendLatency=2 +mm_schedulerWindowSize=8 +mm_transactionQueueDepth=8 +mm_dramTimings_tAL=0 +mm_dramTimings_tCAS=14 +mm_dramTimings_tCMD=1 +mm_dramTimings_tCWD=10 +mm_dramTimings_tCCD=4 +mm_dramTimings_tFAW=25 +mm_dramTimings_tRAS=33 +mm_dramTimings_tREFI=7800 +mm_dramTimings_tRC=47 +mm_dramTimings_tRCD=14 +mm_dramTimings_tRFC=160 +mm_dramTimings_tRRD=8 +mm_dramTimings_tRP=14 +mm_dramTimings_tRTP=8 +mm_dramTimings_tRTRS=2 +mm_dramTimings_tWR=15 +mm_dramTimings_tWTR=8 +mm_rowAddr_offset=18 +mm_rowAddr_mask=65535 +mm_rankAddr_offset=16 +mm_rankAddr_mask=3 +mm_bankAddr_offset=13 +mm_bankAddr_mask=7 +mm_llc_wayBits=3 +mm_llc_setBits=9 +mm_llc_blockBits=6 +macaddr0=00:12:6D:00:00:02 +slotid=0 +niclog0=niclog +trace-start0=0 +trace-end0=-1 +linklatency0=6405 +netbw0=200 +profile-interval=-1 +zero-out-dram +shmemportname0=default +permissive-off br-base-bin +blkdev0=br-base.img
using link latency: 6405 cycles
using netbw: 200
using netburst: 8
Using non-slot-id associated shmemportname:
opening/creating shmem region
/port_ntsdefault_0
Using non-slot-id associated shmemportname:
opening/creating shmem region
/port_stndefault_0
Using non-slot-id associated shmemportname:
opening/creating shmem region
/port_ntsdefault_1
Using non-slot-id associated shmemportname:
opening/creating shmem region
/port_stndefault_1
random min: 0x0, random max: 0xffffffffffffffff
On init, 915 token slots available on input.
Zeroing out FPGA DRAM. This will take a few seconds...
Commencing simulation.

Any help will be appreciated.
Thanks

Frankta Tian

[farzadfch]

1、 I can also build a HW config and can simulate it( to login ):

The software for small configuration is not stable yet and is planned to be supported in the future releases.

And 2、 when I tried to build a Rocket Siglecore with NVDLALarge and NIC as follows:

When you build a target with NIC you need to specify a topology with network support in config_runtime.ini. Setting topology to example_1config should fix this issue.

As for your original question about building the hardware, I just kicked off a build run. I get back to you on this.

[farzadfch]

By default, the build is launched on AMI 1.5.0. Is there any particular reason that you changed it to AMI 1.6.0?

[franktaTian]

@farzadfch Thanks for reply. I am trying to merge NVDLA into the Firesim master branch.It seems just support 1.6.0 and up now?

[franktaTian]

@farzadfch Thanks again. I can boot the design with NIC(single core + nvdla small) now.Yes ,use the topology=example_1config