HammerBlade on Gorgonzola

Here are some brief instructions for running HammerBlade simulations on Gorgonzola.

Standalone Kernel Development with Baseline

1. Connect to Gorgonzola via SSH.

2. Run these commands from the Gorgonzola page to set up the Xilinx and Synopsys tools:

   export VCS_HOME=/opt/synopsys/packages/vcs-mx/O-2018.09-SP2
   export PATH=$PATH:/opt/synopsys/packages/vcs-mx/O-2018.09-SP2/bin
   export [email protected]
   

   And then:

   source /opt/Xilinx/Vivado/2019.1/settings64.sh
   

   Or, you can add these lines to your .bash_profile or similar.

3. Go to /data/hb, where I'm trying to keep all the infrastructure centralized.

4. Go into bsg_bladerunner therein, which is the main chunk of HammerBlade infrastructure known as Bladerunner.

5. Go into baseline, which is the directory is the repository of examples known as Baseline. Or you can clone your own copy of this repository to avoid conflicting with other people—just give it a unique name.

6. To build an example, change directory to examples/conv1d (or some other directory under examples there).

7. Then type make default to build and run a simulation. You can also type make help to see what other make targets are available.

If you ever get an error like this:

cc1plus: error: unrecognized command line option ‘-std=c++11’

That's telling you that the GCC version we got from CentOS is too old. I installed a newer version but it's annoying to use; you have to do something like this:

scl enable devtoolset-8 -- make

…where make is whatever command you want to run. Or, to avoid typing this whole thing every time, you can use source /opt/rh/devtoolset-8/enable to enable the toolset for your shell session.

PyTorch

The next big step is adding HammerBlade kernels to PyTorch, the ludicrously popular machine learning library that currently primarily targets GPUs. Fortunately, the framework for targeting HammerBlade (and even several kernels) are already underway in the hb-pytorch repository.

This setup also comes with an "emulation" mode, which convenient way to development in a way that is faster and more convenient than "cosimulation," as we used when doing kernels in isolation. The difference is:

• Cosimulation actually simulates what the real hardware does. (In jargon, it uses "RTL simulation.") It is a very accurate reflection of what real silicon will do once we have it, and it has no real limitations in what programs it can run. However, it is extremely slow and requires proprietary (i.e., expensive) software that can only run on the server.
• Emulation is a new execution mode that does not try to mimic the hardware in detail—it just tried to run your code normally on any old computer, with all the HammerBlade-specific calls "faked" to do something functionally equivalent to the real hardware. It doesn't support all of the HammerBlade API (but it supports a lot of it already and we're working on expanding it). Unlike cosimulation, it cannot tell you anything about performance, i.e., how many cycles your code would have taken to run on real hardware. It is, however, far faster than cosimulation, and it only needs "normal" compilers, so you can do development right on your laptop if you want.

So the workflow we'll use for PyTorch is this: start writing code and debugging it using emulation. You can do this anywhere you like (on Gorgonzola or on your own machine). Then, when it's working, try running it in cosimulation mode on Gorgonzola to make sure it still works. Then you can open a pull request in the hb-pytorch repository.

To set up for emulation development (the first step), follow these instructions in the hb-pytorch README. You can either do this on your own laptop or on Gorgonzola. If you use Gorgonzola, create yourself a directory under /data/hb to do your work and clone the hb-pytorch repository in there (instead of using your home directory).

When building PyTorch on Gorgonzola (the python setup.py develop step), it might complain that your version of GCC is too old. If so, see the instructions above (at the bottom of the previous section) about enabling devtoolset-8.

To try out your changes in cosimulation (the second step), use Gorgonzola. The setup is at /data/hb/bsg_bladerunner/hb-pytorch on Gorgonzola.