Proposal to Extend RISC-V DMI Transport in OpenOCD

[anicic-nikola]

Why?

Our company, Esperanto Technologies, is actively working with RISC-V hardware and identified a need to support alternative transport methods for Debug Module Interface (DMI) communication beyond JTAG. While JTAG serves many use cases, expanding the supported transport layers to PCIe and Socket would enable:
Flexibility for developers with varying hardware setups.
Networking capabilities through socket-based communication.
Easier integration for RISC-V devices using PCIe for debugging.
These additions would not only meet our internal needs but also benefit the broader OpenOCD and RISC-V communities by demonstrating how to extend transport layers and enabling similar contributions from others.

What?

We propose introducing PCIe and Socket as new transport layers for OpenOCD in the riscv-openocd repository.
Key changes include:

1. Transport Layer Selection - users will configure the desired transport (jtag, pcie, or socket) in OpenOCD .cfg files, e.g.,: transport select socket . For the socket layer, additional configuration options like host and port will be added.
2. Dynamic Routing of DMI Operations - the dmi_read and dmi_write functions for RISC-V will dynamically route to the appropriate transport implementation (JTAG, PCIe, or socket) based on user configuration.
3. Extensibility - new transport implementations will reside in src/transport and be modularly registered, making it straightforward for others to add custom transport mechanisms in the future.
   The selected transport implementation will be injected into the RISCV_INFO structure, ensuring seamless functionality during debugging.

How?

Implementation: We plan on introducing modular transport layers, starting with PCIe and socket, under src/transport. Then to modify RISC-V-specific dmi_read and dmi_write to invoke the correct transport function dynamically.
Extending the OpenOCD configuration parser to allow transport selection (jtag, pcie, socket) will be done if needed, but we believe the transport selection function should follow OpenOCD transport infrastructure pattern. and additional parameters for the socket transport will be done.
Our goal is also to engage the OpenOCD community early, seeking feedback on design and implementation to align with the project's goals and to maintain high code quality.

We believe these changes will make OpenOCD more flexible and powerful for RISC-V developers, both inside and outside of our organization.
Does this proposal align with the community's goals?
Are there additional considerations we should address before proceeding?
Looking forward to your thoughts and suggestions!

Best regards,
Nikola Anicic and Shankar Jayaratnam @NinjaDev108
Esperanto Technologies

[TommyMurphyTM1234]

Does this proposal align with the community's goals?
Are there additional considerations we should address before proceeding?

It might be a good idea to also run your proposal by the upstream OpenOCD community in case they have any views on it or anybody there might be undertaking similar work?

• https://openocd.org/pages/mailing-lists.html

[en-sc]

Thanks for this proposal! I believe it will be a great improvement.

First of all, I'd like to support @TommyMurphyTM1234's suggestion to post this in the mainline's mailing list. I think you will be able to get a lot of valuable feedback there.

1. Transport Layer Selection - users will configure the desired transport (jtag, pcie, or socket) in OpenOCD .cfg files, e.g.,: transport select socket . For the socket layer, additional configuration options like host and port will be added

I'd like to disagree. socket in this case is not the same level entity as jtag. E.g. there is remote bitbang -- JTAG over TCP/IP sockets.

Then to modify RISC-V-specific dmi_read and dmi_write to invoke the correct transport function dynamically.

I'm a bit concerned with this part.

OpenOCD benefits a whole lot from bundling a bunch of RISC-V DMI transactions into a single USB packet. This is possible thanks to sticky error state of RISC-V JTAG DTM.
Without this the performance will plummet, since USB (and especially TCP/IP) has comparatively high latency.
Utilization of this feature is facilitated in software by using JTAG queue.
The queue is filled up with JTAG operations and then all these operations are executed at once.
Also, it's possible to minimize the traffic over USB or TCP/IP by specifying that the software does not care about the result of a particular operation and therefore there is no need to send it back.

What I'm trying to say is: efficient communication over USB or TCP/IP will require a more complex interface then the pair of dmi_read/dmi_write. At least there is the need to queue these operations.

I'd suggest you to look into target/riscv/batch.h. This is the interface currently used to access DMI over JTAG. Though it is a bit JTAG-specific and there are parts that are done separately (e.g. the select_dmi function).

[TommyMurphyTM1234]

First of all, I'd like to support @TommyMurphyTM1234's suggestion to post this in the mainline's mailing list. I think you will be able to get a lot of valuable feedback there.

Just to clarify, my reasoning is that the suggested changes sound broad enough that that will most likely require changes beyond the RISC-V target specific parts of OpenOCD and, as such, will benefit from input from, and require acceptance by, the wider OpenOCD community and developers. If the changes were restricted to OpenOCD's RISC-V target specific support then this would be less critical, albeit still a good idea and good manners. 🙂

[NinjaDev108]

Actually we were thinking, this proposal might be restricted to RISC V implementation since we were gonna make DMI as the transport module API interface. Since other Arch dont directly support DMI, we might need to provide a different abstraction if we want to extend this beyond RISC V support. Below is a graphical overview of what we are thinking in terms of how the newly proposed Transport layer would interface with RISCV specific implementation. [image: Screenshot 2024-11-26 at 11.25.56 AM.png] Pls let us know your thoughts. Thank you WR ~S

…

On Tue, Nov 26, 2024 at 10:01 AM Tommy Murphy ***@***.***> wrote: First of all, I'd like to support @TommyMurphyTM1234 <https://github.com/TommyMurphyTM1234>'s suggestion to post this in the mainline's mailing list. I think you will be able to get a lot of valuable feedback there. Just to clarify, my reasoning is that the suggested changes sound broad enough that that will most likely require changes beyond the RISC-V target specific parts of OpenOCD and, as such, will benefit from input and require acceptance by the wider OpenOCD community and developers. If the changes were restricted to OpenOCD's RISC-V target specific support then this would be less critical, albeit still a good idea. — Reply to this email directly, view it on GitHub <#1175 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APHNMDUCIQL6IOZLDMH35RT2CSZN5AVCNFSM6AAAAABSQZV64WVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCMZYGY3DKNI> . You are receiving this because you were mentioned.Message ID: ***@***.*** .com>

[TommyMurphyTM1234]

Actually we were thinking, this proposal might be restricted to RISC V implementation since we were gonna make DMI as the transport module API interface.

I don't see how it can be given the original description which seems to imply changes to layers other than just the RISC-V target support and even mentions some of it being in src/transport and not under src/target/riscv?

Below is a graphical overview of what we are thinking in terms of how the newly proposed Transport layer would interface with RISCV specific implementation. [image: Screenshot 2024-11-26 at 11.25.56 AM.png]

The image isn't working for me.

[NinjaDev108]

I don't see how it can be given the original description which seems to imply changes to layers other than just the RISC-V target support and even mentions some of it being in src/transport and not under src/target/riscv?

Actually this is a very good point.. We need to rethink about how to limit the changes to within the RISCV scope i.e. src/target/riscv.

The image isn't working for me.

Here is the image - should show up now

[TommyMurphyTM1234]

Thanks, I can see the image now. But it doesn't really make sense to me. E.g. it suggests that GDB knows or will know about DMI commands but this isn't the case and I can't really see the GDB community accepting changes that introduce such low level details into GDB when, in this context, the GDB Remote Serial Protocol is the dividing line between GDB and the debug target. I think that this whole proposal needs a clear and comprehensive specification that can be reviewed systematically - here and probably upstream in the OpenOCD community. And maybe even in an appropriate RISC-V SIG or WG?

[aap-sc]

But it doesn't really make sense to me. E.g. it suggests that GDB knows or will know about DMI commands but this isn't the case and I can't really see the GDB community accepting changes that introduce such low level details into GDB when, in this context, the GDB Remote Serial Protocol is the dividing line between GDB and the debug target.

"+1". It does not make sense to me too. DMI is RISC-V specific abstraction and GDB has no idea about it. And to be fair, I don't think it should.

[NinjaDev108]

Great points, we will modify the diagram with better abstractions to avoid any misinterpretation. In summary the proposal limits the change-set to the RISCV implementation within OpenOCD target, specifically to adapt the DMI functions i.e. DMI Read and Write, to be able to switch to different physical transport layers i.e. JTAG, PCIE, Network Socket based on user configuration preference during launching of OpenOCD executable. Hence technically here is a redrawn picture to show the affect of this proposal.

[aap-sc]

This new diagram (#1175 (reply in thread)) makes much more sense, though it may still require some adjustments.

, specifically to adapt the DMI functions i.e. DMI Read and Write, to be able to switch to different physical transport layers i.e. JTAG, PCIE, Network Socket based on user configuration preference during launching of OpenOCD executable.

If it comes to RISC-V implementation - we'll have the concern raised by @en-sc (#1175 (comment)):

efficient communication over USB or TCP/IP will require a more complex interface then the pair of dmi_read/dmi_write. At least there is the need to queue these operations.

Regarding this point:

In summary the proposal limits the change-set to the RISCV implementation

There are at least two issues known to me :

1. The issue here is that AFAIK most targets in openocd do not allow to easily switch between transport layers
2. These transport layers themselves often require a dedicated configuration interface. A good example would be SWD vs JTAG.

SWD had DAP (debug access point), JTAG has TAP (test access point).

If you introduce, say PCIe interface - it does not make sense to configure jtag scan-chain that consists of TAPs (and each TAP requires certain transport-specific characteristics like IRLEN to be set).

To me it looks like the presence of these new transports may require introduction of "access points" types - and this is the change that should be definitely discussed with upstream OpenOCD folks.

[anicic-nikola]

Thanks for your answer Anatoly.

Ideally, we want to limit all of the changes in this proposal to remain within the RISCV port.
Hence, we are trying to brainstorm options on how to address your concerns.

Here are a few proposals:

1. Sketch depiction of access point implementation.pdf

2. High level block diagram
   

• A new AccessPoint layer is created in src/target/riscv/ specifically for DMI transport. This layer would define a common interface for DMI operations, to select the right AccessPoint for transport.
  Then the concrete transport modules for JTAG, PCIe, Socket will be implemented, all conforming to this interface mentioned. They would exist in src/target/riscv/transport/ folder for example. The core RISC-V debugging logic would interact with DMI transport abstraction layer avoiding any knowledge about the underlying physical transport.

• OpenOCD configuration will be limited to RISCV transport selection, we will extend it to allow users to select desired DMI transport layer (like riscv_transport jtag, riscv_transport pcie, riscv_transport socket). For socket there could be host and port given next to it. PCIe might have device identifiers, memory addresses, etc.
  Of course, switching from JTAG, for example, to PCIe, will not leave JTAG-specific settings active.

• For PCIe and socket a new optional queuing mechanism can be implemented depending on the performance requirements.

• During RISC-V target init we use the used-configured transport selection to instantiate the right transport module.
  Implementation detail would be to inject a pointer to the selected transport module into the RISCV_INFO generated struct making it accessible to dmi_read and dmi_write functions. These functions can be modified to delegate the actual communication to the selected transport module.

Example of config file:

adapter driver dummy 

set _CHIPNAME riscv
target create $_CHIPNAME.cpu riscv -endian big

riscv_transport socket
init
halt

What do you think about this proposal?

[JanMatCodasip]

My recommendation would be to approach this somewhat differently:

1. Create a RISC-V specific transport that would work on the RISC-V DMI level: batches of DMI reads and writes

transport select riscv_dtm
# dtm = RISC-V Debug Transport Module 
# (or a similar suitable name)

2. Create individual adapters for different debug transports (like TCP, PCIe, USB, Websockets, ..., ...) and let the drivers handle the specifics of the given transport channel.

adapter driver riscv_websockets_dtm
riscv_websockets_dtm set_host <...>
riscv_websockets_dtm set_port <...>

# or

adapter driver riscv_pcie_dtm
riscv_pcie_dtm set_something <...>

# or

# ...

3. For the high-level transports, do not use the JTAG transport (transport select jtag) or the dummy adapter (adapter driver dummy) at all.

[JanMatCodasip]

Hi @anicic-nikola and all,

General comments

Overall, I like this proposal of having a higher-level RISC-V debug transport in OpenOCD very much. In fact, we have developed something similar in Codasip internally. It would be beneficial for the whole community to have a unified solution.

Also thank you for describing the proposal in a very clear and understandable manner.

Where to make the change - riscv-openocd vs. vanilla openocd.org

Since there would be target-independent changes (in the transports), I am slightly more inclined to suggest the vanilla openocd.org upstream. I don't feel strongly, though. I will be happy to provide code reviews in either place.

Comments to the concrete proposal

I have posted my recommendation in a separate comment.

Thanks and regards
Jan

[anicic-nikola]

Hi, Jan.

The adapter driver command implies JTAG/SWD, which is not the case, when we use the socket, for example.
We plan on creating a new transport driver to bypass JTAG/SWD entirely, so we would not specify a traditional JTAG/SWD adapter driver in the OpenOCD configuration file.
In oour proposed approach, we are not extending or using OpenOCD's traditional adapter driver framework. Instead, we are creating a new abstraction layer specifically for RISC-V Debug Transport Modules (DTMs), which is a much cleaner and more architecturally aligned solution given the RISC-V debug specification.

Therefore, we do not need to specify an adapter driver <adapter_name> command in the OpenOCD configuration. Our configuration would focus on selecting and configuring the RISC-V DTM transport.

OpenOCD Core handles high-level debugging logic, GDB communication, etc.
Adapter Driver provides a low-level interface to physical JTAG/SWD adapters (e.g., FTDI, J-Link). It also translates OpenOCD commands into JTAG/SWD bit sequences. Like its name suggests - it adapts something to something.

In our proposal OpenOCD Core will be the same as before.
But what would change is:
RISC-V Target:

• DTM Abstraction Layer => A new layer within the RISC-V target code that provides a common interface for interacting with different DTMs.
• DTM Transport Drivers => Implementations for specific DTM transport mechanisms (JTAG, PCIe, Socket, potentially USB in the future). These drivers would reside in src/target/riscv/transport/ as already suggested, specific to RISC-V targets.

There would be No Traditional Adapter Driver: The DTM transport drivers handle the low-level communication directly, bypassing the need for a JTAG/SWD adapter driver.

Then the configuration would look like:

      
# Select the RISC-V DTM transport mechanism
transport select riscv_dtm

# Configure the specific DTM type
riscv_dtm pcie  ; # Or: riscv_dtm socket, riscv_dtm jtag

# Optional: DTM-specific parameters (e.g., PCIe address, socket port)
riscv_dtm_pcie_address 0x...
riscv_dtm_socket_port 5555

# Other RISC-V target settings
target create riscv hart0 -endian little -dmi-base 0x1000
riscv hart0 dmi_scan
# ... the rest of the configuration ...

transport select riscv_dtm tells OpenOCD to use the RISC-V DTM abstraction layer for communication.
riscv_dtm <type> selects the specific DTM transport driver (PCIe, Socket, JTAG).
riscv_dtm_<type>_<parameter> is an optional command to configure parameters specific to the chosen DTM transport (e.g., PCIe address, socket port).

Advantages of this approach

This approach directly reflects the structure and terminology of the RISC-V debug specification.
It separates DTM transport concerns from the rest of the RISC-V target code.
It makes it easy to add support for new DTM transport mechanisms in the future.
No JTAG/SWD Dependency since this allows for debugging over non-JTAG/SWD interfaces.

---

We planed this approach to be well-structured and to avoid the need for traditional OpenOCD adapter drivers. In our eyes it provides a clean and extensible way to support different RISC-V DTM transport mechanisms within OpenOCD.

[JanMatCodasip]

Hi @anicic-nikola

Thank you for describing your intentions in more detail.

anicic-nikola: The adapter driver command implies JTAG/SWD [...]

In my opinion, an adapter driver does not imply JTAG or SWD.

It looks perfectly OK to me to implement adapters that would transmit RISC-V DMI transactions (DMI reads and writes) over a socket connection, PCIe or other channels. There is already an adapter called vdebug (vdebug.c) which works on a similar principle.

anicic-nikola: There would be No Traditional Adapter Driver [...]

anicic-nikola: [...] Instead, we are creating a new abstraction layer specifically for RISC-V Debug Transport Modules (DTMs), which is a much cleaner and more architecturally aligned solution given the RISC-V debug specification. [...]

I am sorry, but this appears to be exactly the part of the proposal which I am not very comfortable with.

The architecture of OpenOCD does not map 1:1 to the layers specified by the RISC-V debug spec, as shown on the diagram ¹.

On one hand, your proposal respects the layering of the RISC-V debug spec very well. On the other hand, it would mean architectural (principal) changes in OpenOCD.

For that reason, if you intend to carry on with the proposal, please, seek an approval in the openocd upstream first (openocd.org). My concerns are:

• The fact that principal changes would be made
• The scale of the task
• The fact that RISC-V would behave differently than the other architectures supported by OpenOCD
• Acceptance in the upstream - introducing features in riscv-openocd which we may later have troubles reintegrating into upstream.

ancic-nikola: Advantages of this approach [numbering mine]

1. This approach directly reflects the structure and terminology of the RISC-V debug specification.
2. It separates DTM transport concerns from the rest of the RISC-V target code.
3. It makes it easy to add support for new DTM transport mechanisms in the future.
4. No JTAG/SWD Dependency since this allows for debugging over non-JTAG/SWD interfaces.

1. It would be nice to have 1:1 mapping between the layers of debug spec and the layers of OpenOCD, but this cleanliness would come at a cost, as I commented above. In my opinion, the cost is high.

2. DTM concerns can be separated nicely even if the non-JTAG DTMs are implemented as adapters: JTAG DTM implementation would reside in target/riscv. Other DTMs would reside in the adapters. Each DTM would be separated from the RISC-V target code by DMI read/write batch API.

3. If non-JTAG DTMs are implemented as adapters, adding new ones will be easy as well: It would mean adding a new adapter.

4. As explained above, adapters that are completely unrelated to JTAG/SWD can be created.

Footnotes

1. The today's version of OpenOCD is adapter-centric: it is mandatory to have an adapter selected. Furthermore, it is not possible to select a transport prior to selecting an adapter. ↩

[anicic-nikola]

Hi, Jan.

Perfectly understandable.

On one side we strive for architectural purity, clear separation of concerns, extensibility and JTAG/SWD independence. But on the other hand we do understand that this approach requires significant change to OpenOCD's core architecture, upstream acceptance risk, creates a divergence how RISC-V and other architectures are handled in OpenOCD and potentially requires more complex initial implementation and development effort due to the new abstraction layer.

Especially bad thing for us would be to not have our changes accepted in the upstream OpenOCD and broader community.

Maybe there can be some more pragmatic incremental approach here.
Like, for example, to go in several steps:

• Implement PCIe and socket-based DTM support as adapters within the existing OpenOCD framework. This serves as a proof of concept and demonstrates feasibility without requiring major architectural changes.
• Then we focus on creating clean and well-defined interfaces within the adapter code to isolate DMI read/write operations.

This approach directly addresses the concerns about architectural impact and upstream acceptance. It also allows us to leverage the existing infrastructure and learn more about its capabilities and limitations.

After successfully implementing PCIe and socket adapters, we would thoroughly evaluate the solution. We would identify pain points (if there are some), limitations, or areas where the adapter approach feels cumbersome or unnatural.
If the adapter approach proves to be fundamentally inadequate or leads to significant complexities, we can then revisit the original proposal for a dedicated DTM abstraction layer.
However, this time we'll have a concrete implementation to compare against and a deeper understanding of the challenges involved. We'll also have concrete data and experience to support our arguments when seeking upstream approval.

Maybe even, throughout the whole process we would need to consult with the upstream community, since in the end they will be the ones that would need to integrate this port to the upstream.

We still believe firmly in the clean architecture proposed, but we are aware of the problems, most of all the potential inability to merge the changes into the upstream OpenOCD.
On the other hand, RISC-V behaving differently than other architectures maybe speaks more about the openness of the RISC-V and this fact somehow comes natural.

RISC-V's debug specification is publicly available and designed for extensibility. This openness encourages innovation and allows for the development of new debug transport mechanisms beyond the traditional JTAG/SWD. Other architectures often have more closed or proprietary debug interfaces, limiting the flexibility to explore alternatives.
The proposal we made above to support PCIe and socket transports directly stems from this extensibility. RISC-V's debug specification explicitly allows for different Debug Transport Modules (DTMs), paving the way for these non-traditional approaches. Other architectures might not have a standardized way to incorporate such transports.

RISC-V's open nature fosters a wide variety of implementations and hardware designs. Different vendors and researchers can create custom RISC-V systems with diverse debugging needs. PCIe or socket-based debugging might be more suitable for certain high-performance or embedded systems where JTAG/SWD is less convenient or performant.
Other architectures, being more tightly controlled, often have more standardized hardware platforms. This leads to a more uniform approach to debugging, typically centered around JTAG/SWD.

The RISC-V ecosystem thrives on community involvement and contributions. This proposal itself is an example of this, as we're actively seeking ways to improve the debug capabilities of RISC-V based on real-world needs and the specification's potential.
Other architectures might have more centralized development processes, with changes and extensions primarily driven by the chip vendors themselves. This can make it harder to introduce novel approaches like what we did here.

Also, RISC-V's debug specification clearly defines the different layers and components involved in debugging, such as the DTM. This well-defined architecture encourages a more structured and modular approach to implementing debug tools.
Other architectures might have less clearly delineated debug interfaces, potentially making it more challenging to cleanly integrate new transport mechanisms. The sole fact that this proposal maps so well to the RISC-V specification highlights this clarity.

What I mean to say is that I do not see RISC-V behaving differently than other architectures as a concern, it is just the nature of it and it is impossible to fit all architectures in one mold. But the main concern about upstream acceptance due to changing of core OpenOCD's architecture remains a main concern for us.

[aap-sc]

From @anicic-nikola , few additional points (to add to an excellent summary provided by Jan):

Point 1:

The concern that comes from @JanMatCodasip :

I am sorry, but this appears to be exactly the part of the proposal which I am not very comfortable with.

My opinion matches this statement expressed by @JanMatCodasip exactly.

On the other hand, this:

Like, for example, to go in several steps:

- Implement PCIe and socket-based DTM support as adapters within the existing OpenOCD
framework. This serves as a proof of concept and demonstrates feasibility without requiring 
major architectural changes.
- Then we focus on creating clean and well-defined interfaces within the adapter code
to isolate DMI read/write operations.

Looks as a reasonable approach to me.

Point 2.

I'm not sure that we need OpenOCD to be aware of the implementation details related to PCIe, MMIO or whatever custom interfaces there are. I have few concerns regarding the matter:

I. Primary concern is that these interfaces will be too vendor specific. If I understand correctly, this is the concern that was implied by @TommyMurphyTM1234 in his comment here (#1175 (reply in thread)) . Looks like standardization of relevant interfaces may indeed require a dedicated working group in RISC-V International.
II. Interaction with these interfaces will require an elevated privileges granted by host operating system. It looks like this may introduce way too much details in OpenOCD codebase related to interaction with these custom layers. Instead, can we focus only on implementing an "Abstract Socket Interface" to DMI that will operate only on abstractions available in RISC-V DMI. The actual interaction with PCIe/MMIO/whatever devices could be performed by an external (to OpenOCD) driver which is vendor specific.

[JanMatCodasip]

Hi @anicic-nikola and @aap-sc,

Thank you for your comments.

anicic-nikola: Maybe there can be some more pragmatic incremental approach here. [...] Implement PCIe and socket-based DTM support as adapters within the existing OpenOCD framework. This serves as a proof of concept [...] Then we focus on creating clean and well-defined interfaces within the adapter code to isolate DMI read/write operations.

I am very much in favor of such incremental approach and will be happy to support it.

anicic-nikola: RISC-V's debug specification is publicly available and designed for extensibility [...] Other architectures often have more closed or proprietary debug interfaces [...]

I agree. I believe that due to the RISC-V ISA openness, the use of high-level debug transport than JTAG/SWD will be more common than in other ISAs/architectures. This can eventually have an influence on the overall OpenOCD architecture.

aap-sc: I'm not sure that we need OpenOCD to be aware of the implementation details related to PCIe, MMIO or whatever custom interfaces there are. I have few concerns regarding the matter: I. Primary concern is that these interfaces will be too vendor specific. [...]

I do not maintain a strict standpoint on this topic. In my opinion, whether to maintain the given transport inside OpenOCD or whether to have an external adapter, that would be probably good to decide on case by case basis - depending on how generic the transport is. If the given DTM proves generic enough, my preference would be to eventually submit the DTM specification for inclusion into the RISC-V debug spec.

Regards,
Jan

[NinjaDev108]

Hi @JanMatCodasip @TommyMurphyTM1234 @en-sc @aap-sc

Firstly thank you so much for reviewing this proposal. This is very helpful for us.

Meantime we made some comments

Pls take a look and let us know what you think

Regards,

Shankar/Nikola

[zqb-all]

Thanks for the proposal!

If we consider running OpenOCD on a RISC-V hart, debug other RISC-V harts on the same chip. Assume the hardware is implemented in the most direct way, allowing hart to access the DMI bus, then OpenOCD can directly read and write the specified physical memory address equivalent to reading and writing the registers of the DMs.
With this proposal, it's basically a Mem option just like PCIE/Socket, right?

[TommyMurphyTM1234]

Thanks for the proposal!

If we consider running OpenOCD on a RISC-V hart, debug other RISC-V harts on the same chip. Assume the hardware is implemented in the most direct way, allowing hart to access the DMI bus, then OpenOCD can directly read and write the specified physical memory address equivalent to reading and writing the registers of the DMs. With this proposal, it's basically a Mem option just like PCIE/Socket, right?

Why would one use OpenOCD at all in this context instead of just using GDB natively? E.g. same way that one runs GDB natively to debug executables/processes running on Linux/Windows/macOS/etc. without the need for any additional middleware (e.g. OpenOCD) sitting between GDB Remote Serial Protocol and the RISC-V target? Is it because no OS supporting such native GDB usage is being assumed or something?

[zqb-all]

Yes, when debugging a userspace program, we can use native gdb directly, but for other cases such as debugging the linux kernel itself or debugging baremental programs, we need to debug through the hardware DM.

Debugging via jtag access is of course the most reliable way, but it is not always easy to connect to jtag, such as when the target device does not have a reserved jtag interface, or the target device is remote and can only be connected via a network.

[TommyMurphyTM1234]

Yes, when debugging a userspace program, we can use native gdb directly, but for other cases such as debugging the linux kernel itself or debugging baremental programs, we need to debug through the hardware DM.

In the latter case why wouldn't something like kdb/kgdb make more sense?

• https://www.kernel.org/doc/html/v5.1/dev-tools/kgdb.html

[zqb-all]

In the latter case why wouldn't something like kdb/kgdb make more sense?

I was just thinking about having an alternative option when jtag is not available. Where native gdb/kdb/kgdb is available, of course, it is more convenient to use them directly

[aap-sc]

@TommyMurphyTM1234 it all depends on the usage scenario.

kgdb and simullar essentially use self-hosted debug facilities. That is the hart does not actually stops and it all happens with the help of dedicated exception handler. The suggested approach just allows to perform debugging using a standardized interface to external debug - that is no support from the SW is required.

And one more thing. Using of GDB is often discouraged because GDB can perform lot's of unexpected (to the user) operations like reading from unexpected locations because of corrupted/incorrect/unavailable debug information.

[anicic-nikola]

Hi, guys, @TommyMurphyTM1234 @en-sc @JanMatCodasip @aap-sc @zqb-all

We appreciate a lot your proposals.
There is an initial prototype we come up with and we would like to hear your opinion about it.

Here is the structure of the files added for this prototype:

src/
│
├── target/
│          └──── riscv/
|               └── transport
|                      └── socket_dmi.h
|                      └── socket_dmi.c
|                      └── pcie.h
|                      └── pcie.c
│
├── jtag/
│   └── drivers
│      └── riscv_dtm
│          └── dtm.c
│          └── dtm.h

and of course Makefiles updated.

The interface of the dtm.h looks like this:

// Defines the abstract interface for the Debug Transport Module (DTM)

#ifndef RISCV_TRANSPORT_DTM_H
#define RISCV_TRANSPORT_DTM_H

#include <stdint.h>
#include <stdbool.h>

typedef struct dtm_driver {
    const char *name; // e.g., "jtag", "pcie", "socket"
    int (*init)(struct dtm_driver *driver);
    int (*deinit)(struct dtm_driver *driver);
    int (*read_dmi)(struct dtm_driver *driver, uint32_t *data, uint32_t address);
    int (*write_dmi)(struct dtm_driver *driver, uint32_t address, uint32_t data);
    void *priv;  // Private data for the driver
} dtm_driver_t;

int register_dtm_driver(dtm_driver_t *driver);

dtm_driver_t *get_dtm_driver(const char *name);

int select_dtm_driver(const char *name);

dtm_driver_t *get_active_dtm_driver(void);

#endif /* RISCV_TRANSPORT_DTM_H */

And then in dtm.c new adapter driver is implemented and registered so it appears when running openocd -c "adapter driver list".
See the image:

dtm.c has a line (current list) of the transports it supports:

static const char * const riscv_dtm_transports[] = { "jtag", "socket", "pcie", NULL };

socket_dmi.c is going to implement these init, deinit, read, write commands and also to have

static dtm_driver_t socket_driver = {
    .name = "socket",
    .init = socket_dmi_init,
    .deinit = socket_dmi_deinit,
    .read_dmi = socket_dmi_read_dmi,
    .write_dmi = socket_dmi_write_dmi,
    .priv = NULL
};

to be compatible to this driver, so all these methods to create a socket, or send/receive dmi commands are registered to the driver itself.
socket_dmi and pcie (and others) transport layers will be registered via transport_register function to be shown in the list, output of the command `openocd -c "transport list".
See the image:

[anicic-nikola]

Ah, yes, of course.
Thanks, Jan.
That would come later on after the proof of concept - to just send some DMI commands over the socket, regardless how slow and then we planned on utilizing the batching, queuing, async sockets, host and port parsing from config, etc.

In that case dtm_driver interface would be extended.

[JanMatCodasip]

Few more ideas:

• I am not sure if it is beneficial to create a new custom struct typedef struct dtm_driver. You can also use the existing struct adapter_driver and add a new sub-structure with RISC-V DTM API calls - something similar to jtag_ops or swd_ops.

• Your example shows just a single adapter called riscv_dtm. My suggestion would be to have different adapters for different transports, like riscv_dtm_socket, riscv_dtm_pcie, ... (An exception would be if the transports are be very similar, in which case it would make sense to collapse the functionality into a single adapter from the user point of view.)

• It looks like the new transports (like socket in your example) are RISC-V specific. If so, I'd consider making riscv part of the transport name: riscv_socket.

• Is the API between the target code and the adapter(s) going to be the same for all the new transports (riscv_socket, riscv_pcie, ...)? If so, you may even consider having just a single transport called riscv_dtm, but I do not have a strict viewpoint on this.

[JanMatCodasip]

I am not sure whether the ideas are relevant to you at this early proof-of-concept stage. But in any case, the discussion about those will start eventually. (Hope to help by mentioning these early.)

[anicic-nikola]

The ideas are very relevant.
Makes sense about prepending riscv keyword for riscv specific transports.

So, you are essentially proposing many different adapters and potentially one transport for all of them (if the API between the target code and the adapters is the same) instead of what is proposed here: one adapter and multiple transports?

The bulk of the functionality is intended to be transport-agnostic (e.g., DMI command handling), so for us a single adapter with multiple transport options makes sense.
Also, if we (this community) plan to add more transports (e.g., USB or custom hardware links), a unified adapter is more future-proof.
Also, this is the gist of a proposal (idea):
We start with 1 adapter and consider separating adapters later if:

• Transport-specific complexities grow significantly.
• Different transports require unique optimizations or have vastly different workflows.

[JanMatCodasip]

So, you are essentially proposing many different adapters and potentially one transport for all of them (if the API between the target code and the adapters is the same)

Yes, I have been thinking along these lines and am just bringing it as an idea for your consideration. (I do not maintain a strict viewpoint on this.)

We start with 1 adapter and consider separating adapters later if [...]

Sounds good. Feel free to start with a single adapter and then we will see how it unfolds.

Also there is a possibility that we may have multiple separate adapters from the user point of view but share some of the code internally between adapters. But that's a discussion for a later point, not for now.

[anicic-nikola]

Hi, guys @TommyMurphyTM1234 @JanMatCodasip @en-sc @aap-sc @zqb-all
I would like if somebody could add me to the list (of contributors, or whatever is there for access rights to be able to push a branch) so that I can push a branch of a proposal discussed above and that we can work our way towards the proposed solution.
Can somebody add me?
Thanks :)
Cheers.

[aap-sc]

@anicic-nikola I believe you can just create a MR or a fork with a branch. Do you see any issues with this ?

[anicic-nikola]

I do see issues:

ERROR: Permission to riscv-collab/riscv-openocd.git denied to anicic-nikola.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Initially I thought it was because I cloned repo as https, but eventually I made a new one and cloned it via ssh and applied all of my changes and got this message, so I am not sure if I can just push my separate branch.
My branch is offspring of riscv branch.
Fetching works, for example...

[TommyMurphyTM1234]

I do see issues:

ERROR: Permission to riscv-collab/riscv-openocd.git denied to anicic-nikola.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Initially I thought it was because I cloned repo as https, but eventually I made a new one and cloned it via ssh and applied all of my changes and got this message, so I am not sure if I can just push my separate branch. My branch is offspring of riscv branch. Fetching works, for example...

You need to create your own fork of the RISC-V OpenOCD repo, make your changes there (add/commit/push) and only then create a PR targeting the RISC-V OpenOCD repo. You don't add/commit/push directly to the RISC-V OpenOCD repo.

[anicic-nikola]

Hi, guys @TommyMurphyTM1234 @zqb-all @aap-sc @en-sc @JanMatCodasip
The branch of proposed changes is in progress, but there are some things I might need help from you and some clarifications.
Namely, there is one particular commit I have that is hacky now, but it serves to go around JTAG.
OpenOCD has JTAG woven deeply in it and going around it seems like not an easy feat.
When I use my openocd cfg file where I have adapter driver riscv_dtm and transport select socket I would need to have target created already and that itself does not work as JTAG does.
Creation of target assumes JTAG and has this obligatory -chain-position and then certain number of arguments and in some places in code it relies on target's tap field, that is JTAG specific, like in riscv_init_target function in src/target/riscv/riscv.c file.

As you can see from this commit from my forked repo (that had some previous commits to it, too) I went around it in some hacky way to just see what needs to be changed and to later examine all of the necessary changes.
But now I have another problem: seems like .examine nor target_examine/target_examine_one is never called that sets the dtm_version in only one place in the code in riscv_examine function. This is not called now.
I believe that if we are not using JTAG or SWD, and are instead trying to implement halt/resume and other debug features through a different channel (e.g., a custom UART protocol, socket, some sort of backdoor access), here are the issues we'll encounter:

1. OpenOCD will not have a reliable way to halt the target. When we issue a monitor halt command, OpenOCD might attempt to use JTAG/SWD commands that will simply fail, or it might not do anything if our target_type doesn't implement the JTAG/SWD related functions.
2. OpenOCD's internal state (which tracks whether the target is halted, the current program counter, register values, etc.) will become out of sync with the actual state of the target because OpenOCD is not receiving the expected responses from the debug interface.
3. Since OpenOCD doesn't detect a proper halt event through JTAG/SWD, it will not call target_examine(), and therefore our .examine method will never be invoked.
4. Hardware breakpoints and single-stepping will likely not work because OpenOCD will not be able to use the standard JTAG/SWD commands to configure and manage them.

To halt a RISC-V core via DMI, we typically need to write to the dmcontrol register, setting the haltreq bit - is this right?
The monitor halt command does not inherently know how to perform this DMI write when using a socket connection? The standard behavior of monitor halt is more closely tied to how JTAG/SWD transports implement halting. I blieve. And is it true that the code path that monitor halt takes through openocd core logic doesn't directly have the DMI translation.

Instead of just hacking around the JTAG dependencies, I'm exploring how we can extend OpenOCD to better support non-JTAG transports like our socket-based DMI communication. One idea is to introduce somehow a more abstract interface for target operations (halt, resume, etc.) that isn't tied to JTAG specifics. This would allow us to implement these operations in our custom transport driver.

What are your thoughts on it?
Do you have some proposal or suggestion how to approach this differently?

In the end, the idea is to bypass the JTAG if it is not needed, like it is logically not needed for the socket DMI communication with some server that accepts DMI commands, it does not make sense to have JTAG interface here in our case.

[aap-sc]

 One idea is to introduce somehow a more abstract interface for target operations (halt, resume, etc.) that isn't tied to JTAG specifics.  This would allow us to implement these operations in our custom transport driver.

The existing interface for target operations is not tied to JTAG in any way. I would suggest to explore a way to factor-out jtag-specifics details. Implementing halt/resume in a custom driver does not make sense to me (if I understand the situation correctly)>

[anicic-nikola]

Thank you for your feedback, Anatoly.
I understand your point about the existing target operation interface not being intrinsically tied to JTAG and that the issue is likely with how JTAG specifics are currently handled. I agree that refactoring to factor out those JTAG details is a better approach than creating a new interface, if the current one is not tied to JTAG in any way.

I'll analyze the code paths for target operations (halt, resume, etc.) to identify the specific points where JTAG dependencies are introduced. The goal is to isolate and abstract this JTAG-specific logic, possibly by introducing intermediate layers that can translate between abstract target commands and transport-specific operations (including my socket-based DMI communication). This way, the core target logic can remain transport-agnostic. Do you agree with this?

I appreciate your comment on this.

[NinjaDev108]

Here is an example where RISCV target assumes there is only JTAG transport under the hood:

Source: src/target/riscv/riscv.c

The idea behind this multi-transport proposal is to abstract the underlying transport assumption and only be able to chose that during Runtime. Hence we would need move all the existing logic where assumption is made thats there is a JTAG transport, should be moved to the JTAG transport implementation.