Improper message length issues with Rigol DS1102E #472
Comments
|
With the main branch, I get all timeout exceptions for every message, including pyvisa-info |
|
Problems Identified
|
|
first thanks for all information. i need a little time to go through all information in depth. but just to clarify some of your comments in the last message.
|
I'll continue investigating with #470 applied and see if I can figure out what's going on, including logging the packet data. Maybe something can be learned from the progression of |
|
|
One more log with With a 128-byte read size, it's strangely consistent at failing on the second Also nothing strange in the system log other than the reporting of the USB timeout. |
|
Same behavior in Linux. Packets just aren't there in Wireshark either, so maybe yet another bug in this 10 year old scope. Perhaps only a certain packet size or timing works. |
|
good to hear this fixes the problem you are facing
Here I think the bytes you get is actually correct. You are requesting 20480 bytes, which are also returned. see below:
Now you have read 10 bytes less then the scope would have returned when requested enough bytes from the device.
I will have a look into |
Got it this is a definite arbitrary block format read from the device. where see code snippet. this is different then py-visa does as they try to def readArbitraryBlock(self) -> bytes:
"""Read an Arbitrary block from the device with bytes as output from the instrument.
Returns:
List[float]: bytes
"""
DATA_BYTES_SIZE_SIZE = 2
DATA_BYTES_SIZE_OFFSET = 1
data_bytes_size_array = self.base_instrument.read_bytes(DATA_BYTES_SIZE_SIZE)
if data_bytes_size_array[0] != ord('#'):
raise Exception("Invalid arbitrary block Read data does not start with #")
data_bytes_size = int(data_bytes_size_array[DATA_BYTES_SIZE_OFFSET]) - ord('0')
data_bytes_array = self.base_instrument.read_bytes(data_bytes_size)
data_bytes = int(data_bytes_array.decode('utf-8'))
# we adjust the chuck size to required buffer!
if data_bytes < 20 * 1024: # 20KB is default chunck size of PyVISA
self.base_instrument.chunk_size = 20 * 1024
elif self.base_instrument.chunk_size < data_bytes:
self.base_instrument.chunk_size = data_bytes
data = self.base_instrument.read_bytes(data_bytes + 1)
data = data[:-1]
return data **** |
|
For small block that do not contain a termchar it means you do more query than you could could. I would accept a PR to change the chunk_size for subsequent reading in pyvisa. And I would be interested ion whether or not you see a difference in term of performance. |
|
@MatthieuDartiailh I do not follow your comment read more data than you could. As this problem seems to be related to arbitrary block read (IEEE 488.2 specification), which is specified as shown in this comment.
|
|
Ok, I actually have some interesting findings to report. Setting Setting So, I propose something like this with two transfers that should be the most efficient for reading data of this specific format. scope.write(':ACQ:MEMD LONG')
scope.write(':RUN')
time.sleep(1)
scope.write(query)
# First read enough to see the IEEE data header (plus some to fill wMaxPacketSize)
read_len = 52
data = scope.read_raw(read_len)
# Read expected length from IEEE header we just got
# This obviously assumes the header is a certain format so is not robust
data_len = int(data[2:10])
print(f'header data length = {data_len}') # 1048576
read_len = data_len - (len(data) - 10) # IEEE header takes up 10 bytes of data already read -> need to get 1048534 more bytes
# Read the rest of the data all in one go
data += scope.read_raw(read_len)
print(f'actual data length = {len(data)}') # 1048576 incorrect -- see below, but at least no more timeout issues and lost packetsNow I still have the problem that the scope does not send the correct length in the first USBTMC header and the total data length is 1048576 instead of 1048586. If I remove the truncation based on the USBTMC header length, this is the first time I have been able to transfer all of the long memory data successfully. To further explain, this is what's happening with the truncation over the two transfers:
Takeaway here is that splitting up the transfer into many smaller ones ( The former would be something to be implemented in |
please let me know if there is other documentation that specifies the return data of the Rigol. (as rigol downloads seem not to be working) |
|
Thanks for the clarification. With above information, this will bring me back to the following code snipped, which should enable to read the IEEE data block in 3 parts. Where pyvisa does following for a IEEE block which is well defined and so they first read data untill 20*1024 is reached, and if needed then continue, which does not seem to be supported by the rigol. But i am curious if you ask and read according the code snippet and the code pushed to #470 small explanation
i think we can increase the data that is being read from the device without using the |
|
I have made pyvisa/pyvisa#874 which is a small change that implements the reading behavior we've been discussing. Its effectiveness relies on #470. Now with those two PRs, the only remaining issue is the truncation, which I'll have to think about the best way to implement. I was thinking some kind of extra parameter that can be passed to disable it, but there are so many functions that would need to be modified to support that, it seems very unclean for such an edge case. Perhaps some kind of class member flag would be better. |
|
Glad to hear you found a solution, and i get that you needed to change binary block now. @aWZHY0yQH81uOYvH are you able to test the PR #470 again if it still works. see comment below @MatthieuDartiailh
|
|
#470 still works and does seem to be more efficient since it only calls into libUSB with two reads when combined with my PR. I still need to disable truncation for my scope but that's unrelated. The only downside I notice is since the reads are much larger, if the user has specified a low timeout (e.g. 1s instead of the default infinity), those reads can time out. Your PR seems to be robust to this since the inner while loop continues to read until the correct amount of data has been transferred. However, when that happens with my setup, data is lost as I was seeing before. Increasing the timeout so the first read completes without needing to fall into the inner loop solves this problem (which I suspect may again be an idiosyncrasy of my scope). Also, |
|
@aWZHY0yQH81uOYvH thanks for the feedback and glad it still works with the larger chunks. The truncation issue and not being able to run the inner loop, maybe is has something to do with the scope/device driver indeed. As it seems to send back more data than asked. Looking at usbtmc specification. That only talks in ‘wMaxPacketSize’ messages and should only send back data when ready to send it back. So still I would not expect those 10 bytes to be sent (and therefore to be lost). I will have a look at the IEEE 422 spec but don’t think they will specify different than USB or USBTMC layer for this part. Ps: Updated the typo 😉👍 |
I have a Rigol DS1102E connected over USB using the
pyvisa-pybackend. I am unable to receive waveform data correctly. This manifests in a few ways.In normal memory depth mode, the scope should return 16384 bytes of data with an IEEE header.
I think this used to give the exception show below. It seems to be very inconsistent.
I can get some data using
read_rawBut this sometimes doesn't work (throws the exception shown below), and it returns the incorrect amount of data (should be 16394 bytes including the header).
#470 seems to fix this issue.
query_binary_valuesworks as expected with that patch, andread_rawreturns the correct amount of data.However, if I request more data by switching into long memory mode (should return 1048576 samples),
I get the following exception, where it appears to be trying to parse a zero-length packet.
Sometimes it takes a very long time for this to finally show up. It used to happen much faster; not sure what is different now.
Attempting to use
read_rawresults in the same error.Using @Jimmyvandenbergh's #470, I get a timeout error using
query_binary_valuesand get a bunch of 20480-byte blocks usingread_raw. Attempting to usescope.read_bytes(1048586)also results in a timeout error.From the GitHub issues, it seems like a lot of Rigol/Siglent instruments are triggering this zero-length packet exception, so not sure if what I'm encountering is the same bug as the others. #470 fixes this but appears to be losing data somewhere given the timeout exceptions.
Output of
pyvisa-infoThe text was updated successfully, but these errors were encountered: