Replacing values stored in table from/to a certain position

[damiensellier]

Hi,
I have some trouble achieving what I want with my current project.

Ctrlr receives midi messages, they are filtered by type beforehand.
Some messages need to be displayed as text to simulate a LCD scribble screen.

The different text on the screen are not updated as a whole. Instead to optimise bandwith, only the text to be updated is sent with a byte-flag from the message telling the positon from where to update the text.

>>> ABCDEFGHIJKLM

from #4 1234

>>> ABCD1234IJKLM

In my script I set a table to store the different characters and only the new characters should be replaced in the table.
I tried with table.remove + table.insert but I can't get it work properly. In python there's a function table.replace but not in LUA and some Juce function are also missing.

Here is my script :

-- Decode Scribble Display

LcdScribbleDecode = function(midi)

local midiLength = midi:getSize()
local LcdUpdateStartPos = midi:getLuaData():getByte(6) -- the start position from where to update the text only

local t={}
local offset,t=7,{}
local TotalLength = midiLength-1  -- to remove the F7 ender
local ScribbleByteLength = TotalLength-offset -- length of the useful part

local ScribbleData = midi:getLuaData():getRange(offset,ScribbleByteLength)

ScribbleData:toLuaTable(t) -- insert data in table

local FullScribble = string.format("%s",ScribbleData:toString())
--console("Displays the FullScribble : "..(FullScribble))

for i,v in ipairs (t) do
console(String(i.." = "..v))
-- I need to replace v for each i in the table with new value from LcdStartPos #i until #ScribbleByteLength position
end

local LcdScribbleLine = table.concat(t)

panel:getLabel("debug"):setText(LcdScribbleLine)
-- I get numbers not text

end --f

I have another problem, I can't figure how to get the text stored as characters instead of numbers.
If I write the following code, I get an error. (attempt to call method 'toString' (a nil value)

local LcdScribbleLine1 = table.concat(t)
LcdScribbleLine1 = string.format("%s",LcdScribbleLine1:toString())
panel:getLabel("debug"):setText(LcdScribbleLine1)

I tried so many things, first I tried only with setting the incoming text to the label then change the text with string.sub but I can't find how to set the update start position, and the new text to replace with.

I probably turn something very easy to very difficult and combersome.

That would be really nice if you could help me with that ;)

thanks

Damien

[dobo365]

Why not using a memory block to store your texts?
Then you can just use copyFrom which would be close to what you know.
I'm using that all the time to copy/move/swap patches in sysex bank files.
Just giving you some quick answer here, you need to verify and adjust

mbText = MemoryBlock ({0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0X20}) -- Adjust according to your max length
iSize = MidiMessage:getData():getSize() - 8   -- 8 = header + F7
iPos = MidiMessage:getData():getByte(6)
mbNew = MemoryBlock(MidiMessage:getData():getRange(7, iSize)
mbText = mbLCD:copyFrom(mbNew, iPos, iSize)

[dnaldoog]

Hi Damien,

Here is how to convert a sysex message to a string:

decodeSysex = function()
    local m = MemoryBlock("F0 00 00 66 10 12 00 68 65 6c 6c 6f 20 77 6f 72 6c 64 F7") -- your sysex
    local offset, str = 6, ""
    for i = offset, m:getSize() - 2 do
        str = string.format("%s%s", str, string.char(m:getByte(i)))
    end
    console(str)
end

-- or alternatively

decodeSysex = function()
    local m = MemoryBlock(panel:getLabel("SYSEX"):getText())
    local offset = 7
	local sLen=m:getSize()-1-offset
    panel:getLabel("OUTPUT"):setText(m:getRange(offset,sLen):toString())
end

> hello world

Here is how to convert a string back to sysex:

encodeString = function()
    local str ="hello world"
    local t={}

    for i = 1, str:len() do
        local w = str:sub(i, i)
        table.insert(t, string.byte(w))
    end

    local m1 = MemoryBlock()
    m1:createFromTable(t)

    console(m1:toHexString(1))
end

> 68 65 6c 6c 6f 20 77 6f 72 6c 64

Scribble Damien_1_0_Hell-O-Kitty_2021-03-26_21-14.zip

So in each case you can see how to access each element, so then you would just need to work out a way to insert the data you need into whichever element in the table or in the memory block When looping.

[damiensellier]

memoryBlock absolutely! How could I pass on this one ???
Thank you so much to both of you. Shame on me, I forgot everything you tought me :)
I did not code for Ctrlr for a long time and I forgot the basics.

dnldoog thanks for the panel, It's super useful, it should be added to the tools.

I'll try it right away and keep you updated.

Thanks so much!

[damiensellier]

@dobo365 When you write :
mbText = mbLCD:copyFrom(mbNew, iPos, iSize)

Where does the "mbLCD" comes from?

Can you please explain me the different arguments in the formula?
mbText = memoryBlock
mbLCD ???
mbNew = part to update
iPos = position from where to update the memoryblock
iSize = the number of bytes of the new part

thanks

Damien

[dobo365]

Désolé Damien, faute de frappe ;-)
Sorry Damien, I wanted to change the name of the variables and I messed things up...

mbText = MemoryBlock ({0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0X20}) -- Adjust according to your max length
iSize = MidiMessage:getData():getSize() - 8   -- 8 = header + F7
iPos = MidiMessage:getData():getByte(6)
mbNew = MemoryBlock(MidiMessage:getData():getRange(7, iSize)
mbText = mbText:copyFrom(mbNew, iPos, iSize)

Look in Juce documentation for Memoryblock to see the different possibilities.
I only found one limitation in Ctrlr, it is to initialize a memoryblock of a defined length with a single value (for example 0 or 0x20) which is not working. So in my panels, I do that as
mbNameEmpty = MemoryBlock ({0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00})

To convert from sysex (hex) to string, no need to use a complex home made function ;-)
Just use
sName = mbText:getRange(start_pos, length):toString())

To convert from string to sysex (hex) you can use:

--
-- Returns HEX representation of a String
--
function str2hex(str)
    local hex = ''
    while #str > 0 do
        local hb = dec2hex(string.byte(str, 1, 1))
        if #hb < 2 then hb = '0' .. hb end
        hex = hex .. hb
        str = string.sub(str, 2)
    end
    return hex
end

To display a memory block in ASCII, it is just needed to use the built-in Lua function toString()

[damiensellier]

Hi,
I really struggle to have my script working. I spent hours to find a way to replace those characters without erasing the one that need to stay.

1. the function "copyFrom" always crashes the app.

2. I tried at least to store properly the message received in a memoryBlock, then creating a table to be able to replace the updated characters.
   I face an issue which is, even with "table.insert" that must keep the values from the table but shift the old ones to the right, The table is entirely updated. I don't know where it's coming from.

3. I don't know if my default memoryBlock must contain hex value 0xVV or regular VV.

4. for the copyFrom function, if I understand well, I can put values from the new message stored in a MB to the default memoryBlock "mbScribble" from a certain position defined by the ByteValue(6)

5. I understood the right to convert values to letters with using a loop, that's the good point :)

6. When setting up a table t{}, it seems that the user can specify a length with argument Content changed callback on uiLCDLabel is never called #3 n=length, I don't know how to write it properly, it would be nice to set it up to 112 bytes by default.

If you could check what I have in my following script that would save me, I spent the whole afternoon to check the Lua docs but it did not helped that much. I just want to replace a portion of text in a label after all, it shouldn't be that hard. But I'm stuck, I don't get it.

LcdScribbleDecode = function(midi)

-- set the default scribble text
local mbScribble = MemoryBlock("F0 00 00 66 10 12 00 68 65 6c 6c 6f 20 77 6f 72 6c 64 F7") -- your sysex
local offset, Scribble = 7, ""    -- offset in string Scribble
for i = offset, mbScribble:getSize() - 2 do
	Scribble = string.format("%s%s", Scribble, string.char(mbScribble:getByte(i)))
end

panel:getLabel("LcdLine1"):setText(Scribble)
console("Displays the mbScribble : "..(Scribble))

-- all good


-- store the portion of text to update
local mbScribbleUpdate = midi:getData()
local offset, ScribbleUpdate = 7, ""
for i = offset, mbScribbleUpdate:getSize() - 2 do
	ScribbleUpdate = string.format("%s%s", ScribbleUpdate, string.char(mbScribbleUpdate:getByte(i)))
end

panel:getLabel("debug"):setText(ScribbleUpdate)
console("Displays the ScribbleUpdate : "..(ScribbleUpdate))

-- all good


-- set some global variables
midiLength = midi:getData():getSize() - 8   -- 8 = header + F7
LcdUpdateStartPos = midi:getLuaData():getByte(6) -- Position to start update from
mbScribbleUpdate = midi:getLuaData():getRange(7, midiLength) --The Update message to replace with
LcdScribbleUpdate = string.format("%s",mbScribbleUpdate:toString()) -- stringified
console("Displays the LcdScribbleLine : "..(LcdScribbleUpdate)) -- console monitor

-- all good


t,ScribbleByteValue={},0    -- create table from the memoryBlock

-- loop in the memoryBlock and replace the old values by the new incoming ones
for i = offset, midiLength do
	ScribbleByteValue = midi:getLuaData():getByte(i)
--	table.remove(t,i+LcdUpdateStartPos)
--	table.insert(t,i+LcdUpdateStartPos,ScribbleByteValue)
end

-- not good

mbScribbleRaw = MemoryBlock()
mbScribbleRaw:createFromTable(t)
LcdScribbleLine2 = table.concat(t," ")
--console("Displays the LcdScribbleLine2 : "..(LcdScribbleLine2))
--panel:getLabel("LcdLine2"):setText(LcdScribbleLine2)

I commented the table.concat at the bottom because sometimes it crashes (nil) value

Thanks a lot in advance ;)

Damien

[dobo365]

Look at my answer above. Maybe some answers for you.
No need to use tables. You can do all the job with just memory blocks
Use copyFrom, append, getRange, getByte, setByte...

[dnaldoog]

Remind me to use m:toString() or even m:getRange(start,numBytes):toString next time rather than the function I posted! Thanks! Always forget that one.

Here is a potential strategy/solution:

MAX = 112
--------------------------------------------------------------
local init = {}
for i = 1, MAX do
    init[i] = 0x5F -- this is an underscore for visualisation, substitute 0x20 maybe for 'space'
end -- or use buffer:setByte(i,0x20)

buffer = MemoryBlock(MAX) -- initialise a 112 byte memory block at startup
buffer:createFromTable(init)
--------------------------------------------------------------
midiIncoming = function(--[[ CtrlrMidiMessage --]] midi) -- detect incoming midi
    LcdScribbleDecode(midi)
end
--------------------------------------------------------------
LcdScribbleDecode = function(midi)
    midiLength = midi:getData():getSize() - 8 -- 8 = header + F7
    LcdUpdateStartPos = midi:getLuaData():getByte(6) -- Position to start update from
    addStringToBuffer(LcdUpdateStartPos, midi:getLuaData():getRange(7, midiLength))
end
--------------------------------------------------------------
function addStringToBuffer(pos, binSeq)
    buffer:copyFrom(binSeq, binSeq:getSize(), pos) 
end
--------------------------------------------------------------
printBuffer = function()
    consolel(buffer:toString())
end

buffer 112 save state_1_2_save state_2021-03-28_17-20.zip

---

change log 3/28 substituted buffer:copyFrom(binSeq, binSeq:getSize(), pos) for buffer:insert(binSeq, binSeq:getSize(), pos)

[damiensellier]

Thanks to both of you @dnaldoog and @dobo365 .
I read the whole "memoryBlock" chapter in the Juce reference :

https://docs.juce.com/master/classMemoryBlock.html

Now it's way clearer for me. What is a memoryBlock, what is it for, and the functions related to it.
There was no need to create 2 tables from 2 memoryblocks, and swap the updated datas on the fly from A to B as I was thinking about. I am a pro to make easy things complicated :-)

So, I tried John's method which seems very logical to me. I adjusted the methods and strategy to my needs. And it seems to be working pretty well.

There's one thing, since the text needs to be changed : insert() adds up to the string and shifts to right the old one. So I used copyFrom() as Dominique said.

MemoryBlock::copyFrom( srcData,  destinationOffset, numBytes )

I really think I step up my Juce knowledge with this one.

Thank you so much for your precious help!

Damien

[dnaldoog]

Oh I see, so it's copyFrom not Insert. I guess I didn't check the size of MemoryBlock buffer after insert. Makes sense. I will update those panels later. Great way to learn is by making mistakes!

It took me a long time to wrap my head around MemoryBlock. I sometimes need to come back to it to 'reunderstand' it so to speak.

[Ted-juh]

Late at the party, I know, but "Insert" would work as well if you do "removeSection" first. Else some clipping of the MemoryBlock will occur. Not a problem in this case because it's a "set" size and you could do an extra check with "getSize" after you updated the block to see if the size is correct before doing some funky stuff with it.

To ScoobyDoogy, about setSize that's present in Juce bot not in Ctrlr:

But in case some clipping occurs, you could do:
LMemoryBlock (const void *dataToInitialiseFrom, size_t sizeInBytes);

This creates a new MemoryBlock where "dataToInitialiseFrom" is the data to be put in the MemoryBlock and "size_t_sizeInBytes" could stretch or shrink the MemoryBlock to the appropriate size. Still tricky but possible.

Or strip everything after what gets updated, copy it to a new MemoryBlock and append it, also seems to increase it in size.