Dynamically create row of geometrical shapes for each trial

[lpavan98]

Good evening,

in my experiment I am using as stimuli geometrical figures which differ in size (big, small, medium), color (red, green, blue) and shape (triangle, circle or square). There is thus a total of 27 possible figures.
In each trial, participants see 3 figures one next to each other (plus an unrelated image at the right) and need to point at one of the three geometrical shapes.
The stimuli should not appear the same way to each participant: all the times a participant sees blue stimuli, another one might see circles, and so on.

I was thus trying to use the canvas the canvas in order to draw the stimuli. Given an initial list of stimuli in the form:
stimulus = [
{ stim: ['211', '132', '312', image1], correct: 0 },
{ stim: ['131', '322', '231', image2], correct: 1 },
....
I wanted to use the numbers to represent features. For example, the first item in the first row of stimuli (called 211) could be of shape 2, color 1 and size 1, or color 2, shape 1 and size 1, and so on.
I was thus intending to use a function that sets randomly which among the first, second or third digit represents the shape, color and size. Another function would then get as input the character of three numbers and turn it into a shape, according to the mapping given by the first function. Eventually, the canvas keyboard response plugin would be used with the transformed stimuli.

The first problem that I have with this is that I cannot find a way to pass my 3 geometrical figures (as maybe subpart of a canvas) to the canvas keyboard response plugin.
Then, adding the picture after the input is not as straightforward as I thought it would be.
Lastly, I find no way to adapt the division of the screen in two parts that I was using, which Shaobin-Jiang kindly helped with (#3200), when working with the canvas.

Thank you for taking the time to go through this

[Shaobin-Jiang]

This is not going to be an easy one. We are going to have to break it up part from part.

---

We start from defining some constants here, namely the size, color, and shape.

const SIZE = [30, 60, 90]; // use radius to represent size
const COLOR = ['red', 'green', 'blue'];
const SHAPE = [
    // triangle
    (context, center_x, center_y, radius, color) => {
        context.save();
        context.fillStyle = color;

        context.moveTo(center_x, center_y - radius);
        context.beginPath();
        context.lineTo(center_x + radius / 2 * Math.sqrt(3), center_y + radius / 2);
        context.lineTo(center_x - radius / 2 * Math.sqrt(3), center_y + radius / 2);
        context.lineTo(center_x, center_y - radius);
        context.closePath();

        context.fill();
        context.restore();
    },
    // circle
    (context, center_x, center_y, radius, color) => {
        context.save();
        context.fillStyle = color;

        context.beginPath();
        context.arc(center_x, center_y, radius, 0, Math.PI * 2);
        context.closePath();
        
        context.fill();
        context.restore();
    },
    // square
    (context, center_x, center_y, radius, color) => {
        context.save();
        context.fillStyle = color;
        
        let len = radius * Math.sqrt(2);
        context.fillRect(center_x - len / 2, center_y - len / 2, len, len);

        context.restore();
    }
];

const stimulus = [
    { stim: ['211', '132', '312', './temp.png'], correct: 0 },
    { stim: ['131', '322', '231', './temp.png'], correct: 1 },
];

The size and color is quite easy to understand. As for the shapes, we define them as functions that draw the corresponding shape on the canvas. The drawn shape can vary with the center, radius and color in the function arguments.

---

We should then define the trial_id as I have suggested before. However, note that this time using stimulus alone would not suffice to store the already displayed shapes, as you the code does not really correspond to the exact shape, so you will need another variable to store the exact shapes.

let trial_id = 0;
let record = [];

---

Then comes the difficult part. I will put the code here first:

let trial = {
    type: jsPsychCanvasKeyboardResponse,
    stimulus: (canvas) => {
        let stim = stimulus[trial_id];

        let context = canvas.getContext('2d');
        let index = jspsych.randomization.shuffle([0, 1, 2]);

        let record_entry = [];

        // Draw the left part, which is the main stimulus
        for (let i = 0; i < 3; i++) {
            let size = SIZE[stim.stim[i][index[0]] - 1];
            let color = COLOR[stim.stim[i][index[1]] - 1];
            let draw_func = SHAPE[stim.stim[i][index[2]] - 1];
            draw_func(context, i * 200 + 100, 300, size, color);

            record_entry.push([size, color, draw_func]);
        }
        record.push(record_entry);

        let image = new Image();
        image.src = stim.stim[3];
        // Image loading is asynchronous; you need to put it in an onload callback
        image.onload = () => {
            context.drawImage(image, 650, 250, 100, 100);
        };

        // Draw the right part
        let start = Math.max(0, record.length - 5); // Show only, say, the last five
        for (let i = start; i < record.length - 1; i++) {
            let entry = record[i];
            for (let j = 0; j < 3; j++) {
                let s = entry[j];
                let size = s[0] / 4;
                let color = s[1];
                let draw_func = s[2];
                draw_func(context, j * 50 + 850, (i - start) * 50 + 75, size, color);
            }
        }
    },
    canvas_size: [600, 1000],
};

There is a lot to be explained here. First, what you want to do here can be done via another approach; that is to say, other than the one I have suggested here. As you have already a canvas, you can simply draw the stimulus on the left part of it and draw displayed shapes on the right side, so that you do not really need to mess up the DOM tree.

let index = jspsych.randomization.shuffle([0, 1, 2]);

Here, we are deciding which digit represents what. For example, if index is [1, 0, 2], then the second digit represents size, the first represents color, and the third represents shape. With this ready, we can draw the shapes.

let record_entry = [];

for (let i = 0; i < 3; i++) {
    let size = SIZE[stim.stim[i][index[0]] - 1];
    let color = COLOR[stim.stim[i][index[1]] - 1];
    let draw_func = SHAPE[stim.stim[i][index[2]] - 1];
    draw_func(context, i * 200 + 100, 300, size, color);

    record_entry.push([size, color, draw_func]);
}
record.push(record_entry);

You can change the size and position as you wish, of course.

Next, we need to draw the image. Image loading needs time, which is why it is async. Therefore, we need to wrap it up in an onload event:

let image = new Image();
image.src = stim.stim[3];
// Image loading is asynchronous; you need to put it in an onload callback
image.onload = () => {
    context.drawImage(image, 650, 250, 100, 100);
};

Then, you can draw the already displayed shapes on the right. As you can see above, we have stored the size, color and draw function, you can easily replicate whatever shapes the participant has already seen.

---

The last step would be to wrap the thing up in a child timeline:

jspsych.run([
    {
        timeline: [trial],
        loop_function: () => {
            trial_id++;
            return trial_id < stimulus.length;
        }
    }
]);

---

The complete code:

let jspsych = initJsPsych();

const SIZE = [30, 60, 90]; // use radius to represent size
const COLOR = ['red', 'green', 'blue'];
const SHAPE = [
    // triangle
    (context, center_x, center_y, radius, color) => {
        context.save();
        context.fillStyle = color;

        context.moveTo(center_x, center_y - radius);
        context.beginPath();
        context.lineTo(center_x + radius / 2 * Math.sqrt(3), center_y + radius / 2);
        context.lineTo(center_x - radius / 2 * Math.sqrt(3), center_y + radius / 2);
        context.lineTo(center_x, center_y - radius);
        context.closePath();

        context.fill();
        context.restore();
    },
    // circle
    (context, center_x, center_y, radius, color) => {
        context.save();
        context.fillStyle = color;

        context.beginPath();
        context.arc(center_x, center_y, radius, 0, Math.PI * 2);
        context.closePath();
        
        context.fill();
        context.restore();
    },
    // square
    (context, center_x, center_y, radius, color) => {
        context.save();
        context.fillStyle = color;
        
        let len = radius * Math.sqrt(2);
        context.fillRect(center_x - len / 2, center_y - len / 2, len, len);

        context.restore();
    }
];

const stimulus = [
    { stim: ['211', '132', '312', './temp.png'], correct: 0 },
    { stim: ['131', '322', '231', './temp.png'], correct: 1 },
    { stim: ['211', '132', '312', './temp.png'], correct: 0 },
    { stim: ['131', '322', '231', './temp.png'], correct: 1 },
    { stim: ['211', '132', '312', './temp.png'], correct: 0 },
    { stim: ['131', '322', '231', './temp.png'], correct: 1 },
    { stim: ['211', '132', '312', './temp.png'], correct: 0 },
    { stim: ['131', '322', '231', './temp.png'], correct: 1 },
    { stim: ['211', '132', '312', './temp.png'], correct: 0 },
    { stim: ['131', '322', '231', './temp.png'], correct: 1 },
    { stim: ['211', '132', '312', './temp.png'], correct: 0 },
    { stim: ['131', '322', '231', './temp.png'], correct: 1 },
    { stim: ['211', '132', '312', './temp.png'], correct: 0 },
    { stim: ['131', '322', '231', './temp.png'], correct: 1 },
    { stim: ['211', '132', '312', './temp.png'], correct: 0 },
    { stim: ['131', '322', '231', './temp.png'], correct: 1 },
    { stim: ['211', '132', '312', './temp.png'], correct: 0 },
    { stim: ['131', '322', '231', './temp.png'], correct: 1 },
    { stim: ['211', '132', '312', './temp.png'], correct: 0 },
    { stim: ['131', '322', '231', './temp.png'], correct: 1 },
    { stim: ['211', '132', '312', './temp.png'], correct: 0 },
    { stim: ['131', '322', '231', './temp.png'], correct: 1 },
]

let trial_id = 0;
let record = [];

let trial = {
    type: jsPsychCanvasKeyboardResponse,
    stimulus: (canvas) => {
        let stim = stimulus[trial_id];

        let context = canvas.getContext('2d');
        let index = jspsych.randomization.shuffle([0, 1, 2]);

        let record_entry = [];

        // Draw the left part, which is the main stimulus
        for (let i = 0; i < 3; i++) {
            let size = SIZE[stim.stim[i][index[0]] - 1];
            let color = COLOR[stim.stim[i][index[1]] - 1];
            let draw_func = SHAPE[stim.stim[i][index[2]] - 1];
            draw_func(context, i * 200 + 100, 300, size, color);

            record_entry.push([size, color, draw_func]);
        }
        record.push(record_entry);

        let image = new Image();
        image.src = stim.stim[3];
        // Image loading is asynchronous; you need to put it in an onload callback
        image.onload = () => {
            context.drawImage(image, 650, 250, 100, 100);
        };

        // Draw the right part
        let start = Math.max(0, record.length - 5); // Show only, say, the last five
        for (let i = start; i < record.length - 1; i++) {
            let entry = record[i];
            for (let j = 0; j < 3; j++) {
                let s = entry[j];
                let size = s[0] / 4;
                let color = s[1];
                let draw_func = s[2];
                draw_func(context, j * 50 + 850, (i - start) * 50 + 75, size, color);
            }
        }
    },
    canvas_size: [600, 1000],
}

jspsych.run([
    {
        timeline: [trial],
        loop_function: () => {
            trial_id++;
            return trial_id < stimulus.length;
        }
    }
]);

[lpavan98]

Thank you very much for your answer!
I am working using the code you provided as a basis, and I have two columns in the right part, in order to be able to show all my stimuli (and the associated pictures), which are now 22. So, stimulus is:

const stimulus = [
    { stim: ['213', '132', '312', './temp.png'], correct: 0 },
    { stim: ['131', '322', '231', './temp.png'], correct: 1 },
    { stim: ['213', '132', '312', './temp.png'], correct: 0 },
    { stim: ['131', '322', '231', './temp.png'], correct: 1 },
    { stim: ['213', '132', '312', './temp.png'], correct: 0 },
    { stim: ['131', '322', '231', './temp.png'], correct: 1 },
    { stim: ['213', '132', '312', './temp.png'], correct: 0 },
    { stim: ['131', '322', '231', './temp.png'], correct: 1 },
    { stim: ['213', '132', '312', './temp.png'], correct: 0 },
    { stim: ['131', '322', '231', './temp.png'], correct: 1 },
    { stim: ['213', '132', '312', './temp.png'], correct: 0 },
    { stim: ['131', '322', '231', './temp.png'], correct: 1 },
    { stim: ['213', '132', '312', './temp.png'], correct: 0 },
    { stim: ['131', '322', '231', './temp.png'], correct: 1 },
    { stim: ['213', '132', '312', './temp.png'], correct: 0 },
    { stim: ['131', '322', '231', './temp.png'], correct: 1 },
    { stim: ['213', '132', '312', './temp.png'], correct: 0 },
    { stim: ['131', '322', '231', './temp.png'], correct: 1 },
    { stim: ['213', '132', '312', './temp.png'], correct: 0 },
    { stim: ['131', '322', '231', './temp.png'], correct: 1 },
    { stim: ['213', '132', '312', './temp.png'], correct: 0 },
    { stim: ['131', '322', '231', './temp.png'], correct: 1 },
]

and the drawing of the right part is changed as

// Draw the right part
        for (let i = 0; i < record.length - 1; i++) {
            let entry = record[i];
            for (let j = 0; j < 3; j++) {
                let s = entry[j];
                let size = s[0] / 4;
                let color = s[1];
                let draw_func = s[2];
                if (i < 11) {
                    draw_func(context, j * 50 + 850, i * 50 + 75, size, color);
                } else {
                    draw_func(context, j * 50 + 1100, (i - 11) * 50 + 75, size, color);
                }
            }
            if (i < 11) {
                    context.drawImage(image, 1000, i * 50 +50, 50, 50);
                } else {
                    context.drawImage(image, 1300, (i - 11) * 50 +50, 50, 50);
                }
        }
    },
    canvas_size: [600, 1500],

This modification works, apart for the fact that in trial 13 and 20, the pictures in the right are not visible anymore. I cannot find an explanation for this. Would you happen to know why this might be the case? I have been trying with different png files, among which the orange circle from the jspsych tutorial:

Also, if I use two different pictures, they are displayed as wanted in the left part, but in the right part only one type of pictures is displayed. The desired outcome would instead be, for example, to have picture 1 displayed after trial 1, and picture 2 after trial 2, both in the trials themselves (left part) and in the record (right part).

If I should open a new thread instead of answering here, please let me know. I am using the same thread just for simplicity, since it regards modifications of the same code.

[Shaobin-Jiang]

I've been away with other things so apologies for the late reply. I am myself baffled by this bug; I think I know what might have caused this, but I would expect the bug to occur consistently, not in certain trials.

Back to your problem. I have pointed out that image loading is an async process. What does async mean, though? We frequently encounter tasks that might take a lot of time, such as loading an image or awaiting a response from the server. How should we deal with such situations? It would be a bad idea to simply wait for it to finish as the user would not appreciate it if you let them stare at a frozen page for too long. For a multi-threaded language, the solution would be to put the task to another thread, but javascript does not have multiple threads. So what it does is to put the time-consuming task at the end. Imagine a queue of people; now someone at the front want suddenly to use the bathroom, and it is naturally unwise to let everyone else wait for him to get back: you let others move forward and have that guy go to the end of the line when he returns from the bathroom. This is exactly how async works. Note though, that no matter how long that guy uses the bathroom, whether it be an hour or a millisecond, he still needs to drop to the end of the queue. So similarly, in javascript, the time-consuming task, even if it is not time-consuming at all, so long as it is an async process, gets done after everything else.

In your case, the loading of the image is the async process. Setting image.src starts the loading, and while we have put image.onload immediately after that, the callback does not get executed until everything else is done. Therefore, the way you are doing things, i.e., calling drawImage from outside of onload, would mean that you are drawing before the image is loaded. This is also why I am baffled, as I think that no image would be displayed at all. I am guessing there is other async processes from within jspsych, or perhaps it is just that my code is bad. But anyway, we know how to solve things. Instead of calling drawImage from outside of onload, we do it like this:

image.onload = () => {
    context.drawImage(image, 650, 250, 100, 100);
    for (let i = 0; i < record.length - 1; i++) {
        if (i < 11) {
            context.drawImage(image, 1000, i * 50 +50, 50, 50);
        } else {
            context.drawImage(image, 1300, (i - 11) * 50 +50, 50, 50);
        }
    }
};

---

As for your second question:

Also, if I use two different pictures, they are displayed as wanted in the left part, but in the right part only one type of pictures is displayed.

Of course that would be the case because you are always using the same variable image.

[lpavan98]

Thank you for the answer and the explanation, I am always happy to learn. These issues are now solved