Visualization of test results

[saruvora]

Hi,

I would like to know how can I visualize the test results as shown in the /images/.gifs?

[JunweiLiang]

We may release that part of the code in the future. The visualization is pretty trivial:

1. We train 20 models with random initializations (You can use 1 model output as well. Then you need to set a large sigma in the gaussian filter to get a better visualization of the heatmap);
2. Save the path predictions and activity predictions;
3. Plot the observable trajectory line with cv2.arrowedLine;
4. Plot the path prediction using heatmap as follows:

heatmap_layer = np.zeros((video_h, video_w), dtype="float")
# set all prediction point values to 1.0
# use np.linspace to link the prediction points
# use scipy.ndimage.gaussian_filter to get heatmap
# use cv2.addWeighted to add the heatmap to original image

5. Plot the activity prediction labels and the corresponding poses. We manually select a pose template from the pose detection results for each activity class. For visualizing the poses, we use detectron code.

[JunweiLiang]

Added notes for the prepared data here. Could be useful for visualization.

[saruvora]

Hi, @JunweiLiang Thank you for your response and the information will look into them.

[bbidong]

@JunweiLiang , Hi, when using cv2.addWeighted to add the heatmap to original image, the channel of original image is 3, and dtype is uint8 , but the channel of heatmap is 1, and dtype is float, which will lead to error in cv2.addWeighted, so how can I solve it ?
Looking forward to your respond ！

[JunweiLiang]

@bbidong,
Simply do the following:

            f_new_layer = gaussian_filter(new_layer, sigma=7)
            f_new_layer = np.uint8(f_new_layer * 255)
            ret, mask = cv2.threshold(f_new_layer, 1, 255, cv2.THRESH_BINARY)
            heatmap_img = cv2.applyColorMap(f_new_layer, cv2.COLORMAP_WINTER)

            heatmap_img_masked = cv2.bitwise_and(heatmap_img, heatmap_img,
                                                 mask=mask)

            frame_data = cv2.addWeighted(frame_data, 1.0, heatmap_img_masked,
                                         0.9, 0)

frame_data is the original image. new_layer is the trajectory image after linspace.

[bbidong]

Wow, Thanks ! I realized according to your guidance.

[isaryo]

@JunweiLiang Hi, I have a question. I do visualization of the results now. Which variables do we get path predictions and activity predictions? I'm waiting for your answer.

[JunweiLiang]

The outputs of the Tester Class include those variables, namely pred_out, future_act

[isaryo]

No,good,,,I don't understand how to get predictions even though I try to it over and over..
I know you're frustrated because I'm true begginer.
I answered based on this to plot the path prediction using heatmap.
How to save path predictions and activity predictions?

[JunweiLiang]

You can refer to the full visualization code in the social-distancing-early-forecasting repo. Code to save prediction results are here.

[isaryo]

Did you save prediction results and visualize of the results with one code?
I would very much appreciate it if you would help me if I should encounter any problems.

[JunweiLiang]

Just read the code. They are all there (L1041 - L1297).

[isaryo]

I read this code.
Do you have used grid size?
Is this included in the path prediction result?

[JunweiLiang]

Please refer to the paper for model details.
For the visualization, I do not visualize the grid, only the final trajectories

[isaryo]

I want to see the visualization videos.
Something like all videos not like this