How to slice a ggml tensor ?

[Jianhua-Cui]

For example, I have a torch code like this:

# x is a torch.Tensor with shape [1, 1025, 1024]
x = x[:, 1:, :]

I want do the same operation in ggml:

embeddings = ggml_norm(ctx0, embeddings, eps);

// hot to slice embeddings in it's second dim

Could you please guide me on how to perform this slicing operation in GGML? Any advice would be appreciated. Thank you!

[marty1885]

What you are looking for is ggml_view_*. For the specific example x = x[:, 1:, :] should equivalent to the following (note that GGML stores the shape (ne) and strides (nb) in reverse order.

x = ggml_view_3d(ctx. x, x->ne[0], x->ne[1] - 1, x->ne[3], x->nb[1], x->nb[2], x->nb[1]);

[Jianhua-Cui]

That's cool, thank you so much !!!

[Jianhua-Cui]

Additionally, I have another follow-up question: where can I learn concepts similar to tensor->ne and tensor->nb, and why can reshape achieve operations similar to slice? Do I have to learn it by reading the source code?

[Jianhua-Cui]

I see ggml_view_3d 's last arg is offset, so that's why "view" can be used as "slice", right ?

Thanks .

[marty1885]

where can I learn concepts similar to tensor->ne and tensor->nb, and why can reshape achieve operations similar to slice

Strides are how almost every tensor library works internally. It's a powerful and cheap way to express how the tensor should be read. The only problem is it's not super human friendly.

I recommend to read: https://ajcr.net/stride-guide-part-1/

I see ggml_view_3d 's last arg is offset, so that's why "view" can be used as "slice", right ?

Yes, along with the rest of the parameters, it forms a window into the tensor.

[Jianhua-Cui]

Your response was very helpful, thank you so much!