[Enhancement]: simplify segment interface in segcore

[tedxu]

Is there an existing issue for this?

• I have searched the existing issues

What would you like to be added?

The segment interface, as well as its internal interface SegmentInternalInterface, is supposed to have only 2 memory access patterns, namely retrieve (data of 1 row) and sequential scan. Allowing random access (e.g., chunk_data() and chunk_view()) is prohibitive to segcore's future evolve. Take the transparent encryption feature as example, if random access interface is allowed, the scale of decryption operation is expanded to the whole chunk, which is not preferred.

Why is this needed?

No response

Anything else?

No response

[xiaofan-luan]

Is there an existing issue for this?

• I have searched the existing issues

What would you like to be added?

The segment interface, as well as its internal interface SegmentInternalInterface, is supposed to have only 2 memory access patterns, namely retrieve (data of 1 row) and sequential scan. Allowing random access (e.g., chunk_data() and chunk_view()) is prohibitive to segcore's future evolve. Take the transparent encryption feature as example, if random access interface is allowed, the scale of decryption operation is expanded to the whole chunk, which is not preferred.

Why is this needed?

No response

Anything else?

No response

But this seems to be avoidable?
because after search people need to specify output field and retrieve topk result.
I think control the block size might be enough. 16K/64K block size seems to be a great tradeoff under most cases.