Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

To use this library for batch insertion when the amount of data is determined in runtime #51

Open
MelodicDrumstep opened this issue Aug 23, 2024 · 0 comments

Comments

@MelodicDrumstep
Copy link

MelodicDrumstep commented Aug 23, 2024

Hi here! Firstly, thanks a lot for writing this awesome C++ client library for influxdb! It really helps a lot.
Currently, I'm faced up with a situation where the incoming data will be placed in a circular buffer. And I have to read a batch of data from the buffer and insert them in a batch into the database every certain seconds. The problem is that the amount of data I will read each turn is determined in runtime.

The example you mentioned about batch insertion is

influxdb_cpp::builder()
    .meas("foo")  // series 1
    .field("x", 10)

    .meas("bar")  // series 2
    .field("y", 10.3)
    .send_udp("127.0.0.1", 8091);

which is cool but not useful under this situation. Therefore, I wrote the piece of code below to resolve this in a non-graceful way:

    /** 
    @param data a vector of integer data that I want to insert into the database. 
    The bucket name and the measurement name are set before this function.
    */
    void Insert(std::vector<int> & data) 
    {
        influxdb_cpp::builder builder;
        influxdb_cpp::builder * temp_ts_caller = &builder;

        for(int i = 0; i < data.size(); i++)
        {
            temp_ts_caller =  &((temp_ts_caller -> meas(measurement_)).field("value", data[i]));
        }
        reinterpret_cast<influxdb_cpp::detail::ts_caller *>(temp_ts_caller) -> post_http(si_);
    }

However, it turns out that it doesn't work. This is due to the lack of newline character between insertions in the "lines_" variable.

Therefore, I have to convert "lines_" data member to be public and add this line to the code : temp_ts_caller -> lines_ << "\n";
After this, the piece of code would be:

    /** 
    @param data a vector of integer data that I want to insert into the database. 
    The bucket name and the measurement name are set before this function.
    */
    void Insert(std::vector<int> & data) 
    {
        influxdb_cpp::builder builder;
        influxdb_cpp::builder * temp_ts_caller = &builder;

        for(int i = 0; i < data.size(); i++)
        {
            temp_ts_caller =  &((temp_ts_caller -> meas(measurement_)).field("value", data[i]));
            temp_ts_caller -> lines_ << "\n";
        }
        reinterpret_cast<influxdb_cpp::detail::ts_caller *>(temp_ts_caller) -> post_http(si_);
    }

This solves the problem, but it's not graceful. I don't know if it's because I find the wrong way to use batch insertion under this situation. And I do not have time to see which piece goes wrong (if any) in the source code in my situation.


By the way, I find myself failing to use ".timestamp" function for timestamp customization under this situation.

For example, below is the errorous code:

        influxdb_cpp::builder builder;
        influxdb_cpp::builder * temp_ts_caller = &builder;

        for(int i = 0; i < data.size(); i++)
        {
            auto now = std::chrono::system_clock::now();
            auto duration_ns = std::chrono::duration_cast<std::chrono::nanoseconds>(now.time_since_epoch());
            temp_ts_caller =  &((temp_ts_caller -> meas(measurement_)).field("value", data[i]).timestamp(duration_ns.count()));
            temp_ts_caller -> lines_ << "\n";       // Fix for the bug of the library. 
        }
        reinterpret_cast<influxdb_cpp::detail::ts_caller *>(temp_ts_caller) -> post_http(si_);

If I try to add the timestamp in this way, I cannot get any insertion done. What's going wrong?

Sincerely waiting for your reply!

@MelodicDrumstep MelodicDrumstep changed the title To use this client for batch insertion when the amount of data is determined in runtime To use this library for batch insertion when the amount of data is determined in runtime Aug 23, 2024
@orca-zhang orca-zhang pinned this issue Dec 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants