Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Converting RocksDB table to disc_copies empties the table #53

Open
Juliusan opened this issue Aug 7, 2024 · 2 comments
Open

Converting RocksDB table to disc_copies empties the table #53

Juliusan opened this issue Aug 7, 2024 · 2 comments

Comments

@Juliusan
Copy link

Juliusan commented Aug 7, 2024

When rocksdb_copies table gets transformed into disc_copies table in a single node environment, all the records of the table disappear. This can be seen from the following test case:

  1. Start Erlang single node with mnesia_rocksdb in libs. Do the following:
    mnesia:start(),
    {atomic, ok} = mnesia:change_table_copy_type(schema, node(), disc_copies),
    {atomic, ok} = mnesia:create_table(?TABLE_NAME, [
        {type, set},
        {record_name, some_rec},
        {attributes, record_info(fields, some_rec)},
        {disc_copies, [node()]}
    ]),
    mnesia:activity(transaction, fun() ->
        mnesia:write(?TABLE_NAME, #some_rec{some_id = a, some_int = 1, some_string = "something" }, write),
        mnesia:write(?TABLE_NAME, #some_rec{some_id = b, some_int = 2, some_string = "anything"  }, write),
        mnesia:write(?TABLE_NAME, #some_rec{some_id = c, some_int = 3, some_string = "everything"}, write),
        mnesia:write(?TABLE_NAME, #some_rec{some_id = d, some_int = 4, some_string = "nothing"   }, write)
    end),
    {ok, rocksdb_copies} = mnesia_rocksdb:register(),
    {atomic, ok} = mnesia:change_table_copy_type(?TABLE_NAME, node(), rocksdb_copies),
    a = mnesia:dirty_first(?TABLE_NAME),
    % {atomic, ok} = mnesia:change_table_copy_type(?TABLE_NAME, node(), ram_copies),
    % a = mnesia:dirty_first(?TABLE_NAME),
    {atomic, ok} = mnesia:change_table_copy_type(?TABLE_NAME, node(), disc_copies),
    a = mnesia:dirty_first(?TABLE_NAME),
    q().

Note that after table is transformed to disc_copies, the records (at least the first one) are still in it. They are available until the node is restarted.
Also note that two lines are commented out. I'll explain that later.

  1. Restart the node and do the following:
    mnesia:start(),
    ok = mnesia:wait_for_tables([schema, ?TABLE_NAME], 10000),
    a = mnesia:dirty_first(?TABLE_NAME),

The last line throws an exception: "no match of right hand side value '$end_of_table'". Therefore, the table is empty.

The definitions are as follows:

-define(TABLE_NAME, test_table).
-record(some_rec, {
    some_id     :: atom(),
    some_int    :: integer(),
    some_string :: string()
}).

If however the table is converted to ram_copies and only after that to disc_copies, everything works as expected: the table is not cleared. To demonstrate this, use the same code, but uncomment the two lines above.

@uwiger
Copy link
Contributor

uwiger commented Aug 8, 2024

At least in my initial attemps (using OTP 26.1.2), I can't replicate this.

Given how mnesia:change_table_copy_type/3 is implemented, my suspicion would be that if there is a bug, it will be in mnesia.

The relevant code is here:
https://github.com/erlang/otp/blob/master/lib/mnesia/src/mnesia_schema.erl#L2565-L2572

If an 'external copy' is involved, data is dumped to a log file, unless the target is ram_copies, in which case the ets table is created and then filled directly at commit. This would help explain why you get different results by going via ram_copies.

@Juliusan
Copy link
Author

Juliusan commented Aug 9, 2024

It's strange. I've ran into the same issue on OTP 26.2 on several different machines (my local computer - Debian Bookworm Linux and on server using Oracle Enterprise Linux 9). I then also tried using OTPs 26.1, 26.1.2 (the one you used) and 27.0.1 (the newest stable) on my local computer and I can still reliably reproduce the problem. Might that be related to OS?...
Thanks for your help. I'll try to register it as mnesia issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants