Node wrapper

[wshager]

I'm trying to create a node wrapper using node-ffi. It seems pretty straightforward, but I get a segfault when trying to work with cursors. Please consider the code: https://gist.github.com/wshager/edf7bb5d58d4873fea88

[cruppstahl]

Can you tell me how i can run this? I have always been curious about
node.js, but have zero knowledge about it.

2016-02-05 15:18 GMT+01:00 Wouter Hager [email protected]:

I'm trying to create a node wrapper using node-ffi It seems pretty
straightforward, but I get a segfault when trying to work with cursors
Please consider the code:
https://gistgithubcom/wshager/edf7bb5d58d4873fea88

—
Reply to this email directly or view it on GitHub
#70.

[wshager]

Simply install node and run node ups.js
You'll need to install the node-ffi dependency using npm:
npm install ffi

https://github.com/node-ffi/node-ffi

[wshager]

Docs for node-ref: https://tootallnate.github.io/ref/

[cruppstahl]

I got it running.

When I insert a lot of "console.log()" statements, then I see that it
crashes in the function check_error, called by open_env. More precisely, it
crashes when calling ups.ups_strerror(st);

I tried to change the declaration of ups_strerror to

'ups_strerror': ['string', [ups_status_t]]

but this still did not work. When I remove the call to ups_strerror then
everything seems to work, although it runs into a different error:

opened env <Buffer@0xa87170 00 00 00 00 00 00 00 00>

/mnt/f139c910-8da7-4153-a314-ca180468b45b/prj/nodejs/node_modules/ffi/lib/_foreign_function.js:55
throw e
^
TypeError: error setting argument 2 - value is out of bounds
at TypeError ()
at checkInt (buffer.js:784:11)
at Buffer.writeInt32LE (buffer.js:916:5)
at Object.set
(/mnt/f139c910-8da7-4153-a314-ca180468b45b/prj/nodejs/node_modules/ffi/node_modules/ref/lib/ref.js:923:52)
at Object.set
(/mnt/f139c910-8da7-4153-a314-ca180468b45b/prj/nodejs/node_modules/ffi/node_modules/ref/lib/ref.js:482:10)
at Object.alloc
(/mnt/f139c910-8da7-4153-a314-ca180468b45b/prj/nodejs/node_modules/ffi/node_modules/ref/lib/ref.js:516:13)
at Object.ForeignFunction.proxy as ups_env_open_db
at open_db
(/mnt/f139c910-8da7-4153-a314-ca180468b45b/prj/nodejs/ups.js:110:16)
at Object.
(/mnt/f139c910-8da7-4153-a314-ca180468b45b/prj/nodejs/ups.js:66:10)
at Module._compile (module.js:456:26)

Do you know how to fix ups_strerror? It looks to me like node is not able
to convert the temporary char * pointer to a javascript string.

2016-02-05 15:36 GMT+01:00 Wouter Hager [email protected]:

Docs for node-ref: https://tootallnate.github.io/ref/

—
Reply to this email directly or view it on GitHub
#70 (comment)
.

[wshager]

@cruppstahl Ah yes, I was looking in the wrong place. You're right, string seems to be the correct type for anywhere that I've used char_ptr. I get a lot of 'unknow errors' now, unfortunately. I'll update the gist.

[wshager]

@cruppstahl the status is stringified in the line before the actual call to ups_strerror and its subsequent output. Perhaps the contents of the status pointer could tell you something?

[wshager]

@cruppstahl Ah of course, I think it says 'unknown error' because there is none. Please ignore the previous comment, the data of status is empty every time!

[wshager]

@cruppstahl I've updated the gist to a working script, but I don't understand the values of the buffers. It's getting late, I'll continue tomorrow.

https://gist.github.com/wshager/edf7bb5d58d4873fea88

[wshager]

@cruppstahl Are you still interested in supporting node? I would like to give this another try, but I'm having trouble with doing inserts, as I don't have the correct key/record types.

[cruppstahl]

If you want to work on it then go ahead, but I'm so swamped right now that I won't be able to help much. It does not have high priority for me, though.

[wshager]

@cruppstahl Ok, thanks for your reply, I figured you would be. I hope and assume your hard work will pay off.

I made little progress, but it seems I was able to remove some data. One of the problems is that node can't seem to write an empty db file on its own, so I wondered if that is something you could help with.

[wshager]

Ah my bad, I was using the wrong function to create a db file.

[cruppstahl]

Feel free to send me the code in case you get stuck. Am 29.03.2017 13:20 schrieb "Wouter Hager" <[email protected]>:

…

Ah my bad, I was using the wrong function to create a db file. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#70 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMxg4A4S6-TB9aC7SXFcWt3jEjU3oghks5rqj5sgaJpZM4HUTiU> .

[wshager]

Hi, I got something working! I've updated the gist (https://gist.github.com/wshager/edf7bb5d58d4873fea88).

Some things to consider:

• I tried to create keys/records with generic types, but I couldn't get it to work. I'm not sure, as I don't know enough about ref-struct, but I guess the problem is that CString pointers need to be allocated as such. I have the same problem with key/record pointers in cursors: I need to pass a CString or it will give an 'invalid parameter' error (-8).
• When clearing all keys with cursor_erase, the database is larger than when first created. I can understand space is allocated for the btree, but I can see some record garbage in the file.

Thanks in advance for your response.

[wshager]

Aargh I'm not sharp enough for this job... Ignore the last bit, I forgot to close the db. Will update the gist now.

[wshager]

@cruppstahl I can't set parameters on ups_env_create_db, as I can't reproduce the correct struct array using ref-struct and ref-array. I really want to avoid having to write a generic connector in C.

[wshager]

Persistence pays off once again. I got the parameter array working. The trick was to pass params.buffer.

var Struct = require('ref-struct');
var ArrayType = require('ref-array');
var ups_parameter_t = Struct({
    'name': ref.types.uint32,
    'value': ref.types.uint64
});
const UPS_PARAM_KEY_TYPE = 0x00000104;
const UPS_TYPE_UINT32 = 7;
var Params = ArrayType(ups_parameter_t);
var db = ref.alloc('pointer');
var params = new Params(2);
params[0] = new ups_parameter_t({
    name: UPS_PARAM_KEY_TYPE,
    value: UPS_TYPE_UINT32 
});
params[1] = new ups_parameter_t({
    name: 0,
    value:0
});
var st = ups.ups_env_create_db(env.deref(), db, nr, 0, params.buffer);

[paulocoghi]

Hello Wouter and Christoph, sorry to "revive" this old issue.

I would like to ask Wouter if the Node wrapper worked and ask Christoph if you are still using UpscaleDB in production, because it seems really mature.

Thanks!

[cruppstahl]

Hi @paulocoghi - mature yes, but I'm no longer actively maintaining upscaledb. Using it is at your own risk!