Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CytubeBot fails to create a working database #51

Open
SeleneASong opened this issue Feb 14, 2016 · 3 comments
Open

CytubeBot fails to create a working database #51

SeleneASong opened this issue Feb 14, 2016 · 3 comments

Comments

@SeleneASong
Copy link

When CytubeBot creates it's database, it does it in such a way that it'll crash when it tries to supply values to the users table. I have fixed this on my own machine by manually dropping the users table and creating a new one with this command:
CREATE TABLE users(uname TEXT, blacklisted TEXT, block TEXT, rank INTEGER, primary key(uname));

I believe the bug exists in line 22 of lib/database.js, which reads:

this.db.run("CREATE TABLE IF NOT EXISTS users(uname TEXT, blacklisted TEXT, block TEXT, primary key(uname))")

but should read:

this.db.run("CREATE TABLE IF NOT EXISTS users(uname TEXT, blacklisted TEXT, block TEXT, rank INTEGER, primary key(uname))")

Adding "rank INTEGER," which seems to have been left out.

@nuclearace
Copy link
Owner

Rank should be added when it updates the tables though. https://github.com/nuclearace/CytubeBot/blob/master/lib/database.js#L40

@SeleneASong
Copy link
Author

For me, it crashed before it got to that point.

http://pastebin.com/QkWEVWy9

@awused
Copy link

awused commented Sep 15, 2016

The database initialization code is fragile and has race conditions. What probably happened is the disk was slow and the network was fast so the bot joined the room and got the list of users before running ALTER TABLE users ADD rank INTEGER.

  1. serialize() does not lock the database. Other threads can still put new commands onto the queue. Your use of update.run(callback) gives another thread ample opportunity to run a command (inserting users, in this case) before the ALTER TABLE command is put onto the queue. Even without the callback other threads could still interleave commands with the initialization code.
  2. The version number is incremented before issuing the ALTER TABLE command. If the ALTER TABLE command fails it will never be reattempted because the version is already '1'. These should be done together in a transaction so that if one fails both are rolled back and reattempted when the bot restarts.
  3. parallelize() is never called unless it's the first time the bot is run. On all subsequent runs the database will remain in serial mode. I'd image the database is one of your bottlenecks, especially with no indices.

The trick for #1 is to lock the database during initialization but not require threads to acquire locks after that point or you'll remove any parallelism.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants