Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Character set conversion #4

Open
toddh opened this issue Aug 6, 2011 · 7 comments
Open

Character set conversion #4

toddh opened this issue Aug 6, 2011 · 7 comments

Comments

@toddh
Copy link

toddh commented Aug 6, 2011

When I tried it in a Ubuntu environment (admittedly hosted via VirtualBox in Windows 7 64-bit), I got::

rake aborted!
“\xE2″ from ASCII-8BIT to UTF-8

I looked at my database.yml and they’re set to utf8 and the collation on the database tables is “utf8_unicode_ci.” Mysql is running in the Ubuntu machine.
@mattconnolly
Copy link
Owner

I'm setting up some tests at the moment, but everything I am trying is just working fine in UTF8. Can you try it with rake --trace so I can see where the exception is coming from? Where is the Ascii 8 bit string coming from?

Otherwise, I can't reproduce it and that'll make it much harder to help fix.

@toddh
Copy link
Author

toddh commented Aug 9, 2011

Here's an output from rake --trace. I think I mentioned this, but in case I
didn't. I'm running in Ubuntu (VirtualBox inside of Windows 7) and the
MySQL server is also in Ubuntu on the same machine. As near as I can tell,
everything should be UTF8.

--Todd

/home/todd/.rvm/rubies/ruby-1.9.2-p290/bin/ruby -e
$stdout.sync=true;$stderr.sync=true;load($0=ARGV.shift)
/home/todd/.rvm/gems/ruby-1.9.2-p290@global/bin/rake
site:backup[backup_file] --trace
** Invoke site:backup (first_time)
** Invoke site:backup:_save_db_schema (first_time)
** Invoke environment (first_time)
** Execute environment
** Invoke db:schema:dump (first_time)
** Invoke db:load_config (first_time)
** Invoke rails_env (first_time)
** Execute rails_env
** Execute db:load_config
** Execute db:schema:dump
** Execute site:backup:_save_db_schema
** Invoke site:backup:_save_db_to_yml (first_time)
** Invoke environment
** Execute site:backup:_save_db_to_yml
rake aborted!
"\xE2" from ASCII-8BIT to UTF-8
/home/todd/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:17:in
write' /home/todd/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:17:in end_document'
/home/todd/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:17:in
visit_Psych_Nodes_Document' /home/todd/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/visitor.rb:10:in accept'
/home/todd/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:10:in
block in visit_Psych_Nodes_Stream' /home/todd/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:10:in each'
/home/todd/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:10:in
visit_Psych_Nodes_Stream' /home/todd/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/visitor.rb:11:in accept'
/home/todd/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/nodes/node.rb:36:in
to_yaml' /home/todd/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych.rb:166:indump'
/home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rails-backup-migrate-0.0.8/lib/rails-backup-migrate.rb:104:in
block (2 levels) in save_db_to_yml' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rails-backup-migrate-0.0.8/lib/rails-backup-migrate.rb:104:in open'
/home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rails-backup-migrate-0.0.8/lib/rails-backup-migrate.rb:104:in
block in save_db_to_yml' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rails-backup-migrate-0.0.8/lib/rails-backup-migrate.rb:102:in each'
/home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rails-backup-migrate-0.0.8/lib/rails-backup-migrate.rb:102:in
save_db_to_yml' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rails-backup-migrate-0.0.8/lib/tasks/rails-backup-migrate.rake:25:in block (3 levels) in <top (required)>'
/home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/task.rb:205:in
call' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/task.rb:205:in block in execute'
/home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/task.rb:200:in
each' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/task.rb:200:in execute'
/home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/task.rb:158:in
block in invoke_with_call_chain' /home/todd/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/monitor.rb:201:in mon_synchronize'
/home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/task.rb:151:in
invoke_with_call_chain' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/task.rb:176:in block in invoke_prerequisites'
/home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/task.rb:174:in
each' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/task.rb:174:in invoke_prerequisites'
/home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/task.rb:157:in
block in invoke_with_call_chain' /home/todd/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/monitor.rb:201:in mon_synchronize'
/home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/task.rb:151:in
invoke_with_call_chain' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/task.rb:144:in invoke'
/home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/application.rb:112:in
invoke_task' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/application.rb:90:in block (2 levels) in top_level'
/home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/application.rb:90:in
each' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/application.rb:90:in block in top_level'
/home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/application.rb:129:in
standard_exception_handling' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/application.rb:84:in top_level'
/home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/application.rb:62:in
block in run' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/application.rb:129:in standard_exception_handling'
/home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/application.rb:59:in
run' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/bin/rake:32:in <top (required)>'
/home/todd/.rvm/gems/ruby-1.9.2-p290@global/bin/rake:19:in load' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/bin/rake:19:in<top
(required)>'
-e:1:in load' -e:1:in

'
Tasks: TOP => site:backup => site:backup:_save_db_to_yml

Process finished with exit code 1

On Tue, Aug 9, 2011 at 7:06 AM, soundevolution <
[email protected]>wrote:

I'm setting up some tests at the moment, but everything I am trying is just
working fine in UTF8. Can you try it with rake --trace so I can see where
the exception is coming from? Where is the Ascii 8 bit string coming from?

Otherwise, I can't reproduce it and that'll make it much harder to help
fix.

Reply to this email directly or view it on GitHub:

https://github.com/soundevolution/rails-backup-migrate/issues/4#issuecomment-1762460

############################################
# Please note new email address: [email protected]
# (I'm migrating away from Yahoo mail)

@toddh
Copy link
Author

toddh commented Aug 10, 2011

I see I didn't ask your second question. Not being character set savvy, I'm not sure how to figure out where the Ascii 8 bit string is coming from. Is there a query I can run on the DB? I could perhaps do mysqldump and send that. Would that provide a clue?

@mattconnolly
Copy link
Owner

The stack trace helps. I can see it's happening in the yaml export in ruby 1.9.2. Admittedly, I'd only tested on 1.8.7 since the rails apps I've used all used that. It's easy to repro on ruby 1.9.2 with rails 3, and I'm taking a look into it.

mattconnolly added a commit that referenced this issue Aug 10, 2011
@mattconnolly
Adding known issue #4 in github to notes for others to read until fixed.
f60d656
@mattconnolly
Copy link
Owner

Ruby 1.9.2 definitely treats YML files very differently to 1.8.7. I have a solution not too far away...

@mattconnolly
Copy link
Owner

After chasing my tail round in circles, I undid a whole bunch of my changes, and then with only some minor changes, couldn't reproduce the issue in 1.9.2. I've updated version 0.0.9 with these minimal changes. Let me know if this helps or not...

@toddh
Copy link
Author

toddh commented Aug 22, 2011

I think that this does solve the problem -- but can't completely verify it because I'm running into potentially another issue (remember I'm a noob at Ruby):

It fails to create the tar file because files_to_archive is empty in save_db_to_yml. On my system, the culprit appears to be add_to_archive which only adds something to list if it starts with RAILS_ROOT (which I think is deprecate in 3.1). The string being passed in is "db/schema.rb" and "db/backup", neither of which starts with RAILS_ROOT.

I added an:

      else
        @files_to_archive << path

to add_to_archive and that seems to have fixed the problem.

Thoughts?

The tar file does look correct, but I haven't had time to try a restore yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mattconnolly @toddh