XMPP Input/Output: Chinese characters crashes Logstash

[untergeek]

I have an XMPP chat that I use as input for logstash (1.4.2). When I tried to write Chinese characters in the chat, Logstash crashes quitely.

_Command (Logstash.1.4.2)_

bin/logstash agent --debug -vf logstash-simple.conf

_Value input to chat (sorry forgot half ~~)_

...怎么样?

_longstash-simple.conf_

input {
  xmpp {
    password => "help!"
    rooms => ["[email protected]/rat-helperman"]
    type => "a"
    user => "[email protected]"
  }
}

output {
  # Print each event to stdout.
  stdout {
    # Enabling 'rubydebug' codec on the stdout output will make logstash
    # pretty-print the entire event as something similar to a JSON representation.
    codec => rubydebug
  }
}

_Error message_

[2014-07-14T20:21:22.767000 #618] DEBUG -- : PROCESSING:
    :beer::kiss:=-O:-/8-) (Jabber::Message)
D, [2014-07-14T20:21:22.768000 #618] DEBUG -- : TRYING stanzacbs...
D, [2014-07-14T20:21:22.769000 #618] DEBUG -- : TRYING message/iq/presence/cbs...
output received {:event=>{"message"=>":beer::kiss:=-O:-/8-)", "@version"=>"1", "@timestamp"=>"2014-07-14T18:21:22.780Z", "type"=>"a", "room"=>"rat-help@xxx/rat-helperman", "from"=>"ewamagn"}, :level=>:debug, :file=>"(eval)", :line=>"16"}
W, [2014-07-14T20:21:22.792000 #618]  WARN -- : EXCEPTION:
    REXML::ParseException
    #
    org/jruby/RubyRegexp.java:1697:in `match'
    file:/home/ewamagn/programming/search/logstash2/logstash-1.4.2/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/1.9/rexml/source.rb:210:in `match'
    file:/home/ewamagn/programming/search/logstash2/logstash-1.4.2/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/1.9/rexml/parsers/baseparser.rb:419:in `pull_event'
    file:/home/ewamagn/programming/search/logstash2/logstash-1.4.2/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/1.9/rexml/parsers/baseparser.rb:183:in `pull'
    file:/home/ewamagn/programming/search/logstash2/logstash-1.4.2/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/1.9/rexml/parsers/sax2parser.rb:92:in `parse'
    /home/ewamagn/programming/search/logstash2/logstash-1.4.2/vendor/bundle/jruby/1.9/gems/xmpp4r-0.5/lib/xmpp4r/streamparser.rb:79:in `parse'
    /home/ewamagn/programming/search/logstash2/logstash-1.4.2/vendor/bundle/jruby/1.9/gems/xmpp4r-0.5/lib/xmpp4r/stream.rb:75:in `start'
    ...
    Exception parsing
    Line: 
    Position: 0
    Last 80 unconsumed characters:
    丬å½çæ±å¬ï¼æä¹æ •?
    file:/home/ewamagn/programming/search/logstash2/logstash-1.4.2/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/1.9/rexml/parsers/baseparser.rb:435:in `pull_event'
    file:/home/ewamagn/programming/search/logstash2/logstash-1.4.2/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/1.9/rexml/parsers/baseparser.rb:183:in `pull'
    file:/home/ewamagn/programming/search/logstash2/logstash-1.4.2/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/1.9/rexml/parsers/sax2parser.rb:92:in `parse'
    /home/ewamagn/programming/search/logstash2/logstash-1.4.2/vendor/bundle/jruby/1.9/gems/xmpp4r-0.5/lib/xmpp4r/streamparser.rb:79:in `parse'
    /home/ewamagn/programming/search/logstash2/logstash-1.4.2/vendor/bundle/jruby/1.9/gems/xmpp4r-0.5/lib/xmpp4r/stream.rb:75:in `start'
{
       "message" => ":beer::kiss:=-O:-/8-)",
      "@version" => "1",
    "@timestamp" => "2014-07-14T18:21:22.780Z",
          "type" => "a",
          "room" => "[email protected]/rat-helperman",
          "from" => "ewamagn"
}
W, [2014-07-14T20:21:23.260000 #618]  WARN -- : EXCEPTION:
    TypeError
    can't convert REXML::ParseException into String
    org/jruby/RubyString.java:1159:in `+'
    /home/ewamagn/programming/search/logstash2/logstash-1.4.2/vendor/bundle/jruby/1.9/gems/xmpp4r-0.5/lib/xmpp4r/stream.rb:136:in `parse_failure'
    /home/ewamagn/programming/search/logstash2/logstash-1.4.2/vendor/bundle/jruby/1.9/gems/xmpp4r-0.5/lib/xmpp4r/streamparser.rb:81:in `parse'
    /home/ewamagn/programming/search/logstash2/logstash-1.4.2/vendor/bundle/jruby/1.9/gems/xmpp4r-0.5/lib/xmpp4r/stream.rb:75:in `start'
W, [2014-07-14T20:21:23.261000 #618]  WARN -- : Exception caught in Parser thread! (TypeError)
    org/jruby/RubyString.java:1159:in `+'
    /home/ewamagn/programming/search/logstash2/logstash-1.4.2/vendor/bundle/jruby/1.9/gems/xmpp4r-0.5/lib/xmpp4r/stream.rb:136:in `parse_failure'
    /home/ewamagn/programming/search/logstash2/logstash-1.4.2/vendor/bundle/jruby/1.9/gems/xmpp4r-0.5/lib/xmpp4r/streamparser.rb:81:in `parse'
    /home/ewamagn/programming/search/logstash2/logstash-1.4.2/vendor/bundle/jruby/1.9/gems/xmpp4r-0.5/lib/xmpp4r/stream.rb:75:in `start'

[untergeek]

Until the upstream issue at lnussbaum/xmpp4r#3 is cleared up, this cannot move forward.

[sammcj]

@untergeek I think you linked to the wrong upstream for xmpp4r, gem version 0.5.6 fixes the issue and was released back in 2014 - https://github.com/xmpp4r/xmpp4r/releases

[ghost]

I can confirm that xmpp4r-0.5.6 fixes the problem.

[sammcj]

Hi @jsvd, thanks for the merge - we noticed however that the gem hasn't been updated on rubygems - is there anything I can do to help that happen or is there something I've missed that should trigger this to be updated?

https://rubygems.org/gems/logstash-output-xmpp