deobfuscate: Squash server reports Psych::SyntaxError "mapping keys are not allowed in this context"

[richarddbarnett]

Using deobfuscate from 1.0.3 (repo head from earlier today) to upload a mapping file generated by yGuard, to Squash web (commit cb0b3ca), deobfuscate fails:

/Library/Ruby/Gems/1.8/gems/squash_uploader-1.0.1/lib/squash/uploader.rb:112:in `http_post': Unexpected response from Squash host: 500 (RuntimeError)

and the server reports (truncated for brevity):

Started POST "/api/1.0/deobfuscation.json" for 124.149.71.108 at 2013-10-15 04:19:37 +0000
Processing by Api::V1Controller#deobfuscation as JSON
  Parameters: {"environment"=>"workstation", "api_key"=>"b642ded2-feba-4882-a183-24cb6f02882f", "namespace"=>"...", "build"=>"280b639ad95b39c2aebb5dca088c993beeb9ccce"}
Completed 500 Internal Server Error in 123ms

Psych::SyntaxError ((<unknown>): mapping keys are not allowed in this context at line 4 column 40):
  app/controllers/api/v1_controller.rb:117:in `deobfuscation'


  Rendered /usr/local/share/gems1.9/gems/actionpack-4.0.0/lib/action_dispatch/middleware/templates/rescues/_source.erb (0.9ms)
  Rendered /usr/local/share/gems1.9/gems/actionpack-4.0.0/lib/action_dispatch/middleware/templates/rescues/_trace.erb (1.7ms)
  Rendered /usr/local/share/gems1.9/gems/actionpack-4.0.0/lib/action_dispatch/middleware/templates/rescues/_request_and_response.erb (26.1ms)
  Rendered /usr/local/share/gems1.9/gems/actionpack-4.0.0/lib/action_dispatch/middleware/templates/rescues/diagnostics.erb within rescues/layout (45.5ms)

deobfuscate is running on Mac OS 10.6.8, Ruby 1.8.7
Squash web is running on EC2 Amazon Linux 2013.03, Ruby 1.9.3p448

I am a Ruby package/app novice so I don't know how to investigate this any further.

[richarddbarnett]

I ran deobfuscate on the machine where Squash web is running, assuming (perhaps incorrectly) that it would use the same YAML library.

The mapping file was the same, the source largely the same (some files have been added that weren't mentioned in the mapping file, no files removed).

The server now logs

Completed 500 Internal Server Error in 203ms

SystemStackError (stack level too deep):
  squash_rails (1.2.0) lib/squash/rails/rack.rb:47

I modified deobfuscate to log the YAML before sending, and it's quite different. The Ruby 1.8 machine produced YAML like:

--- !ruby/object:Squash::Java::Namespace
package_roots: !ruby/object:Set
  hash:
    !ruby/object:Squash::Java::Package ?
      children: !ruby/object:Set
        hash: {}

      classes: !ruby/object:Set
        hash: {}

      name: TwilioHttpSmsReceiver
      parent:
    : true

while the Ruby 1.9 machine produced YAML like:

--- !ruby/object:Squash::Java::Namespace
package_roots: !ruby/object:Set
  hash:
    ? !ruby/object:Squash::Java::Package
      name: Thread
      parent:
      children: !ruby/object:Set
        hash: {}
      classes: !ruby/object:Set
        hash: {}
    : true

so (based on http://en.wikipedia.org/wiki/YAML#Syntax) the Ruby 1.8 machine YAML is clearly wrong in its placement of ? & the Psych error makes sense given this.

Does this help at all?

[richarddbarnett]

Another data point: running deobfuscate on another EC2 instance with Ruby 1.8.7 produces the "bad" YAML too.

[RISCfuture]

Is your code open-source by any chance? I can try running deobfuscate against it on a variety of environments to try and figure this out.

[richarddbarnett]

Sorry, I should have said that this is against a private repo.
I might see if I can set up a public repo for you to test with.
I might also see if upgrading to Ruby 1.9 on the other machines fixes this.

[richarddbarnett]

Am I hitting http://bugs.ruby-lang.org/issues/1331?
I tried running deobfuscate on an EC2 machine with only Ruby 1.9 and it failed badly.