examples of the closure frame sharing across threads. these are in the context of a class level dsl with a code block registration and its execution from its instance. DSL 1, 2 and 3 all reproduce the problem, DSL 4 works around it but defeats the dsl idea. for the actual bug see https://jira.codehaus.org/browse/JRUBY-7167

frame_sharing.rb

# block executed using instance_exec
class DSL1
  def self.register(&block)
    @block = block
  end

  def execute(str)
    instance_exec(str, &self.class.block)
  end

  def self.block
    @block
  end
end
class Test1 < DSL1
  register do |str|
    puts "before: #{$~}"
    /.*/ =~ str
    puts "after: #{$~}\n\n"
  end
end

puts("DSL1")
Thread.new {Test1.new.execute("foo")}.join
Test1.new.execute("bar")
Thread.new {Test1.new.execute("baz")}.join

# block executed as a method using define_method
class DSL2
  def self.register(&block)
    define_method(:execute, block)
  end
end
class Test2 < DSL2
  register do |str|
    puts "before: #{$~}"
    /.*/ =~ str
    puts "after: #{$~}\n\n"
  end
end

puts("DSL2")
Thread.new {Test2.new.execute("foo")}.join
Test2.new.execute("bar")
Thread.new {Test2.new.execute("baz")}.join

# block executed as a method using define_method but called from another method
class DSL3
  def self.register(&block)
    define_method(:execute_block, block)
  end

  def execute(str)
    execute_block(str)
  end
end
class Test3 < DSL3
  register do |str|
    puts "before: #{$~}"
    /.*/ =~ str
    puts "after: #{$~}\n\n"
  end
end

puts("DSL3")
Thread.new {Test3.new.execute("foo")}.join
Test3.new.execute("bar")
Thread.new {Test3.new.execute("baz")}.join


# block executed as a method using define_method, calling out method with actual code,
class DSL4
  def self.register(&block)
    define_method(:execute, block)
  end
end
class Test4 < DSL4
  register do |str|
    do_match(str)
  end

  def do_match(str)
    puts "before: #{$~}"
    /.*/ =~ str
    puts "after: #{$~}\n\n"
  end
end

puts("DSL4")
Thread.new {Test4.new.execute("foo")}.join
Test4.new.execute("bar")
Thread.new {Test4.new.execute("baz")}.join

output.txt

DSL1
before:
after: foo

before: foo
after: bar

before: bar
after: baz

DSL2
before:
after: foo

before: foo
after: bar

before: bar
after: baz

DSL3
before:
after: foo

before: foo
after: bar

before: bar
after: baz

DSL4
before:
after: foo

before:
after: bar

before:
after: baz

So this is what I found from my explorations.

MRI does share $~ on the frame across all threads hitting a given closure. This probably qualifies as a bug, but it has been this way for as long as I can tell:

ruby
l = ->(a){ puts "before #{$}"; /.*/ = a; puts "after: #{$~}" }
l["foo"]
Thread.new{l["bar"]}
l["baz"]

outputs:

before
after: foo
before foo
after: baz

So any logic using the implicit frame-global $~ will have threading issues.

The way to avoid this is to use Regexp#match directly, which will return a MatchData. It also updates $~, but if you use its return value directly it is safe.

The JRuby issues were caused by us using $~ internally as a way to pass the MatchData back through a long call stack in RubyRegexp.java. Multiple threads could then interfere with each other and break #match in various ugly ways.

Thanks @headius - I am wondering if there's a way to avoid the problem without "breaking" the DSL as in the 4th example? in the PR you mention «Instead of a proc/lambda, use a procified method object. No sharing will occur» - I am not sure what you mean by "procified method object" and how I could apply this here?

Procified method object basically just means a #method that has been to_proc'ed:

2.1.0dev :001 > module Blah
2.1.0dev :002?>   def self.to_s(obj)
2.1.0dev :003?>     obj.to_s
2.1.0dev :004?>     end
2.1.0dev :005?>   end
 => nil 
2.1.0dev :006 > to_s = Blah.method(:to_s)
 => #<Method: Blah.to_s> 
2.1.0dev :007 > [1,2,3].map(&to_s)
 => ["1", "2", "3"] 

But as @tarcieri pointed out elsewhere, you can't easily rebind such a proc to a new self, so for DSL purposes it's perhaps not useful.

As for fixing the $~ issue...I have no answers for you. It is a flaw of how Ruby handles closures, and there's no clear fix. It probably should be raised with ruby-core.