explanation

I've added more comments to this code to make it more clear what
everything does.

ActiveSupport::Testing::Isolation runs its tests in parallel, and
JRuby has real multithreading. So, what happens under JRuby is
the first test executes, sets the environment variables, and
spawns a child process. In parallel, a second test executes, sets
the environment variables to different values, and spawns another
child process. It's this setting of environment variables that is
a problem - the second test is being run in parallel and sets the
environment variables to different values after the first test
has set the environment variables but before the first test has
spawned its child process.

In short, it's just a race condition introduced because
ActiveSupport::Testing::Isolation isn't thread-safe. Fixing this
will involve either passing data to the child process using some
other data structure that isn't shared among all threads like ENV
is, or adding some synchronization to the code.

This would break on any Ruby implementation that has true
multithreading, not just JRuby. So it's probably broken on
Rubinius as well.

isolation.rb

      module Subprocess
        ORIG_ARGV = ARGV.dup unless defined?(ORIG_ARGV)

        # Crazy H4X to get this working in windows / jruby with
        # no forking.
        def run_in_isolation(&blk)
          require "tempfile"

          # This evaluates to false in the parent process and true
          # in the child process
          if ENV["ISOLATION_TEST"]
            # Run the passed-in test block
            yield
            # Write the child's results to the path given to it by the parent
            File.open(ENV["ISOLATION_OUTPUT"], "w") do |file|
              file.puts [Marshal.dump(self.dup)].pack("m")
            end
            exit!
          else
            # Create a temporary file to pass results from the child
            # process back to the parent
            Tempfile.open("isolation") do |tmpfile|
              # Set ISOLATION_TEST so the child process knows it is
              # the child
              ENV["ISOLATION_TEST"]   = self.class.name
              # Specify the path the child should write its results to
              ENV["ISOLATION_OUTPUT"] = tmpfile.path

              load_paths = $-I.map {|p| "-I\"#{File.expand_path(p)}\"" }.join(" ")
              # Spawn the child process
              `#{Gem.ruby} #{load_paths} #{$0} #{ORIG_ARGV.join(" ")}`

              # Unset these environment variables once the child has finished
              ENV.delete("ISOLATION_TEST")
              ENV.delete("ISOLATION_OUTPUT")

              # Read and return the child's results
              return tmpfile.read.unpack("m")[0]
            end
          end
        end
      end

@bbrowning @robin850: RBX uses ActiveSupport::Testing::Isolation::Forking instead of ActiveSupport::Testing::Isolation::Subprocess so I don't think it's affected by this (the tests finish, 13 failures mostly related to YAML marshalling). Let me know if I can be of any help.

I didn't see the updated version of the gist nor the comment, sorry guys!

@febuiles: Yes, actually this won't break on Rubinius because it has forking so this part is never reached by it but if this piece of code was used, this'd break.

Thank you guys! ❤️