gistfile1.txt

# Am I naive for thinking logs 2 through 10 should be approximately the same?
# Why does touching a single file have completely non-deterministic results?
# (I won't ask why everything has to be recompiled most of the time because a
# timestamp was updated.)

% for i in {1..10}; do touch src/compiler/scala/reflect/internal/Printers.scala && sbt --debug package &> log-$i.txt; done
% grep -i recompiling *.txt
log-1.txt:[debug] Recompiling all 1 sources: invalidated sources (1) exceeded 50.0% of all sources
log-1.txt:[debug] Recompiling all 77 sources: invalidated sources (77) exceeded 50.0% of all sources
log-1.txt:[debug] Recompiling all 450 sources: invalidated sources (450) exceeded 50.0% of all sources
log-1.txt:[debug] Recompiling all 48 sources: invalidated sources (48) exceeded 50.0% of all sources
log-2.txt:[debug] Recompiling all 450 sources: invalidated sources (450) exceeded 50.0% of all sources
log-2.txt:[debug] Recompiling all 48 sources: invalidated sources (48) exceeded 50.0% of all sources
log-3.txt:[debug] Recompiling all 450 sources: invalidated sources (417) exceeded 50.0% of all sources
log-4.txt:[debug] Recompiling all 48 sources: invalidated sources (48) exceeded 50.0% of all sources
log-5.txt:[debug] Recompiling all 48 sources: invalidated sources (48) exceeded 50.0% of all sources
log-6.txt:[debug] Recompiling all 48 sources: invalidated sources (48) exceeded 50.0% of all sources
log-7.txt:[debug] Recompiling all 48 sources: invalidated sources (48) exceeded 50.0% of all sources
log-8.txt:[debug] Recompiling all 48 sources: invalidated sources (48) exceeded 50.0% of all sources
log-9.txt:[debug] Recompiling all 48 sources: invalidated sources (48) exceeded 50.0% of all sources
log-10.txt:[debug] Recompiling all 48 sources: invalidated sources (48) exceeded 50.0% of all sources

It seems we can dispense with the "touch", it's the same if I just keep re-running package.

Nope, not the same. Without the touch I do get a long run of nothing-to-dos.

Oh, ha ha. The "stable" period there where it's recompiling 48 sources is only because it was broken thanks to the deletion of in-use classfiles. Those smaller logs are all full of this:

log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/Interpreter.scala:9: not found: type Settings
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/interpreter/IMain.scala:150: not found: type Global
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/interpreter/IMain.scala:171: not found: type Type
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/interpreter/IMain.scala:103: not found: type Settings
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/interpreter/IMain.scala:102: not found: type Settings
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/interpreter/IMain.scala:101: not found: type Settings
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/interpreter/IMain.scala:57: not found: type Settings
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/Interpreter.scala:10: not found: type Settings
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/Interpreter.scala:10: not found: type NewLinePrintWriter
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/Interpreter.scala:10: not found: type ConsoleWriter
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/Interpreter.scala:11: not found: type Settings
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/interpreter/ILoop.scala:19: not found: object ScalaClassLoader
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/MainGenericRunner.scala:11: not found: object Properties
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/MainGenericRunner.scala:12: not found: object GenericRunnerCommand
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/MainGenericRunner.scala:14: not found: type CommonRunner
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/MainGenericRunner.scala:15: not found: type GenericRunnerSettings
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/MainGenericRunner.scala:19: not found: value File
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/MainGenericRunner.scala:26: not found: value runAndCatch
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/MainGenericRunner.scala:45: not found: type GenericRunnerCommand
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/MainGenericRunner.scala:47: not found: type Global
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/MainGenericRunner.scala:47: not found: value settings
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/MainGenericRunner.scala:50: not found: value settings
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/MainGenericRunner.scala:50: not found: value versionString
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/MainGenericRunner.scala:53: not found: value settings
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/MainGenericRunner.scala:54: not found: value settings
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/MainGenericRunner.scala:56: not found: value settings
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/MainGenericRunner.scala:57: not found: value settings
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/MainGenericRunner.scala:61: not found: value settings
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/MainGenericRunner.scala:70: not found: value howToRun
log-4.txt:[error] /l/scala/src/repl/scala/tools/nsc/MainGenericRunner.scala:71: not found: value AsObject

Touching file shouldn't trigger any recompilation. Incremental compiler keeps track of hashes of source files and will recompile only upon change of hash sum.

Is there any way I could get my hands on your project? If not, could you enable logging of ClassfileManager activity so we would get better insight what's going on here?

What you discovered already gives me some clues so I'll poke around the code that calls into ClassfileManager. The full log of activity would be more revealing.

I looked at all calls to ClassfileManager and I didn't see anything suspicious. The code looks straightforward and correct. It would be great to learn when exactly class files get mistakenly deleted.

@paulp, any news on this?