| //================================================================== | |
| // SPARK INSTRUMENTATION | |
| //================================================================== | |
| import com.codahale.metrics.{MetricRegistry, Meter, Gauge} | |
| import org.apache.spark.{SparkEnv, Accumulator} | |
| import org.apache.spark.metrics.source.Source | |
| import org.joda.time.DateTime | |
| import scala.collection.mutable |
If native libraries are not available the following message is displayed with every hadoop command: hadoop checknative
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Clone hadoop source code
| ### | |
| ### [2023-06-19] UPDATE: Just tried to use my instructions again on a fresh install and it failed in a number of places. | |
| ###. Not sure if I'll update this gist (though I realise it seems to still have some traffic), but here's a list of | |
| ###. things to watch out for: | |
| ### - Check out the `nix-darwin` instructions, as they have changed. | |
| ### - There's a home manager gotcha https://github.com/nix-community/home-manager/issues/4026 | |
| ### | |
| # I found some good resources but they seem to do a bit too much (maybe from a time when there were more bugs). | |
| # So here's a minimal Gist which worked for me as an install on a new M1 Pro. |