Skip to content

Instantly share code, notes, and snippets.

@pablete
Last active September 5, 2017 18:44
Show Gist options
  • Save pablete/79112b41ef60dc80401f88d318656c15 to your computer and use it in GitHub Desktop.
Save pablete/79112b41ef60dc80401f88d318656c15 to your computer and use it in GitHub Desktop.
BDASSpark("com.netflix.simulation.terasort.TeraGen",
None,
List("$sortSize", "20", "/terasort/IN"),
Some("2.0"),
List(
("--conf", "spark.driver.cores=2"),
("--conf", "spark.driver.memory=24G"),
("--conf", "spark.executor.cores=2"),
("--conf", "spark.executor.memory=24G"),
("--conf", "spark.cores.max=40"),
("--conf", "spark.scheduler.minRegisteredResourcesRatio=1.0"),
("--conf", "spark.scheduler.maxRegisteredResourcesWaitingTime=1200s"),
("--conf", "spark.mesos.role=pvr")
)
)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment