Created
September 24, 2012 14:08
-
-
Save costin/3776128 to your computer and use it in GitHub Desktop.
Mahout SHDP Tool Config
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 17:33:32,579 INFO main context.TestContextManager - @TestExecutionListeners is not present for class [class org.springframework.data.hadoop.MahoutTests]: using defaults. | |
| 17:33:32,710 INFO main xml.XmlBeanDefinitionReader - Loading XML bean definitions from class path resource [org/springframework/data/hadoop/MahoutTests-context.xml] | |
| 17:33:32,917 INFO main xml.XmlBeanDefinitionReader - Loading XML bean definitions from class path resource [org/springframework/data/hadoop/hadoop-ctx.xml] | |
| 17:33:33,074 INFO main support.GenericApplicationContext - Refreshing org.springframework.context.support.GenericApplicationContext@59556d12: startup date [Mon Sep 24 17:33:33 EEST 2012]; root of context hierarchy | |
| 17:33:33,196 INFO main config.PropertyPlaceholderConfigurer - Loading properties file from class path resource [test.properties] | |
| 17:33:33,234 INFO main support.DefaultListableBeanFactory - Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@7a2431b9: defining beans [org.springframework.beans.factory.config.PropertyPlaceholderConfigurer#0,hadoopFs,org.springframework.data.hadoop.mapreduce.MapReducePropertyEditorRegistrar.ns.registration,hadoopResourceLoader,hadoopConfiguration,org.springframework.data.hadoop.scripting.HdfsScriptFactoryBean#0,mahout-tool,org.springframework.context.annotation.internalConfigurationAnnotationProcessor,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequiredAnnotationProcessor,org.springframework.context.annotation.internalCommonAnnotationProcessor]; root of factory hierarchy | |
| 17:33:33,279 WARN main conf.Configuration - DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively | |
| 17:33:33,688 WARN main util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable | |
| 17:33:33,688 WARN main snappy.LoadSnappy - Snappy native library not loaded | |
| Resource type class org.springframework.core.io.UrlResource file:/libs/hadoop/mahout/mahout-distribution-0.7/mahout-examples-0.7-job.jar 30037387 | |
| SLF4J: Class path contains multiple SLF4J bindings. | |
| SLF4J: Found binding in [jar:file:/libs/hadoop/mahout/mahout-distribution-0.7/mahout-examples-0.7-job.jar!/org/slf4j/impl/StaticLoggerBinder.class] | |
| SLF4J: Found binding in [jar:file:/Users/costin/.gradle/caches/artifacts-14/filestore/org.slf4j/slf4j-log4j12/1.6.6/jar/5cd9b4fbc3ff6a97beaade3206137d76f65df805/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class] | |
| SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. | |
| log4j:ERROR A "org.apache.log4j.ConsoleAppender" object is not assignable to a "org.apache.log4j.Appender" variable. | |
| log4j:ERROR The class "org.apache.log4j.Appender" was loaded by | |
| log4j:ERROR [ParentLastURLCL | |
| URLs: [file:/libs/hadoop/mahout/mahout-distribution-0.7/mahout-examples-0.7-job.jar] | |
| Parent CL: sun.misc.Launcher$AppClassLoader@63c78e57 | |
| System CL: sun.misc.Launcher$ExtClassLoader@425224ee | |
| ] whereas object of type | |
| log4j:ERROR "org.apache.log4j.ConsoleAppender" was loaded by [sun.misc.Launcher$AppClassLoader@63c78e57]. | |
| log4j:ERROR Could not instantiate appender named "stdout". | |
| 17:33:35,091 INFO main mapreduce.ToolRunner - Invoking [org.apache.mahout.cf.taste.hadoop.item.RecommenderJob@252a78ee] from jar [file:/libs/hadoop/mahout/mahout-distribution-0.7/mahout-examples-0.7-job.jar] with args [[-s SIMILARITY_LOGLIKELIHOOD, -i/tmp/mah/intro.csv, -o recommendations]] | |
| log4j:WARN No appenders could be found for logger (org.apache.mahout.common.AbstractJob). | |
| log4j:WARN Please initialize the log4j system properly. | |
| log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. | |
| 17:33:35,634 INFO main input.FileInputFormat - Total input paths to process : 1 | |
| 17:33:36,079 INFO main mapred.JobClient - Running job: job_local_0001 | |
| 17:33:36,147 INFO Thread-10 mapred.Task - Using ResourceCalculatorPlugin : null | |
| 17:33:36,176 INFO Thread-10 mapred.MapTask - io.sort.mb = 100 | |
| 17:33:36,221 INFO Thread-10 mapred.MapTask - data buffer = 79691776/99614720 | |
| 17:33:36,221 INFO Thread-10 mapred.MapTask - record buffer = 262144/327680 | |
| 17:33:36,254 WARN Thread-10 mapred.LocalJobRunner - job_local_0001 | |
| java.lang.ArrayIndexOutOfBoundsException: 1 | |
| at org.apache.mahout.cf.taste.hadoop.item.ItemIDIndexMapper.map(ItemIDIndexMapper.java:47) | |
| at org.apache.mahout.cf.taste.hadoop.item.ItemIDIndexMapper.map(ItemIDIndexMapper.java:31) | |
| at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) | |
| at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) | |
| at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) | |
| at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212) | |
| 17:33:37,083 INFO main mapred.JobClient - map 0% reduce 0% | |
| 17:33:37,086 INFO main mapred.JobClient - Job complete: job_local_0001 | |
| 17:33:37,086 INFO main mapred.JobClient - Counters: 0 | |
| 17:33:37,189 WARN main mapreduce.ExecutionUtils - Trying to patch leaked cl [ParentLastURLCL | |
| URLs: [file:/libs/hadoop/mahout/mahout-distribution-0.7/mahout-examples-0.7-job.jar] | |
| Parent CL: sun.misc.Launcher$AppClassLoader@63c78e57 | |
| System CL: sun.misc.Launcher$ExtClassLoader@425224ee | |
| ] in thread [Thread[communication thread,5,main]] | |
| 17:33:37,189 WARN main mapreduce.ExecutionUtils - Trying to patch leaked cl [ParentLastURLCL | |
| URLs: [file:/libs/hadoop/mahout/mahout-distribution-0.7/mahout-examples-0.7-job.jar] | |
| Parent CL: sun.misc.Launcher$AppClassLoader@63c78e57 | |
| System CL: sun.misc.Launcher$ExtClassLoader@425224ee | |
| ] in thread [Thread[SpillThread,5,main]] | |
| 17:33:37,200 INFO Thread-3 support.GenericApplicationContext - Closing org.springframework.context.support.GenericApplicationContext@59556d12: startup date [Mon Sep 24 17:33:33 EEST 2012]; root of context hierarchy | |
| 17:33:37,201 INFO Thread-3 support.DefaultListableBeanFactory - Destroying singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@7a2431b9: defining beans [org.springframework.beans.factory.config.PropertyPlaceholderConfigurer#0,hadoopFs,org.springframework.data.hadoop.mapreduce.MapReducePropertyEditorRegistrar.ns.registration,hadoopResourceLoader,hadoopConfiguration,org.springframework.data.hadoop.scripting.HdfsScriptFactoryBean#0,mahout-tool,org.springframework.context.annotation.internalConfigurationAnnotationProcessor,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequiredAnnotationProcessor,org.springframework.context.annotation.internalCommonAnnotationProcessor]; root of factory hierarchy |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <?xml version="1.0" encoding="UTF-8"?> | |
| <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | |
| xmlns:hdp="http://www.springframework.org/schema/hadoop" | |
| xmlns:batch="http://www.springframework.org/schema/batch" | |
| xmlns:util="http://www.springframework.org/schema/util" | |
| xmlns:p="http://www.springframework.org/schema/p" | |
| xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd | |
| http://www.springframework.org/schema/hadoop http://www.springframework.org/schema/hadoop/spring-hadoop.xsd | |
| http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd"> | |
| <import resource="./hadoop-ctx.xml"/> | |
| <hdp:script run-at-startup="true" language="javascript"> | |
| if (fsh.test("temp")) | |
| fsh.rmr("temp") | |
| if (fsh.test("tmp/mah/intro.csv")) | |
| fsh.put("/libs/hadoop/mahout/mahout-distribution-0.7/intro.csv", "/tmp/mah/intro.csv" | |
| </hdp:script> | |
| <hdp:tool-runner id="mahout-tool" | |
| tool-class="org.apache.mahout.cf.taste.hadoop.item.RecommenderJob" | |
| jar="/libs/hadoop/mahout/mahout-distribution-0.7/mahout-examples-0.7-job.jar"> | |
| <hdp:arg value="-s SIMILARITY_LOGLIKELIHOOD" /> | |
| <hdp:arg value="-i/tmp/mah/intro.csv" /> | |
| <hdp:arg value="-o recommendations" /> | |
| </hdp:tool-runner> | |
| </beans> |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment