jeromatron

java.lang.IllegalArgumentException: This file system object (hdfs://ip-10-114-89-36.ec2.internal:9000) does not support access to the request path 's3n://backlog.dev/1296648900000/32763897924550656' You possibly called FileSystem.get(conf) when you should of called FileSystem.get(uri, conf) to obtain a file system supporting your path.
	at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:351)
	at org.apache.hadoop.hdfs.DistributedFileSystem.checkPath(DistributedFileSystem.java:99)
	at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:155)
	at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:178)
	at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:396)
	at analytics.hadoop.socialdata.RawSignalFileInputFormat$MultiFileLineRecordReader.<init>(RawSignalFileInputFormat.java:53)
	at analytics.hadoop.socialdata.RawSignalFileInputFormat.getRecordReader(RawSignalFileInputFormat.java:22)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask

jeromatron

compile-test:
    [javac] /Users/jeremyhanna/Work/hive/working/build-common.xml:317: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
    [javac] Compiling 2 source files to /Users/jeremyhanna/Work/hive/working/build/cassandra-handler/test/classes
    [javac] /Users/jeremyhanna/Work/hive/working/cassandra-handler/src/test/org/apache/hadoop/hive/cassandra/CassandraQTestUtil.java:3: cannot find symbol
    [javac] symbol  : class QTestUtil
    [javac] location: package org.apache.hadoop.hive.ql
    [javac] import org.apache.hadoop.hive.ql.QTestUtil;
    [javac]                                 ^
    [javac] /Users/jeremyhanna/Work/hive/working/cassandra-handler/src/test/org/apache/hadoop/hive/cassandra/CassandraQTestUtil.java:8: cannot find symbol
    [javac] symbol: class QTestUtil

jeromatron

Thanks Maxim - I'll just go ahead and BCC you and Hentschel and move the discussion to the dev list.

Based on the comments on 1311 - did you have anything else to add to that - could we unify around 1016 or 1311 and work on getting that to a general state of acceptance?  Were there any that were able to do some work on either these days?  Or are we not at that point?

jeromatron

root@ip-10-202-81-155:~# ulimit -l
64
root@ip-10-202-81-155:~# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 20
file size               (blocks, -f) unlimited
pending signals                 (-i) 16382
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited

jeromatron

cassandra@ip-10-203-86-255:~$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 20
file size               (blocks, -f) unlimited
pending signals                 (-i) 16382
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8

jeromatron

1624 Jps -Dapplication.home=/usr/lib/jvm/java-6-sun-1.6.0.22 -Xms8m
1519  -Dlog4j.configuration=log4j-server.properties -XX:HeapDumpPath=/var/lib/cassandra/java_1297889929.hprof -XX:ErrorFile=/var/lib/cassandra/hs_err_1297889929.log -ea -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms3840M -Xmx3840M -Xmn200M -XX:+HeapDumpOnOutOfMemoryError -Xss128k -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.net.preferIPv4Stack=true -Dcom.sun.management.jmxremote.port=8080 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false abort

jeromatron

Error before Pig is launched
----------------------------
ERROR 2999: Unexpected internal error. Failed to create DataStorage

java.lang.RuntimeException: Failed to create DataStorage
        at org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:75)
        at org.apache.pig.backend.hadoop.datastorage.HDataStorage.<init>(HDataStorage.java:58)
        at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:213)
        at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:133)
        at org.apache.pig.impl.PigContext.connect(PigContext.java:183)

jeromatron

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
  <property>
    <name>mapred.job.tracker</name>
    <value>localhost:9001</value>
  </property>

jeromatron

***************
*** 18,52 ****
  
  import java.io.IOException;
  import java.nio.ByteBuffer;
- import java.util.*;
  
  import org.apache.cassandra.db.Column;
  import org.apache.cassandra.db.IColumn;
  import org.apache.cassandra.db.SuperColumn;

jeromatron

Index: src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordWriter.java
===================================================================
--- src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordWriter.java	(revision 1075592)
+++ src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordWriter.java	(revision )
@@ -325,7 +325,7 @@
                 }
 
                 Map<ByteBuffer, Map<String, List<Mutation>>> batch = new HashMap<ByteBuffer, Map<String, List<Mutation>>>();
-                while (batch.size() < batchThreshold)
+                while (mutation != null)