http://www.slideshare.net/frsyuki/hadoop-source-code-reading-15-in-japan-presto
Wataru Yukawa wyukawa
wyukawa
/ odbc32.png
Last active
August 29, 2015 14:02
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| java.lang.NoClassDefFoundError: org/apache/avro/Schema | |
| at org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils.determineSchemaOrThrowException(AvroSerdeUtils.java:61) ~[na:na] | |
| at org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils.determineSchemaOrReturnErrorSchema(AvroSerdeUtils.java:87) ~[na:na] | |
| at org.apache.hadoop.hive.serde2.avro.AvroSerDe.initialize(AvroSerDe.java:60) ~[na:na] | |
| at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:178) ~[na:na] | |
| at com.facebook.presto.hive.HiveUtil.getTableObjectInspector(HiveUtil.java:148) ~[na:na] | |
| at com.facebook.presto.hive.HiveUtil.getTableStructFields(HiveUtil.java:156) ~[na:na] | |
| at com.facebook.presto.hive.HiveClient.getColumnHandles(HiveClient.java:300) ~[na:na] | |
| at com.facebook.presto.hive.HiveClient.getTableMetadata(HiveClient.java:235) ~[na:na] | |
| at com.facebook.presto.hive.HiveClient.listTableColumns(HiveClient.java:332) ~[na:na] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 2014-07-29 15:58:15,994 FATAL [main] org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.OutOfMemoryError: GC overhead limit exceeded | |
| at com.mysql.jdbc.MysqlIO.nextRowFast(MysqlIO.java:1649) | |
| at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1426) | |
| at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:2924) | |
| at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:477) | |
| at com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:2619) | |
| at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:1788) | |
| at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2209) | |
| at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2625) | |
| at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2119) |
wyukawa
/ gist:248745b088fa3b60a26e
Created
September 11, 2014 07:52
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import sys | |
| import requests | |
| import os.path | |
| from optparse import OptionParser | |
| parser = OptionParser() | |
| parser.add_option("-u", "--url", dest="url", help="azkaban url") | |
| parser.add_option("-s", "--username", dest="username", help="azkaban username") | |
| parser.add_option("-p", "--password", dest="password", help="azkaban password") | |
| parser.add_option("-r", "--project", dest="project", help="azkaban project") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| java.lang.NumberFormatException: For input string: "" | |
| at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) ~[na:1.7.0_60] | |
| at java.lang.Integer.parseInt(Integer.java:504) ~[na:1.7.0_60] | |
| at java.lang.Integer.parseInt(Integer.java:527) ~[na:1.7.0_60] | |
| at org.apache.hadoop.hive.serde2.ColumnProjectionUtils.getReadColumnIDs(ColumnProjectionUtils.java:116) ~[na:na] | |
| at org.apache.hadoop.hive.ql.io.RCFile$Reader.<init>(RCFile.java:1383) ~[na:na] | |
| at org.apache.hadoop.hive.ql.io.RCFile$Reader.<init>(RCFile.java:1342) ~[na:na] | |
| at com.facebook.presto.hive.rcfile.RcFilePageSourceFactory.createPageSource(RcFilePageSourceFactory.java:124) ~[na:na] | |
| at com.facebook.presto.hive.HivePageSourceProvider.createPageSource(HivePageSourceProvider.java:92) ~[na:na] | |
| at com.facebook.presto.spi.classloader.ClassLoaderSafeConnectorPageSourceProvider.createPageSource(ClassLoaderSafeConnectorPageSourceProvider.java:42) ~[presto-spi-0.80.jar:0. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import java.sql.Connection; | |
| import java.sql.DatabaseMetaData; | |
| import java.sql.Driver; | |
| import java.sql.DriverManager; | |
| import java.util.Properties; | |
| public class Main { | |
| public static void main(String[] args) throws Throwable { | |
wyukawa
/ gist:c320b6c12189a2df2166
Last active
August 29, 2015 14:10
prestogres 5.5 + postgresql 9.2 + cognos 10.2.2(32bit)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ sudo prestogres-ctl pgpool | |
| /usr/local/bin/prestogres-pgpool -n | |
| ... | |
| 2014-12-04 14:37:34: pid 106236: LOG: perhaps failed to create INET domain socket | |
| 2014-12-04 14:37:34: pid 106236: DETAIL: socket(::) failed: "Address family not supported by protocol" | |
| 2014-12-04 14:37:34: pid 106236: LOG: prestogres successfully started. version 0.5.5 (tataraboshi) | |
| 2014-12-04 14:53:47: pid 106262: LOG: statement: set datestyle = 'ISO'; select version(), case when pg_encoding_to_char(1) = 'SQL_ASCII' then 'UNKNOWN' else getdatabaseencoding() end | |
| 2014-12-04 14:53:49: pid 106262: LOG: statement: begin; set autocommit = on; set client_encoding = 'UTF8'; SET extra_float_digits=3; commit | |
| 2014-12-04 14:53:49: pid 106262: LOG: statement: select current_setting('standard_conforming_strings') | |
| 2014-12-04 14:53:49: pid 106262: LOG: statement: BEGIN;SELECT t.typlen FROM pg_catalog.pg_type t, pg_catalog.pg_namespace n WHERE t.typnamespace=n.oid AND t.typname='name' AND n.nspname='pg_catalog' |
wyukawa
/ gist:443b9a1227a5a606dda9
Created
December 4, 2014 08:31
prestogres 0.6 + cognos + postgresql 9.2
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 2014-12-04 17:29:22: pid 119207: LOG: statement: set datestyle = 'ISO'; select version(), case when pg_encoding_to_char(1) = 'SQL_ASCII' then 'UNKNOWN' else getdatabaseencoding() end | |
| 2014-12-04 17:29:23: pid 119207: LOG: statement: begin; set autocommit = on; set client_encoding = 'UTF8'; SET extra_float_digits=3; commit | |
| 2014-12-04 17:29:23: pid 119207: LOG: statement: select current_setting('standard_conforming_strings') | |
| 2014-12-04 17:29:24: pid 119217: LOG: statement: set datestyle = 'ISO'; select version(), case when pg_encoding_to_char(1) = 'SQL_ASCII' then 'UNKNOWN' else getdatabaseencoding() end | |
| 2014-12-04 17:29:25: pid 119217: LOG: statement: begin; set autocommit = on; set client_encoding = 'UTF8'; SET extra_float_digits=3; commit | |
| 2014-12-04 17:29:25: pid 119217: LOG: statement: select current_setting('standard_conforming_strings') | |
| 2014-12-04 17:29:25: pid 119217: LOG: statement: BEGIN;SELECT nspname AS TABLE_SCHEM , NULL AS TABLE_CATALOG FROM pg_catalog.pg_namespace WHERE nspname <> 'pg_toast |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 2014-12-05 13:57:59: pid 23122: DEBUG: I am 23122 accept fd 8 | |
| 2014-12-05 13:57:59: pid 23122: DEBUG: reading startup packet | |
| 2014-12-05 13:57:59: pid 23122: DETAIL: Protocol Major: 2 Minor: 0 database: hive user: SYSTEM | |
| 2014-12-05 13:57:59: pid 23122: DEBUG: prestogres_init_hba: presto_server: localhost:8080 | |
| 2014-12-05 13:57:59: pid 23122: DEBUG: prestogres_init_hba: presto_user: SYSTEM | |
| 2014-12-05 13:57:59: pid 23122: DEBUG: prestogres_init_hba: presto_catalog: hive | |
| 2014-12-05 13:57:59: pid 23122: DEBUG: prestogres_init_hba: presto_schema: default | |
| 2014-12-05 13:57:59: pid 23122: ERROR: error while initializing database pg_database = "hive" for pg_user = "SYSTEM": "ERROR: syntax error at or near "#" | |
| CONTEXT: Error occurred on dblink connection named "unnamed": could not execute command. | |
| PL/pgSQL function prestogres_init_database(text,text,text) line 7 at RETURN |