- This document is created for Presto Source Code Reading #1.
- Target: trunk code, Feb 10th, 2014
- Main Topic: Coordinator and Executor ./presto-main/src/main/java/com/facebook/presto/executions
- http://www.slideshare.net/frsyuki/hadoop-source-code-reading-15-in-japan-presto/10
- NOTE
- All source code noted here is quoted from https://github.com/facebook/presto. License bases on https://github.com/facebook/presto/blob/master/LICENSE.
- This document can include something wrong. Pull requests to fix them are always welcome :-)
OZAWA Tsuyoshi oza
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [INFO] Scanning for projects... | |
| [WARNING] | |
| [WARNING] Some problems were encountered while building the effective model for org.apache.tez:tez-api:jar:0.5.0-incubating-SNAPSHOT | |
| [WARNING] 'build.pluginManagement.plugins.plugin.(groupId:artifactId)' must be unique but found duplicate declaration of plugin org.codehaus.mojo:findbugs-maven-plugin @ org.apache.tez:tez:0.5.0-incubating-SNAPSHOT, /Users/ozawa/workspace/tez/pom.xml, line 496, column 17 | |
| [WARNING] | |
| [WARNING] Some problems were encountered while building the effective model for org.apache.tez:tez-common:jar:0.5.0-incubating-SNAPSHOT | |
| [WARNING] 'build.pluginManagement.plugins.plugin.(groupId:artifactId)' must be unique but found duplicate declaration of plugin org.codehaus.mojo:findbugs-maven-plugin @ org.apache.tez:tez:0.5.0-incubating-SNAPSHOT, /Users/ozawa/workspace/tez/pom.xml, line 496, column 17 | |
| [WARNING] | |
| [WARNING] Some problems were encountered while building the effective model for org.apache.tez:tez-runtime-library:jar:0.5.0-incubating-SNAPSHOT | |
| [ |
oza
/ hadoop-yarn-project-test.log
Last active
August 29, 2015 13:58
A result of "mvn test" with hadoop-2.4.0-rc0 source code.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Running org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell | |
| Tests run: 8, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 219.414 sec <<< FAILURE! - in org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell | |
| testDSShell(org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell) Time elapsed: 30.716 sec <<< FAILURE! | |
| junit.framework.AssertionFailedError: null | |
| at junit.framework.Assert.fail(Assert.java:48) | |
| at junit.framework.Assert.assertTrue(Ass |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| private DAGProtos.DAGPlan getDAGPlan() throws IOException { | |
| String stagingDir = conf.get(TezConfiguration.TEZ_AM_STAGING_DIR, | |
| TezConfiguration.TEZ_AM_STAGING_DIR_DEFAULT); | |
| Path binaryPlan = new Path( | |
| stagingDir, TezConfiguration.TEZ_PB_PLAN_BINARY_NAME); | |
| FileInputStream dagPBBinaryStream = null; | |
| DAGProtos.DAGPlan dagPlan = null; | |
| try { | |
| // Read the protobuf DAG | |
| dagPBBinaryStream = new FileInputStream(binaryPlan.getName()); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| private DAGProtos.DAGPlan getDAGPlan() throws IOException { | |
| String stagingDir = conf.get(TezConfiguration.TEZ_AM_STAGING_DIR, | |
| TezConfiguration.TEZ_AM_STAGING_DIR_DEFAULT); | |
| Path binaryPlan = new Path( | |
| stagingDir, TezConfiguration.TEZ_PB_PLAN_BINARY_NAME); | |
| FileInputStream dagPBBinaryStream = null; | |
| DAGProtos.DAGPlan dagPlan = null; | |
| try { | |
| // Read the protobuf DAG | |
| dagPBBinaryStream = new FileInputStream(binaryPlan.getName()); |
oza
/ presto_executor_and_coordinator.md
Last active
December 10, 2019 22:43
oza
/ TestNamenodeRetryCacheMetrics.java
Created
January 21, 2014 09:03
TestNamenodeRetryCacheMetrics
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| /** | |
| * Licensed to the Apache Software Foundation (ASF) under one | |
| * or more contributor license agreements. See the NOTICE file | |
| * distributed with this work for additional information | |
| * regarding copyright ownership. The ASF licenses this file | |
| * to you under the Apache License, Version 2.0 (the | |
| * "License"); you may not use this file except in compliance | |
| * with the License. You may obtain a copy of the License at | |
| * | |
| * http://www.apache.org/licenses/LICENSE-2.0 |
- Generic Resource Manager
- http://www.slideshare.net/ozax86/ntt-meets-hadoop-and-cloudera-world-tokyo#17
- ResourceManager/NodeManager
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ubuntu@localhost:~/tez$ mvn clean install | |
| [INFO] Scanning for projects... | |
| [WARNING] | |
| [WARNING] Some problems were encountered while building the effective model for org.apache.tez:tez-docs:pom:0.2.0 | |
| [WARNING] 'reporting.plugins.plugin.version' for org.apache.maven.plugins:maven-project-info-reports-plugin is missing. @ line 195, column 17 | |
| [WARNING] | |
| [WARNING] It is highly recommended to fix these problems because they threaten the stability of your build. | |
| [WARNING] | |
| [WARNING] For this reason, future Maven versions might no longer support building such malformed projects. | |
| [WARNING] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <?xml version="1.0" encoding="UTF-8"?> | |
| <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> | |
| <!-- | |
| Licensed under the Apache License, Version 2.0 (the "License"); | |
| you may not use this file except in compliance with the License. | |
| You may obtain a copy of the License at | |
| http://www.apache.org/licenses/LICENSE-2.0 | |
| Unless required by applicable law or agreed to in writing, software |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <?xml version="1.0" encoding="UTF-8"?> | |
| <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> | |
| <!-- | |
| Licensed under the Apache License, Version 2.0 (the "License"); | |
| you may not use this file except in compliance with the License. | |
| You may obtain a copy of the License at | |
| http://www.apache.org/licenses/LICENSE-2.0 | |
| Unless required by applicable law or agreed to in writing, software |