本文章は,Impala のメモ の TODO のうち,BackEnd の特にクエリの実行部を読み,文章化したものである.
BackEnd(BE) は単体のデーモンとして動作し,FE 側でパースした結果(Job ID/Plan)に基づいて処理を実行する.FE と BE のやりとりには Thrift が用いられている.
|FE| - パース結果(Job ID/実行Planなど) thrift -> |BE|
> ++ 2.10.0-M7 | |
Setting version to 2.10.0-M7 | |
[info] Set current project to msgpack-scala (in build file:/Users/ozawa/workspace/msgpack/msgpack-scala/) | |
> test | |
[info] Updating {file:/Users/ozawa/workspace/msgpack/msgpack-scala/}msgpack-scala... | |
[info] Resolving org.slf4j#slf4j-api;1.6.4 ... | |
[info] downloading http://repo1.maven.org/maven2/org/specs2/specs2_2.10.0-M7/1.12.1.1/specs2_2.10.0-M7-1.12.1.1.jar ... | |
[info] [SUCCESSFUL ] org.specs2#specs2_2.10.0-M7;1.12.1.1!specs2_2.10.0-M7.jar (17054ms) | |
[info] downloading http://repo1.maven.org/maven2/org/specs2/specs2-scalaz-core_2.10.0-M7/6.0.1/specs2-scalaz-core_2.10.0-M7-6.0.1.jar ... | |
[info] [SUCCESSFUL ] org.specs2#specs2-scalaz-core_2.10.0-M7;6.0.1!specs2-scalaz-core_2.10.0-M7.jar (18773ms) |
# Tried to work sample program(2. Use in normal way), but failed. http://wiki.msgpack.org/display/MSGPACK/QuickStart+for+Scala | |
# Compiler version : 2.9.1 | |
scala> import org.msgpack._ | |
import org.msgpack._ | |
scala> import org.msgpack.annotation.Message | |
import org.msgpack.annotation.Message | |
scala> import org.msgpack.ScalaMessagePack._ |
本文章は,Impala のメモ の TODO のうち,BackEnd の特にクエリの実行部を読み,文章化したものである.
BackEnd(BE) は単体のデーモンとして動作し,FE 側でパースした結果(Job ID/Plan)に基づいて処理を実行する.FE と BE のやりとりには Thrift が用いられている.
|FE| - パース結果(Job ID/実行Planなど) thrift -> |BE|
# import A | |
class A: | |
def say_hello(self): | |
print("hi!") | |
def say_goodbye(self): |
ozawa@ozamba:~/workspace/fluent/fluentd/com/lib/fluent$ ../../bin/fluentd -c ../../fluent.conf | |
2012-03-01 23:15:18 +0900: starting fluentd-0.10.13 | |
2012-03-01 23:15:18 +0900: reading config file path="../../fluent.conf" | |
2012-03-01 23:15:18 +0900: adding source type="forward" | |
2012-03-01 23:15:18 +0900: adding source type="http" | |
2012-03-01 23:15:18 +0900: adding source type="tail" | |
2012-03-01 23:15:18 +0900: sample | |
2012-03-01 23:15:18 +0900: /Users/ozawa/size_limit.rb | |
2012-03-01 23:15:18 +0900: adding match pattern="debug.**" type="stdout" | |
2012-03-01 23:15:18 +0900: listening fluent socket on 0.0.0.0:24224 |
require 'sinatra' | |
require 'dalli' | |
require 'json' | |
get '/count' do | |
res = '' | |
services = JSON.parse(ENV['VCAP_SERVICES']) | |
credentials = services["memcached-1.4.10"].first["credentials"] | |
host = credentials['host'] |
source "http://rubygems.org" | |
gem "datamapper" | |
gem "sinatra" | |
gem "dm-postgres-adapter" |
#!/usr/bin/env ruby | |
# The accompanying source code is provided under the MIT license. | |
# | |
# Copyright (c) 2011 OZAWA Tsuyoshi | |
# | |
# Permission is hereby granted, free of charge, to any person obtaining a copy | |
# of this software and associated documentation files (the "Software"), to | |
# deal in the Software without restriction, including without limitation the | |
# rights to use, copy, modify, merge, publish, distribute, sub-license, and/or | |
# sell copies of the Software, and to permit persons to whom the Software is |
~/build/q4m-0.9.5$ USE_C_CLIENT=1 MESSAGES=400000 CONCURRENCY=130 DBI='dbi:mysql:test;mysql_socket=/var/run/mysqld/mysqld.sock;user=root;password=root' t/05-multireader.t | |
1..4 | |
ok 1 - check number of messages | |
ok 2 - min value of received message | |
ok 3 - max value of received message | |
ok 4 - should have no rows in table | |
Multireader benchmark result: | |
Number of messages: 400000 |
# ./configure --with-sync=yes --with-mysql=../mysql-5.1.55 | |
# Intel(R) Xeon(R) CPU E5506 @ 2.13GHz | |
~/build/q4m-0.9.5$ USE_C_CLIENT=1 MESSAGES=400000 CONCURRENCY=200 DBI='dbi:mysql:test;mysql_socket=/var/run/mysqld/mysqld.sock;user=root;password=root' t/05-multireader.t | |
1..4 | |
ok 1 - check number of messages | |
ok 2 - min value of received message | |
ok 3 - max value of received message | |
ok 4 - should have no rows in table |