AKB428’s gists

AKB428 / gist:b45ad561ae4d2a9bdd9c

Last active August 29, 2015 14:15

rabbit_swift OK 5G over down

	[root@10-134-25-12 rabbit_swift]# vim lib/rabbit_swift/client.rb
	[root@10-134-25-12 rabbit_swift]# bundle exec ruby -I./lib bin/get_object.rb -t /nico_archive/anime_c123_2015_02_19_SPLIT1G.zip -d ./private/ -c ../Chino/conf/conf.json
	Content-Length = 7548785363
	Etag = "3772025e360d2d99fb813d151ba7d875"
	Accept-Ranges = bytes
	Last-Modified = Sat, 21 Feb 2015 09:23:37 GMT
	X-Object-Manifest = nico_archive/anime_c123_2015_02_19_SPLIT1G.zip_
	X-Timestamp = 1424510616.48291
	X-Static-Large-Object = true
	Content-Type = application/json

AKB428 / memory_error.log

Created February 21, 2015 10:03

rubyで大きいファイルをHTTPダウンロードする時の注意点

	[root@10-134-25-12 rabbit_swift]# bundle exec ruby -I./lib bin/get_object.rb -t /nico_archive/anime_c123_2015_02_19PLIT1G.zip -d ./private/ -c ../Chino/conf/conf.json
	Content-Length = 7548785363
	Etag = "3772025e360d2d99fb813d151ba7d875"
	Accept-Ranges = bytes
	Last-Modified = Sat, 21 Feb 2015 09:23:37 GMT
	X-Object-Manifest = nico_archive/anime_c123_2015_02_19_SPLIT1G.zip_
	X-Timestamp = 1424510616.48291
	X-Static-Large-Object = true
	Content-Type = application/json
	X-Trans-Id = tx17d36543427f47ac915f3-0054e85060

AKB428 / gist:f224cc4021800f4016f9

Created February 21, 2015 10:04

largefile check md5sum

	[root@10-134-25-12 private]# md5sum anime_c123_2015_02_19_SPLIT1G.zip
	29ad30cffdee71e84dc53f794a04ebb3 anime_c123_2015_02_19_SPLIT1G.zip
	[root@10-134-25-12 private]# md5sum ../
	.git/ Gemfile.lock RELEASE.md lib/
	.gitignore LICENSE.txt Rakefile private/
	Gemfile README.md bin/ rabbit_swift.gemspec
	[root@10-134-25-12 private]# md5sum ../../
	0.zip Nico/ maki/ nico_tools/
	2ch_thread_alert_system/ SutemaBurstStream/ n2/ rabbit_swift/
	Chino/ kodachi/ n3/

AKB428 / client1.rb

Created February 22, 2015 05:15

Ruby HTTPClientでギガクラスの大きいファイルをHTTPダウンロードする時の注意点 ref: http://qiita.com/AKB428/items/a0a7c269e0e910dc18d4

	http_client = HTTPClient.new
	open(dest_file, 'wb') do \|file\|
	file.write http_client.get_content(URI.parse(URI.encode(url)), query, auth_header)
	end

AKB428 / log.log

Created February 22, 2015 14:37

ForBench1

"C:\Program Files\Java\jdk1.8.0_25\bin\java" -Didea.launcher.port=7534 "-Didea.launcher.bin.path=N:\Program Files (x86)\JetBrains\IntelliJ IDEA 14.0.3\bin" -Dfile.encoding=UTF-8 -classpath "C:\Program Files\Java\jdk1.8.0_25\jre\lib\charsets.jar;C:\Program Files\Java\jdk1.8.0_25\jre\lib\deploy.jar;C:\Program Files\Java\jdk1.8.0_25\jre\lib\javaws.jar;C:\Program Files\Java\jdk1.8.0_25\jre\lib\jce.jar;C:\Program Files\Java\jdk1.8.0_25\jre\lib\jfr.jar;C:\Program Files\Java\jdk1.8.0_25\jre\lib\jfxswt.jar;C:\Program Files\Java\jdk1.8.0_25\jre\lib\jsse.jar;C:\Program Files\Java\jdk1.8.0_25\jre\lib\management-agent.jar;C:\Program Files\Java\jdk1.8.0_25\jre\lib\plugin.jar;C:\Program Files\Java\jdk1.8.0_25\jre\lib\resources.jar;C:\Program Files\Java\jdk1.8.0_25\jre\lib\rt.jar;C:\Program Files\Java\jdk1.8.0_25\jre\lib\ext\access-bridge-64.jar;C:\Program Files\Java\jdk1.8.0_25\jre\lib\ext\cldrdata.jar;C:\Program Files\Java\jdk1.8.0_25\jre\lib\ext\dnsns.jar;C:\Program Files\Java\jdk1.8.0_25\jre\lib\ext\jaccess.jar;C:\Progr

AKB428 / file1.txt

Last active August 29, 2015 14:15

ConoHaオブジェクトストレージで5GBを超えたファイルをアップロードする ref: http://qiita.com/AKB428/items/329d4baf4528cf5b10db

	bundle exec ruby -I./lib bin/slo_client.rb -s ~/Downloads/test.zip -d /test -c ../chino/conf/conf.json -l 100MB
	/Users/Siori/Downloads/test.zip_0001
	/Users/Siori/Downloads/test.zip_0002
	[{"path":"https://objectstore-r1nd1001.cnode.jp/v1/93a6500c0a1e4c68b976e5e46527145c/test/test.zip_0001","etag":"ab30f25f120fead24c4eaea63f5fe096","size_bytes":104857600},{"path":"https://objectstore-r1nd1001.cnode.jp/v1/93a6500c0a1e4c68b976e5e46527145c/test/test.zip_0002","etag":"81160d082d081cb92fae1047f37f274b","size_bytes":77011881}]
	upload_url -> https://objectstore-r1nd1001.cnode.jp/v1/93a6500c0a1e4c68b976e5e46527145c/test/test.zip_0001
	/Users/Siori/Downloads/test.zip_0001
	http_status -> 201
	upload OK
	upload_url -> https://objectstore-r1nd1001.cnode.jp/v1/93a6500c0a1e4c68b976e5e46527145c/test/test.zip_0002
	/Users/Siori/Downloads/test.zip_0002

AKB428 / file1.txt

Last active August 29, 2015 14:16

RabbitMQのインストールと管理画面の有効化(MacOSX/Linux/Windows) ref: http://qiita.com/AKB428/items/a2662fbb624ce7659025

	cd $RABBITMQ_HOME/etc/rabbitmq/
	sudo touch rabbitmq.config
	vi rabbitmq.config

AKB428 / get_monthly_title_list.rb

Created March 5, 2015 14:29

get_monthly_title_list.rb [v1.0]

	require 'nokogiri'
	require 'open-uri'
	require 'uri'

	#引数サンプル
	#'http://www.getchu.com/all/month_title.html?genre=pc_soft&gage=&year=2015&month=02'

	target = ARGV[0]

	doc = nil

AKB428 / file0.txt

Last active August 29, 2015 14:16

CentOS7のOpenJDK7をOracleJDK8に変更して、javaのパスを通す ref: http://qiita.com/AKB428/items/efe22d2a5c5213b9bff1

	$java -version
	$java version "1.7.0_XX"

AKB428 / scan.rb

Last active August 29, 2015 14:17

jruby hbase scan

	include Java

	#shell>hbase org.jruby.Main jruby_scan.rb
	#ref http://www.bigdataeverywhere.com/sites/default/files/BigData2014-JayeshThakrar-UnleashthePowerofHBaseShel.pdf
	import org.apache.hadoop.hbase.HBaseConfiguration
	import org.apache.hadoop.hbase.client.Put
	import org.apache.hadoop.hbase.client.Get
	import org.apache.hadoop.hbase.client.Scan
	import org.apache.hadoop.hbase.client.HTable
	import org.apache.hadoop.hbase.util.Bytes

AKB428 AKB428