Skip to content

Instantly share code, notes, and snippets.

View rubensayshi's full-sized avatar

Ruben de Vries rubensayshi

  • Amsterdam, Netherlands
View GitHub Profile
@rubensayshi
rubensayshi / hiveserver_run.log
Created July 3, 2012 14:28
Hive problem with mapjoin + hiveserver
What I think is wrong;
hive server is running this command:
/usr/lib/hadoop-0.20/bin/hadoop jar /opt/hive-0.8.1-bin/lib/hive-service-0.8.1.jar org.apache.hadoop.hive.ql.exec.ExecDriver
but that jar doesn't contain that class, while the manual run uses the hive-exec-0.8.1.jar, which does contain the jar
@rubensayshi
rubensayshi / test.php
Created July 4, 2012 14:49
Hive MAPJOIN + LATERAL VIEW
execute("SELECT
date_int,
gender,
part,
COUNT( DISTINCT member_id ),
SUM( partcount )
FROM (
SELECT
/*+ MAPJOIN(members_map) */
@rubensayshi
rubensayshi / failing_query.sql
Created July 4, 2012 15:22
Hive MAPJOIN + LATERAL VIEW - explains
SELECT
date_int,
gender,
part,
COUNT( DISTINCT member_id ),
SUM( partcount )
FROM (
SELECT
/*+ MAPJOIN(members_map) */
date_int,
mysql> SELECT listing_date, SUM(unit_price * quantity) / SUM(quantity) FROM (SELECT * FROM listing WHERE item_id = 1140 AND listing_date = '2012-07-22' ORDER BY id DESC) as listing GROUP BY listing_date;
+--------------+--------------------------------------------+
| listing_date | SUM(unit_price * quantity) / SUM(quantity) |
+--------------+--------------------------------------------+
| 2012-07-22 | 21.4561 |
+--------------+--------------------------------------------+
@rubensayshi
rubensayshi / dev.php
Created September 8, 2012 18:31
try this
<?php
use GW2Spidy\TradeMarket;
require dirname(__FILE__) . '/../config/config.inc.php';
require dirname(__FILE__) . '/../autoload.php';
$market = TradeMarket::getInstance();
var_dump($market->getMarketData());
@rubensayshi
rubensayshi / nginx.conf
Created September 18, 2012 11:58
Nginx Config
server {
listen 80;
server_name gw2spidy.rubensayshi.com gw2spidy.com;
rewrite ^(.*) http://www.gw2spidy.com$1 permanent;
}
server {
listen 80;
server_name www.gw2spidy.com;
@rubensayshi
rubensayshi / 0.intro
Created September 20, 2012 15:09
FML I'm losing it
Somehow this query is 5seconds the first time I run it and 0.000smtsmall seconds the 2nd time, even if do SELECT SQL_NO_CACHE
I already tried optimizing / repairing the tables, even completely dropped and recreated the database ...
@rubensayshi
rubensayshi / gist:3762064
Created September 21, 2012 15:09
gw2spidy feature todo list
- crafting recipes
- cost / profit
- proper public API
- JSON and CSV
- updated since last visit
- candlestick charts
- volume in item chart
<?php
$json = array();
foreach ($dbResults as $row) {
$date = new DateTime("{$row['datefield']}");
$date->setTimezone(new DateTimeZone('UTC'));
$timestamp = $date->getTimestamp();
$json[$timestamp] = $row['value'];
}
@rubensayshi
rubensayshi / config
Created October 12, 2012 14:08
git push to prod
[receive]
denyCurrentBranch = ignore