masayuki038

Emacs tips

gist

gist.el(Verion=1.3.1)をインストールしみたが、エラーでまともに動かなかった。そこでyagistも試してみたが、package.el経由でインストールした限りではgist-listから直接コンテンツを表示できなかったので、アンインストール。gist.elのVersion=20151128.1941をインストールし、動くことを確認。

package.el

以下のリポジトリを追加した。

masayuki038

package net.wrap_trap.goju_to.macros

import scala.language.experimental.macros
import scala.reflect.macros.blackbox

/**
  * goju-to: HanoiDB(LSM-trees (Log-Structured Merge Trees) Indexed Storage) clone

  * Copyright (c) 2016 Masayuki Takahashi

masayuki038

package net.wrap_trap.sample;

import java.io.*;
import java.nio.file.Files;

public class Main {
    public static void main(String[] args) throws IOException {
        if(args.length != 2) {
            System.err.println("remove-hardlink-test [source] [hardlink]");
            System.exit(-1);

masayuki038

package net.wrap_trap.parquet_arrow_samples;

import org.apache.arrow.memory.BufferAllocator;
import org.apache.arrow.memory.RootAllocator;
import org.apache.arrow.vector.VectorSchemaRoot;
import org.apache.arrow.vector.NullableIntVector;
import org.apache.arrow.vector.NullableBigIntVector;
import org.apache.arrow.vector.NullableVarCharVector;

import org.apache.parquet.io.api.Binary;

masayuki038

package net.wrap_trap.parquet_sample3;

import org.apache.commons.lang3.builder.ToStringBuilder;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.parquet.column.ColumnDescriptor;
import org.apache.parquet.column.page.*;
import org.apache.parquet.hadoop.ParquetFileReader;
import org.apache.parquet.hadoop.metadata.BlockMetaData;
import org.apache.parquet.hadoop.metadata.ColumnChunkMetaData;

masayuki038

package net.wrap_trap.parquet_sample3;

import org.apache.commons.lang3.builder.ToStringBuilder;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.parquet.column.ColumnDescriptor;
import org.apache.parquet.column.ColumnReader;
import org.apache.parquet.column.impl.ColumnReadStoreImpl;
import org.apache.parquet.column.impl.ColumnReaderImpl;
import org.apache.parquet.column.page.*;

masayuki038

before

cstore_test=# select product_group, count(*) from customer_reviews group by product_group;
 product_group |  count
---------------+---------
 DVD           |  121418
 Video         |  142235
 Music         |  300628
 Book          | 1198218

masayuki038

Test schema/data

• https://github.com/citusdata/cstore_fdw/blob/master/README.md#example

Tables(partitioned or not)

• customer_reviews: FOREIGN TABLE(not-partitioned)
• customer_reviews_cstore: FOREIGN TABLE(partitioned by product_group)

table_type

masayuki038

$ java -cp target/classes;target/dependency/* org.apache.parquet.tools.Main dump D:\tmp\delta\checkpoint2\_delta_log\00000000000000000002.checkpoint.parquet
row group 0
--------------------------------------------------------------------------------
txn:
.appId:                BINARY SNAPPY DO:0 FPO:4 SZ:30/28/0.93 VC:7 ENC [more]...
.version:              INT64 SNAPPY DO:0 FPO:34 SZ:29/27/0.93 VC:7 ENC [more]...
.lastUpdated:          INT64 SNAPPY DO:0 FPO:63 SZ:30/28/0.93 VC:7 ENC [more]...
add:
.path:                 BINARY SNAPPY DO:0 FPO:93 SZ:304/310/1.02 VC:7  [more]...
.partitionValues:

masayuki038

  @Test
  public void testExample() throws ClassNotFoundException, SQLException {
    Class.forName("org.apache.calcite.jdbc.Driver");
    try (Connection conn = DriverManager.getConnection("jdbc:calcite:model=target/test-classes/example-model.json", "foo", "bar")) {
      try (Statement stmt = conn.createStatement()) {
        stmt.executeQuery("select empno, gender, name from EMPS where name = 'John'");
      }
    }
  }