Skip to content

Instantly share code, notes, and snippets.

View tispratik's full-sized avatar

Pratik Khadloya tispratik

View GitHub Profile
@tispratik
tispratik / parquet_columns_reader2.java
Created August 28, 2014 19:43
Parquet columns reader2
package com.company.grid.lookup_new;
import parquet.column.ColumnDescriptor;
import parquet.column.ColumnReader;
import parquet.column.impl.ColumnReadStoreImpl;
import parquet.column.page.PageReadStore;
import parquet.hadoop.ParquetFileReader;
import parquet.hadoop.metadata.BlockMetaData;
import parquet.hadoop.metadata.ParquetMetadata;
import parquet.io.api.Binary;
@tispratik
tispratik / Hive 0.13 sqoop import as parquetfile
Last active August 29, 2015 14:06
Hive 0.13 sqoop import as parquetfile
bin/sqoop import -jt <jt> --connect <dburl> --username <user> --password <pwd> --query "SELECT ..... WHERE \$CONDITIONS" --num-mappers 1 --mapreduce-job-name test_sqoop_import --as-parquetfile --target-dir /user/pkhadloya/sqoop/test --delete-target-dir --hive-import --hive-table test --create-hive-table --verbose
Warning: /hell does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /home/pkhadloya/sqoop-7e14e38/bin/../../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
14/09/19 10:58:12 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-SNAPSHOT
14/09/19 10:58:12 DEBUG tool.BaseSqoopTool: Enabled debug logging.
14/09/19 10:58:12 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
14/09/19 10:58:12 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
14/09/19 10:58:12 INFO tool.BaseSqoopTool: delimi
@tispratik
tispratik / hive 0.13 parquet import through hcatalog
Last active February 8, 2018 07:28
hive 0.13, parquet import through hcatalog
sqoop-7e14e38$ bin/sqoop import -jt <jt> --connect jdbc:mysql://... --username user --password pwd --query "... WHERE \$CONDITIONS" --num-mappers 1 --hcatalog-storage-stanza "STORED AS PARQUET" --create-hcatalog-table --hcatalog-table abc116 --verbose
Warning: /hell does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /home/pkhadloya/sqoop-7e14e38/bin/../../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
14/09/19 11:33:09 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-SNAPSHOT
14/09/19 11:33:09 DEBUG tool.BaseSqoopTool: Enabled debug logging.
14/09/19 11:33:09 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
14/09/19 11:33:09 DEBUG sqoop.ConnFactory: Loaded manager factory: org.apache.sqoop.manager.oracle.OraOopManagerFactory
14/09/19 11:33:09 DEBUG sqoop.ConnFactory: Loaded manager factory: com.cloudera.sqoop.manager.De
@tispratik
tispratik / gist:aebff28b8c5afd7bee59
Created February 5, 2015 18:06
Testing argparse for exception
try:
input_args = []
arg_parse_utils.parse_args(arg_meta_list, input_args)
except SystemExit:
pass
defmodule Myapp.Domain do
import Plug.Conn
import Ecto.Model
@doc false
def init(opts), do: opts
@doc false
def call(conn, _opts) do
IO.inspect conn
@tispratik
tispratik / gist:02628943731d3e2eaf4e
Created January 9, 2016 21:41
Heroku deploy error
Counting objects: 125, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (119/119), done.
Writing objects: 100% (125/125), 125.92 KiB | 0 bytes/s, done.
Total 125 (delta 85), reused 0 (delta 0)
remote: Compressing source files... done.
remote: Building source:
remote:
remote: -----> Fetching set buildpack https://github.com/HashNuke/heroku-buildpack-elixir... done
remote: -----> elixir app detected
@tispratik
tispratik / cuda_check.py
Created February 14, 2020 20:58 — forked from f0k/cuda_check.py
Simple python script to obtain CUDA device information
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
Outputs some information on CUDA-enabled devices on your computer,
including current memory usage.
It's a port of https://gist.github.com/f0k/0d6431e3faa60bffc788f8b4daa029b1
from C to Python with ctypes, so it can run without compiling anything. Note
that this is a direct translation with no attempt to make the code Pythonic.
void automation.pushToFutwork(Int leadID, String leadPhone, String fName)
{
leadId = 274638000051159057;
leadPhone = "+918970926449";
fName = "Sujanadarshan";
pushUrl = "https://dial.futwork.com/leads/push?teleproject=6749a6b643f75800138885cc";
apiKey = zoho.crm.getOrgVariable("FutWork_API_Key");
//Create Header