Skip to content

Instantly share code, notes, and snippets.

View zhu3pang's full-sized avatar
🎯
Focusing

Zhu Jing zhu3pang

🎯
Focusing
View GitHub Profile
@zhu3pang
zhu3pang / lag-lead.sql
Last active June 19, 2021 08:13
window function 「lag」and 「lead」in sql
select
id,
if(id%2=0,
lag(student, 1) over(order by id), -- 如果id是偶数,取上一行「lag」
lead(student, 1, student) over(order by id) -- 如果id是奇数,取下一行,没有下一行时取自身「lead(field, 1, default)」
) as student
from
seat
@zhu3pang
zhu3pang / spark-empty-df.scala
Last active June 19, 2021 08:12
create empty dataframe in scala and java
import org.apache.spark.sql.types.{
StructType, StructField, StringType, IntegerType}
import org.apache.spark.sql.Row
//in scala
val schema = StructType(
StructField("k", StringType, true) ::
StructField("v", IntegerType, false) :: Nil)
// Spark < 2.0
@zhu3pang
zhu3pang / Symantec.md
Last active July 24, 2021 02:18 — forked from steve-jansen/README.md
Stop and start Symantec Endpoint Protection on OS X

This script enables you stop and start Symantec Endpoint Protection on OS X

Installation

sudo curl https://gist.githubusercontent.com/steve-jansen/61a189b6ab961a517f68/raw/sep -o /usr/local/bin/sep
sudo chmod 755 /usr/local/bin/sep
sudo chown root:staff /usr/local/bin/sep
@zhu3pang
zhu3pang / gist:917656dba0b034bfe9113d6f5791d931
Created June 26, 2021 09:15
spark hash partition vs range partiton
https://stackoverflow.com/a/47249212/13545776
@zhu3pang
zhu3pang / spark列转行.py
Last active July 3, 2021 04:54
spark sql 列转行
df = spark.createDataFrame([
{'id': 1, u'姓名': u'张三', u'分数': 88, u'科目': u'数学'},
{'id': 2, u'姓名': u'李雷', u'分数': 67, u'科目': u'数学'},
{'id': 3, u'姓名': u'宫九', u'分数': 77, u'科目': u'数学'},
{'id': 4, u'姓名': u'王五', u'分数': 65, u'科目': u'数学'},
{'id': 1, u'姓名': u'张三', u'分数': 77, u'科目': u'英语'},
{'id': 2, u'姓名': u'李雷', u'分数': 90, u'科目': u'英语'},
{'id': 3, u'姓名': u'宫九', u'分数': 24, u'科目': u'英语'},
{'id': 4, u'姓名': u'王五', u'分数': 90, u'科目': u'英语'},
{'id': 1, u'姓名': u'张三', u'分数': 33, u'科目': u'语文'},
@zhu3pang
zhu3pang / clickhouse_build_centos.sh
Last active December 3, 2022 08:28 — forked from filimonov/clickhouse_build_centos.sh
Clickhouse build on Centos 6/ Centos 7 / RedHat
#! /bin/bash
# Tested on Centos 6.9 and Centos 7.4. ClickHouse vesion 1.1.54236
# Official build instructions:
# https://github.com/yandex/ClickHouse/blob/master/doc/build.md
# See also https://github.com/redsoftbiz/clickhouse-rpm
# and https://github.com/Altinity/clickhouse-rpm
export THREADS=$(grep -c ^processor /proc/cpuinfo)
@zhu3pang
zhu3pang / install-llvm.sh
Created December 3, 2022 22:25 — forked from sixman9/install-llvm.sh
Bash script to download and build LLVM and Clang
#!/bin/bash
# install-llvm.sh
# Copyright (c) 2010 Ben Karel. All rights reserved.
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE.txt file or at http://eschew.org/txt/bsd.txt
# This file was found at http://foster.googlecode.com/hg/scripts and updated by Richard Joseph, Nov 2012
#The current LLVM and CLANG version can be found at http://llvm.org/releases/download.html
@zhu3pang
zhu3pang / Upgrade vim
Last active September 25, 2023 05:26 — forked from yevrah/Upgrade vim
Update to Vim8 on Centos 7
################################################################################
# Method 1: Install using rpm packages (credit to DarkMukke)
#
curl -L https://copr.fedorainfracloud.org/coprs/hnakamur/vim/repo/epel-7/hnakamur-vim-epel-7.repo -o /etc/yum.repos.d/hnakamur-vim-epel-7.repo
yum update -y && yum update vim* -y
#rpm -Uvh http://mirror.ghettoforge.org/distributions/gf/gf-release-latest.gf.el7.noarch.rpm
#rpm --import http://mirror.ghettoforge.org/distributions/gf/RPM-GPG-KEY-gf.el7
# WARNING: removing vim-minimal uninstalls `sudo` if you skip the second step
# make sure to at least run `yum install sudo`
@zhu3pang
zhu3pang / rsync.sh
Created September 26, 2023 22:14 — forked from garethrees/rsync.sh
rsync & scp through jump host
# Upload
rsync -av -e "ssh -A JUMP_HOST ssh" FILE_TO_SEND DEST_HOST:/home/gareth/
# Download
rsync -av -e "ssh -A JUMP_HOST ssh" DEST_HOST:~/FILE_TO_DOWNLOAD ~/Downloads/
@zhu3pang
zhu3pang / [email protected]
Created March 27, 2024 04:01 — forked from gmag11/[email protected]
Mount multiple RClone remotes on boot with a single SystemD forking service definition
# Rclone mount on boot
# Copy file to: /etc/systemd/system
# You need to create a remote on RClone and a folder on your disk, both with same name <rclone-remote>
# This example uses /cloud/ folder as origin to mount all remotes, change it to your needs
# This example use a linux user named rclone. Create it or adapt it to your needs. Rclone will get config from that user's home folder
# Register new service by typing:
# sudo systemctl daemon-reload
# Do the next one for every remote you want to load on boot
# sudo systemctl enable rclone-mount@<rclone-remote>.service
# systemctl start rclone-mount@<rclone-remote>.service