Skip to content

Instantly share code, notes, and snippets.

View LegolasVzla's full-sized avatar
🤓
Focusing

Manuel Carrero LegolasVzla

🤓
Focusing
26 followers · 40 following
View GitHub Profile
@bobquest33
bobquest33 / script_32_load_data.py
Last active March 9, 2020 05:07
The below script helps to load the data to a database using Pyspark. I used the following command to load the below data and it created a new table with appropriate data types in Postgres. This a very good feature I liked of PySpark data frames.
import os
import sys
from pyspark import SparkContext
from pyspark import SparkConf
from pyspark.sql import SQLContext
from pyspark.sql import SparkSession
from pyspark.sql import DataFrameReader
conf = SparkConf().setAppName('Simple App')
sc = SparkContext("local", "Simple App")
@gunesmes
gunesmes / Dockerfile
Last active September 4, 2023 00:03
Run Locust in Dockers for performance testing.
FROM python:alpine3.7
RUN apk -U add ca-certificates python python-dev py-pip build-base && \
pip install --upgrade pip && \
pip install locustio pyzmq && \
apk del python-dev && \
rm -r /var/cache/apk/* && \
mkdir /locust
WORKDIR /locust
@dinhchi27
dinhchi27 / Key Sublime Text 3.2.1 Build 3207 - Sublime Text 3 License Key
Last active July 2, 2025 05:58
Key Sublime Text 3.2.1 Build 3207 - Sublime Text 3 License Key
Key Sublime Text 3.2.1 Build 3207
----- BEGIN LICENSE -----
Member J2TeaM
Single User License
EA7E-1011316
D7DA350E 1B8B0760 972F8B60 F3E64036
B9B4E234 F356F38F 0AD1E3B7 0E9C5FAD
FA0A2ABE 25F65BD8 D51458E5 3923CE80
87428428 79079A01 AA69F319 A1AF29A4
A684C2DC 0B1583D4 19CBD290 217618CD
@zaloogarcia
zaloogarcia / pandas_to_spark.py
Last active April 19, 2022 16:20
Script for converting Pandas DF to Spark's DF
from pyspark.sql.types import *
# Auxiliar functions
# Pandas Types -> Sparks Types
def equivalent_type(f):
if f == 'datetime64[ns]': return DateType()
elif f == 'int64': return LongType()
elif f == 'int32': return IntegerType()
elif f == 'float64': return FloatType()
else: return StringType()