Pierdomenico Fiadino pierdom
pierdom
/ dataframe_to_s3.py
Created
September 20, 2017 09:18
[Export Pandas dataframe to compressed CSV on Amazon S3] #pandas #python #bigdata
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import pandas | |
| import io | |
| import gzip | |
| import boto3 | |
| csv_buffer = io.StringIO() | |
| my_df.to_csv(csv_buffer, index=False) | |
| csv_buffer.seek(0) | |
| gz_buffer = io.BytesIO() |
pierdom
/ pandas_visualization_options.py
Created
September 20, 2017 14:39
[Pandas visualization settings] For example: number of columns and rows showed on terminals, iPython, Jupyter, etc. #python #pandas #jupyter
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import pandas as pd | |
| pd.options.display.max_columns = 40 | |
| pd.options.display.max_rows = 999 | |
| #details here: https://pandas.pydata.org/pandas-docs/stable/options.html |
pierdom
/ annotate.py
Created
September 28, 2017 08:10
[Matplotlib annotate with arrow] #matplotlib #python #visualization
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ax.annotate(my_text, xy=(arrow_x,arrow_y), xytext=(text_x,text_y), | |
| arrowprops=dict(facecolor='gray', shrink=0.05)) |
pierdom
/ histogram_frequency.py
Created
September 28, 2017 09:59
[Matplotlib hystograms with frequencies] #matplotlib #python #visualization #statistics #datascience
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import numpy as np | |
| import matplotlib.pyplot as plt | |
| # my empirical distribution (it could be a numpy array) | |
| mydistr = [...] | |
| # calculate histogram weights | |
| weights = np.ones_like(distr)/float(len(distr)) | |
| # custom binning (just an example with 0.1) |
pierdom
/ bokeh_and_jupyter_controls.ipynb
Created
November 17, 2017 11:32
[Plot distribution in Bokeh with Jupyter widgets] #statistics #datascience #python #visualization #bokeh
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
pierdom
/ pandas_categorical.py
Created
November 21, 2017 13:50
[Pandas and categorical data] #pandas #datascience
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import calendar | |
| import pandas as pd | |
| [...] | |
| df["day_of_the_week"] = pd.Categorical(df["day_of_the_week"], list(calendar.day_abbr)) |
pierdom
/ holoviews_rotate_axis.py
Created
November 27, 2017 14:07
[Axis options in Holoviews/Bokeh plots] includes: tick labels rotation, logarithmic scale and other ticks options #python #visualization #holoviews #bokeh #datascience
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| hm = hv.HeatMap(((x_names, y_names, od_pivot))) | |
| hm = hm.opts(plot={"xrotation": 90}) | |
| #Other axis options | |
| # 'logy': True | |
| # 'yaxis': None | |
| # 'xrotation': 90} | |
| # 'xticks': 3 | |
| # 'xticks': [0, 100, 300, 500] |
pierdom
/ holoviews_dynamicmap_xrange.py
Created
November 27, 2017 14:56
[Holoviews DynamicMap to change dynamically x-axis range] #holoviews #bokeh #visualization #python #datascience
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import holoviews as hv | |
| import numpy as np | |
| hv.extension('bokeh') | |
| hv.DynamicMap(lambda i: hv.Curve(np.arange(i)), kdims=['i']).redim.range(i=((10, 20))).opts(norm=dict(framewise=True)) |
pierdom
/ holoviews_invert_axes.py
Created
November 27, 2017 14:58
[Invert X and Y axis in Holoviews] #holoviews #visualization #python
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| %%opts Histogram [invert_axes=True] | |
| # note that this is different from invert_xaxis, which reverses an axes | |
| # invert_axes, instead, will swap X and Y |
pierdom
/ marginal_spikes.py
Created
November 28, 2017 08:00
[Marginal spikes plot in Holoview scatter plot] #visualization #python #holoviews #bokeh #datascience
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| points = points << hv.Spikes(points['y']) << hv.Spikes(points['x']) |