Vishal Belsare vishalbelsare

UPDATE: I have baked the ideas in this file inside a Python CLI tool called pyds-cli. Please find it here: https://github.com/ericmjl/pyds-cli

How to organize your Python data science project

Having done a number of data projects over the years, and having seen a number of them up on GitHub, I've come to see that there's a wide range in terms of how "readable" a project is. I'd like to share some practices that I have come to adopt in my projects, which I hope will bring some organization to your projects.

Disclaimer: I'm hoping nobody takes this to be "the definitive guide" to organizing a data project; rather, I hope you, the reader, find useful tips that you can adapt to your own projects.

Disclaimer 2: What I’m writing below is primarily geared towards Python language users. Some ideas may be transferable to other languages; others may not be so. Please feel free to remix whatever you see here!

Google Public NTP [AS15169]:

time.google.com

time1.google.com

time2.google.com

time3.google.com

	import numpy as np
	import pandas as pd
	from datetime import datetime
	from fastparquet import write



	def compute_vwap(df):
	q = df['foreignNotional']
	p = df['price']

	"""
	A script to automatically export bookmarks from Firefox's SQLite database.

	There does not seem to be a programmatic way to get Firefox to export its bookmarks in
	the conventional HTML format. However, you can access the bookmark information directly
	in Firefox's internal database, which is what this script does.

	Always be careful when working with the internal database! If you delete data, you will
	likely not be able to recover it.

	"""
	Python code for fractional differencing of pandas time series
	illustrating the concepts of the article "Preserving Memory in Stationary Time Series"
	by Simon Kuttruf

	While this code is dedicated to the public domain for use without permission, the author disclaims any liability in connection with the use of this code.
	"""

	import numpy as np
	import pandas as pd

	#!/usr/bin/env python
	# -- coding: utf-8 --

	import os
	import subprocess
	import platform
	from libqtile.config import Key, Screen, Group, Drag, Click
	from libqtile.command import lazy
	from libqtile import layout, bar, widget, hook
	import custom

	#!/usr/bin/env bash
	#
	# Author: Markus (MawKKe) [email protected]
	# Date: 2018-03-19
	#
	#
	# What?
	#
	# Linux dm-crypt + dm-integrity + dm-raid (RAID1)
	#

	# basic factor analysis
	# http://blog.alphaarchitect.com/2015/05/28/basic-factor-analysis-simple-tools-to-understand-what-drives-performance/

	import pandas as pd
	import pandas.io.data as web
	import datetime, re, copy
	import numpy as np
	import statsmodels.formula.api as sm

	start = datetime.date(2000,1,1)