Jeff Hammerbacher hammer

Default

Powerlevel10k

Scala effect systems

Topological data analysis (TDA) in R

Purpose

Topological data analysis (or TDA) is a growing discipline with great potential. Unfortunately, the difficulty of learning the software used to perform TDA on real datasets is a nontrivial task adds an oft intimidating barrier to entry. This Gist provides a straightforward tutorial to using the TDAstats package to conduct topological data analysis (specifically, persistent homology using Vietoris-Rips simplicial complexes) in R.

N.B. this Gist largely assumes the reader is familiar with a Vietoris-Rips simplicial complex, persistent homology, topological barcodes, and persistence diagrams. Please make sure you have at least a basic understanding of these terms prior to proceeding.

	#
	# Copyright (C) 2013-2020 Vinay Sajip. New BSD License.
	#
	import os
	import os.path
	from subprocess import Popen, PIPE
	import sys
	from threading import Thread
	from urllib.parse import urlparse
	from urllib.request import urlretrieve

	/*
	* Copyright (c) 2013. Regents of the University of California
	*
	* Licensed under the Apache License, Version 2.0 (the "License");
	* you may not use this file except in compliance with the License.
	* You may obtain a copy of the License at
	*
	* http://www.apache.org/licenses/LICENSE-2.0
	*
	* Unless required by applicable law or agreed to in writing, software

	Three comparison points:
	Presto + RCFile vs Impala + RCFile vs Impala + Parquet

	Note: Query time, CPU utilization, Disk read tput (KBRead)

	Impala v1.1.1
	Presto v0.52
	================================================================================================================================
	Presto + RCFile:
	select ss_sold_date_sk, count(*) from store_sales_rcfile group by 1 order by 1 limit 2000;

	(* Purely functional I/O in Ocaml via the Free monad.
	* by Ricky Elrod <[email protected]>.
	*
	* This is free and unencumbered software released into the public domain.
	*
	* Anyone is free to copy, modify, publish, use, compile, sell, or
	* distribute this software, either in source code form or as a compiled
	* binary, for any purpose, commercial or non-commercial, and by any
	* means.
	*