bohdanszymanik’s gists

bohdanszymanik / kfoundry2michlet.fsx

Created May 29, 2012 10:36

MichLet file generator from Kayak Foundry export

	open System
	open System.IO

	let cSs = File.ReadAllLines(@"C:\Extras\kfoundry1.6.4\520x50a.txt")
	(*
	Raw form data exported from kfoundry with baseline at the waterline looks like this:

	Form Position, X (mm), Y (mm)
	...
	Form at 52 cm, 60, 39.3333

bohdanszymanik / IncidentAnalysis.R

Last active December 18, 2015 12:09

Example usage of R data.table in an incident analysis

	# data was closed incidents in a tab separated txt file that looked like this
	# week category P1 P2 P3
	# sucked up into R it formed a data frame like so
	# 1 28/05/2012 some category1 NA NA 1
	# 2 28/05/2012 some category2 NA 2 1
	# 3 28/05/2012 some category3 1 NA 1
	# 4 28/05/2012 some category4 NA NA 2

	# as it happens, this data was manually collected from excel spreadsheets
	# unfortunately the spreadsheets were for differing week ranges and there was some overlap with

bohdanszymanik / RTweetsAnalysis.R

Last active December 19, 2015 12:38

R Tweets Example

	# download tweets using twitter search api, combine text and count +ve -ve words to get sentiment
	# minor modifications to the code taken from
	# http://jeffreybreen.wordpress.com/2011/07/04/twitter-text-mining-r-slides/

	library(RCurl)
	library(twitteR)
	library(ROAuth)

	requestURL <- "https://api.twitter.com/oauth/request_token"
	accessURL = "http://api.twitter.com/oauth/access_token"

bohdanszymanik / azureTableScript.fsx

Created July 11, 2013 10:45

WindowsAzureTableStorage basic F# script sample

	#r @"C:\wd\AzureTxnUploader\packages\WindowsAzure.Storage.2.0.6.0\lib\net40\Microsoft.WindowsAzure.Storage.dll"
	#r @"C:\wd\AzureTxnUploader\packages\Microsoft.Data.OData.5.2.0\lib\net40\Microsoft.Data.OData.dll"
	#r @"C:\wd\AzureTxnUploader\packages\Microsoft.Data.Edm.5.2.0\lib\net40\Microsoft.Data.Edm.dll"
	#r @"C:\wd\AzureTxnUploader\packages\System.Spatial.5.2.0\lib\net40\System.Spatial.dll"

	#r "Microsoft.WindowsAzure.ServiceRuntime"
	#r "System.Data.Services.Client"
	#r "System.Linq"

	open System

bohdanszymanik / txnSimulator.fsx

Created July 16, 2013 04:41

Sample using both Windows Azure Table Storage and a bit of simulation with MathNet.Numerics.Statistics

	//
	// We're going to simulate some financial transactions and experiement with storing them in azure table storage
	//

	//#r "..\packages\Fog.0.1.3.1\Lib\Net40\Fog.dll"
	//#r "..\packages\FogMyBuild\Debug\Fog.dll"
	//#r "Microsoft.WindowsAzure.Diagnostics"
	#r @"C:\wd\AzureTxnUploader\packages\WindowsAzure.Storage.2.0.6.0\lib\net40\Microsoft.WindowsAzure.Storage.dll"

	//#r @"C:\wd\AzureTxnUploader\packages\Microsoft.Data.Edm.5.5.0\lib\net40\Microsoft.Data.Edm.dll"

bohdanszymanik / readAzureTableStorage.fsx

Created July 23, 2013 07:25

F# script example retrieving data from Azure Table Storage - use case is an imaginary store of transactions in Json encoded document format indexed with a partitionkey of year-customer and rowkey of month-account

	#r @"C:\wd\AzureTxnUploader\packages\WindowsAzure.Storage.2.0.6.0\lib\net40\Microsoft.WindowsAzure.Storage.dll"
	#r @"C:\wd\AzureTxnUploader\packages\Microsoft.Data.OData.5.2.0\lib\net40\Microsoft.Data.OData.dll"
	#r @"C:\wd\AzureTxnUploader\packages\Microsoft.Data.Edm.5.2.0\lib\net40\Microsoft.Data.Edm.dll"
	#r @"C:\wd\AzureTxnUploader\packages\System.Spatial.5.2.0\lib\net40\System.Spatial.dll"

	#r "Microsoft.WindowsAzure.ServiceRuntime"

	open System
	open Microsoft.WindowsAzure
	open Microsoft.WindowsAzure.Storage

bohdanszymanik / characterRecognition.fsx

Last active December 28, 2015 04:29

Kaggle numeric character recognition in F# using: 1. CsvFile typeprovider to get the data 2. vectors with cosine similarity to determine k nearest neighbours and 3. the ability to display characters with wpf using wpf Rectangles laid out onto a canvas. Based on the machine learning example from Mathias Brandewinder and the coding dojo here: http…

	open System

	#r @"../packages/FSharp.Data.1.1.10/lib/net40/FSharp.Data.dll"

	#r @"c:\wd\MathDemo\packages\MathNet.Numerics.2.6.2\lib\net40\MathNet.Numerics.dll"
	#r "../packages/MathNet.Numerics.FSharp.2.6.0/lib/net40/MathNet.Numerics.FSharp.dll"

	open MathNet.Numerics.LinearAlgebra
	open MathNet.Numerics.LinearAlgebra.Double

bohdanszymanik / zipLogReader.fsx

Created November 25, 2013 08:58

Work mate's (@LukeGumbley) suggestion - read log files in compressed, very useful for large log files - big data stuff. The reduction in IO makes parsing lines substantially quicker. The test.zip file used for the code below held 100 log files totalling 500MB raw and 16MB zipped and all lines were processed in a just a few seconds on a laptop.

	#r "System.IO"
	#r "System.IO.Compression"
	#r "System.IO.Compression.FileSystem"

	open System.IO
	open System.IO.Compression

	// open up streams from there
	for entry in Compression.ZipFile.OpenRead(@"c:/temp/test.zip").Entries do
	printfn "%s" entry.FullName

bohdanszymanik / sentiment.sql

Last active October 14, 2020 01:22

Sentiment analysis of text data with a plain old t-sql approach using tables for stop words, score words, custom phrases, and messages with trending over time. (Proved to be remarkably good in our work environment for highlighting low message scores on emails at times this was expected to occur.)

	-- T-SQL based sentiment analysis on text following approach in http://dl.dropboxusercontent.com/u/4839225/twitter%20text%20mining%20with%20R.pdf
	-- with trending over time

	-- create scoreword table and populate from a list of word scores such as the opinion lexicon listed in http://www.cs.uic.edu/~liub/FBS/sentiment-analysis.html
	CREATE TABLE [ScoreWords](
	[word] [varchar](50) NOT NULL,
	[score] [int] NOT NULL
	) ON [PRIMARY]

	-- create a table of stopwords and populate them with a suitable list eg check http://en.wikipedia.org/wiki/Stop_words

bohdanszymanik / SCOMDGML.xslt

Created December 4, 2013 02:42

Take a SCOM management pack and transform it to DGML to display as a graph inside visual studio. Helps to visualise your system definition models.

	<?xml version="1.0" encoding="utf-8"?>
	<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://schemas.microsoft.com/vs/2009/dgml" version="1.0">
	<xsl:output method="xml" indent="yes"/>
	<xsl:template match="/">
	<DirectedGraph>
	<Nodes>
	<xsl:apply-templates select="ManagementPack/TypeDefinitions/EntityTypes/ClassTypes/ClassType" />
	</Nodes>
	<Links>
	<xsl:apply-templates select="ManagementPack/TypeDefinitions/EntityTypes/RelationshipTypes/RelationshipType" />

Bohdan Szymanik bohdanszymanik