ccritchfield

---------------------------------------
VBA Excel Utilties
---------------------------------------

Excel is a great tool for quick-n-dirty analysis.
But, staring at raw data in it can be a pain. So,
I created misc utilities to pull in data from SQL
servers, format data, border-split rows to group
similar data together, dupe-check data (made this
before MS got smart and built a version into later

ccritchfield

-----------------------------------------
Sample Sizing Big Data to Reduce Processing
-----------------------------------------

When grinding massive data sets, it'd be nice to 
figure out a statistically supported subset sample
size you could come up with and run instead, that
way you could cut down on the time and resources
needed to come up with some results. IE: instead
of spending tons of time and resources grinding

ccritchfield

--------------------------------------------
Examples of SQL code from my last job.
--------------------------------------------

While I did analytics / BI work, I also wrote a lot of SQL's
to audit data. Due to the nature of telecom work, we had
a few tools used by provisioning department that allowed
them to break business rules. When you're running numbers,
and find out things don't add up, you start digging. So,
I would come across this, and then get with the IT / IS

ccritchfield

Created a viral model, b/c I wanted to see the chance of getting
infected on a daily basis. This is a "brute force" model that
simulates each individual in a population (and lets you run
multiple populations at a time, each interacting with the
others). The code files contain a lot of notes to explain
both the science behind why I did something, and they python
behind why I coded it as I did. I was shooting to be as
simple as possible, to keep the code from obfuscating
what was going on, and to keep the science from confusing
the code. So, it may not be coded the "Pythonic Way", but,

ccritchfield

v2 of the model adds and removes "attributes" from a person's list
as-needed to try to cut down on memory use.

EG: so a person won't be a list[0,0,0,0] tracking all 4 attributes
(status, incubation days, symptoms, transmission rate) all the time.
Instead, if they're uninfected, they'll simply be a 1 element list
with their status. As they get infected and incubate, then an
incubation attribute is added to count down. As they finish
incubating, the incubation attr will then get re-used to see
what symptom they have. As they become contagious, they get

ccritchfield

v3 of the model gets rid of tracking each individual as a list,
and simply tracks aggregate pools of population based on status,
incubation days, etc.

This reduces memory footprint greatly, b/c now we're just dealing
with integer variables instead of massive lists. So, we can now
process massive populations, like US or World population.

But, we lose detail.. if we wanted to add more features to the
model or individuals, we'd have to start bending over backwards