Code to get data from WRDS into PostgreSQL

wrds_fetch.md

Go here for the latest.

@amarder

Above is my rough first pass at translating my wrds_fetch.pl from Perl into Python. It still depends on pipes, but much less so than the Python code (also my Python is much better than my Perl ever was, so this code is a bit easier to follow even though it's somewhat clunky due to some parts being almost line-for-line translations).

I think if one could replace likes like this:

cmd_template = "echo \"%s\" | "
cmd_template += "ssh -C %[email protected] 'qsas -stdio -noterminal ' 2>/dev/null"

with some other way of getting code (what goes in %s) into the SSH connection to the WRDS server, then this would work anywhere (including on Windows).

If you have any suggestion on that part, I could wrap it up to be a more complete replacement for the Perl code (translating the companion code in wrds_update.pl should be easier, as this is higher-level code).

@amarder

As a bonus, I included (see In [16]:) code to pull the data directly from WRDS into a Pandas data frame. The code that fixes types in PostgreSQL could adapted to fix column types in the Pandas data frame.

@amarder

Ignore the above. Better code here.