Created
October 28, 2014 16:46
-
-
Save abelards/fc005fc92ee9c1837629 to your computer and use it in GitHub Desktop.
XLS2PG transform
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
(defun xls2f (str) | |
"Transform a '42,42' string into a 42.42 float." | |
;; (atof (replace-regexp-in-string "," "." "42,42")) | |
(format nil "~{~a~^.~}" (sq:split-sequence #\, str)) | |
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
LOAD CSV FROM FILENAME MATCHING ~/huge_file.csv/ | |
WITH ENCODING iso-8859-1 | |
INTO postgresql://login_x@localhost:5432/db | |
( | |
float num using (xls2f str) | |
) | |
WITH skip header = 1, | |
fields optionally enclosed by '\"', | |
fields escaped by double-quote, | |
fields terminated by ','; |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
See dimitri/pgloader#92 for a full blown example using user defined transformation functions, and to debug use a very small file and both the following options:
--debug --verbose
, when both specified that's the same as--client-min-messages data
which is the more advanced debug logs available.Your transforms file should begin with the form:
Also you can name the file with a
.lisp
extension ;-)