Skip to content

Instantly share code, notes, and snippets.

@DrLulz

DrLulz/converted.csv

Created April 11, 2015 02:39
Show Gist options
  • Save DrLulz/d65e2c2e988bea973677 to your computer and use it in GitHub Desktop.
Save DrLulz/d65e2c2e988bea973677 to your computer and use it in GitHub Desktop.
Download ZIP
For Dr.Raymond
Raw
converted.csv
V1 V2 V3 V7 V8 V9 V12 V14 V15 V16 V17 V18 V19 V20 V21 V22 V23 V24 V25 V26 V27 V29 V30 V31 V32 V34 V35 V36 V37 V40 V41 V42 V43 V49 V52
1234567 1111111-2222222 444444 Smith James M 123 East Street Somewhere NC 90210 2 00:00.0 (752) 811-1234 (752) 811-1375 (752) 811-1379 [email protected] 0 eng White Not Hispanic or Latino 3 3 SYSTEM Healthy People Doctors 1000000000 NPI Nurse Practitioner PA-C 1000000000 1000000000 PA-C ATTENDING Patient words: The first line there is very useful for any script. It, defines a variable,that points, to the folder, containing the scriptThis is essential if your script needs other files that are located in the sameThe second line creates an array of file names that are required for the script to run properly. Weell use this, along with. /note/ for drainage: Another child at day care has pink eye . e_He has been smoking a crack pipe at night.
2345678 3333333-4444444 111111 Johnson John M 1500 NW Ave Boring NC 33304 1 00:00.0 (157) 555-8167 (157) 836-8167 (150) 836-8167 [email protected] 0 eng White Not Hispanic or Latino 3 3 SYSTEM Sick People Doctors 2000000000 NPI Doctor Gray MD 2000000000 2000000000 MD ATTENDING Something & Something MedicationsShitty Doc Pediatric and Adolescent MedicineChildren less than 2 years oldWe do not routinely recommend the use of cold and cough medication is children under 2 years of age. These medications have not been proven to work in these smaller patients. Also, there are questions about possible negative side effects in younger children.What can you do?-Use saline nose drops to help with nasal congestion and drainage. Put 2 drops in each nostril. Wait about 60 seconds and then use the bulb suction syringe to clear the nose.-Make sure you child is taking lots of fluids.-Run a cool mist humidifier to increase air moisture.-Elevate the head of your childs bed to improve drainage and decrease cough.Children older than 2 yearsCold and cough medication may help your childs symptoms. But, there may also be side effects that come with these medications, including increased irritability, restlessness and strange behavior. If you do give your child medication:-Do not give children medicine that is packaged and made for adults.~ Use only products marked for use in babies, infants or children (sometimes called pediatric use).~-Cough and cold medicines come in many different strengths.~ If you are unsure about the right product for your child, ask a healthcare provider.-If other medicines (over-the-counter or prescription) are being given to a child, the childs healthcare provider should review and approve their combined use.-Read all of the information in the Drug Facts box on the package label so that you know the active ingredients and the warnings. Follow the directions in the Drug Facts box. ~Do not give a child medicine more often or in greater amounts than is stated on the package.-Too much medicine may lead to serious and life-threatening side effects, particularly in children aged 2 years and younger.-For liquid products, parents should use the measuring device (dropper, dosing cup or dosing spoon) that is packaged with each different medicine formulation and that is marked to deliver the recommended dose.~ A kitchen teaspoon or tablespoon is not an appropriate measuring device for giving medicines to children.-If you DO NOT UNDERSTAND the instructions on the product, or how to use the dosing device (dropper, dosing cup or dosing spoon), DO NOT USE the medicine.~ Consult your healthcare provider if you have questions or are confused.-Cough and cold medicines only treat the symptoms of the common cold such as runny nose, congestion, fever, aches, and irritability.~ They do not cure the common cold.~ Children get better with time.-If a childs condition worsens or does not improve, stop using the product seek medical advice
3456789 5555555-6666666 222222 Williams Robert M 100 Ward Ave Boondocks NC 17821 2 00:00.0 (298) 731-6180 (298) 731-6185 (298) 731-7315 0 Undefined Undefined 3 3 SYSTEM Hospital 3000000000 NPI Doctor Stevens MD 3000000000 3000000000 MD ATTENDING The patient is a 2 year, 9 month old monkey who presents with odd symptoms. Symptoms include beating chest and eating bananas. Onset was 8000 day(s) ago. Associated symptoms include scratching (phlegm vomit 6x days ago) and excessive hairiness. Note for symptoms: pt was last seen 2x light years ago. no better
4567891 7777777-8888888 333333 Jones Michael M 23 Fox Hollow Way Podunk NC 67592 2 00:00.0 (678) 035-7200 (678) 035-7598 (678) 044-7598 0 Undefined Undefined 3 3 SYSTEM Urgent Care 4000000000 NPI President Obama PS 4000000000 4000000000 PS ATTENDING The patient is a 31 year, 3 month old IT geek who presents with self-loathing symptoms. Symptoms include sleepiness, orange fingers from dorito eating, carpal tunnel, and lack of social interaction. Onset was 9 year(s) ago. Associated symptoms include poor hygeine (stench / dirty hair ), 'obsesity" and online trolling.
Raw
original.csv
V1 V2 V3 V7 V8 V9 V12 V14 V15 V16 V17 V18 V19 V20 V21 V22 V23 V24 V25 V26 V27 V29 V30 V31 V32 V34 V35 V36 V37 V40 V41 V42 V43 V49 V52
1234567 1111111-2222222 444444 Smith James M 123 East Street Somewhere NC 90210 2 00:00.0 (752) 811-1234 (752) 811-1375 (752) 811-1379 [email protected] 0 eng White Not Hispanic or Latino 3 3 SYSTEM Healthy People Doctors 1000000000 NPI Nurse Practitioner PA-C 1000000000 1000000000 PA-C ATTENDING {\\rtf1\\ansi\\ansicpg1252\\deff0\\deflang1033{\\fonttbl{\\f0 Arial;}}{\\colortbl;\\red255\\green0\\blue0;}\\viewkind4\\uc1\\pard\\f0\\fs18 \\pard\\nowidctlpar\\tx288 \\f9\\fs18 \\tab Patient words: The first line there is very useful for any script. It, defines a variable,that points, to the folder, containing the script \\par This is essential if your script needs other files that are located in the same \\par\\par The second line creates an array of file names that are required for the script to run properly. Weêll use this, along with. /note/ for \Eye drainage\: Another child at day care has pink eye . ´è_He has been smoking a crack pipe at night. \\par\\par}
2345678 3333333-4444444 111111 Johnson John M 1500 NW Ave Boring NC 33304 1 00:00.0 (157) 555-8167 (157) 836-8167 (150) 836-8167 [email protected] 0 eng White Not Hispanic or Latino 3 3 SYSTEM Sick People Doctors 2000000000 NPI Doctor Gray MD 2000000000 2000000000 MD ATTENDING {\\rtf1\\ansi\\deflang1033\\ftnbj\\uc1\\deff1 {\\fonttbl{\\f0 \\froman \\fcharset0 Times New Roman;}{\\f1 \\fswiss \\fcharset0 Arial;}} {\\colortbl ;\\red255\\green255\\blue255 ;\\red0\\green0\\blue0 ;} {\\stylesheet{\\fs24\\cf2\\cb1 Normal;}{\\cs1\\cf2\\cb1 Default Paragraph Font;}{\\s2\\cf0\\cb1 ;}} \\paperw12240\\paperh15840\\margl1800\\margr1800\\margt1440\\margb1440\\headery720\\footery720\\deftab720\\formshade\\aendnotes\\aftnnrlc\\pgbrdrhead\\pgbrdrfoot \\sectd\\pgwsxn12240\\pghsxn15840\\marglsxn1800\\margrsxn1800\\margtsxn1440\\margbsxn1440\\headery720\\footery720\\sbkpage\\pgncont\\pgndec \\plain\\plain\\f1\\fs24\\pard\\s0\\tx720\\qc\\widctlpar\\f0\\b Something & Something Medications\\par Shitty Doc Pediatric and Adolescent Medicine\\par\\ql\\plain\\f1\\fs24\\f0\\b\\par\\par\\fs20 Children less than 2 years old\\par\\b0 We do not routinely recommend the use of cold and cough medication is children under 2 years of age. These medications have not been proven to work in these smaller patients. Also, there are questions about possible negative side effects in younger children.\\par\\par What can you do?\\par -Use saline nose drops to help with nasal congestion and drainage. Put 2 drops in each nostril. Wait about 60 seconds and then use the bulb suction syringe to clear the nose.\\par\\par -Make sure you child is taking lots of fluids.\\par\\par -Run a cool mist humidifier to increase air moisture.\\par\\par -Elevate the head of your child\\'92s bed to improve drainage and decrease cough.\\par\\par\\b Children older than 2 years \\par\\b0 Cold and cough medication \\i may \\i0 help your child\\'92s symptoms. But, there may also be side effects that come with these medications, including increased irritability, restlessness and strange behavior. If you do give your child medication:\\par\\par\\cf2 -Do not give children medicine that is packaged and made for adults.\\~ Use only products marked for use in babies, infants or children (sometimes called \\'93pediatric\\'94 use).\\~ \\cf0\\par\\sb100\\sa100\\plain\\f1\\fs24\\f0\\cf2\\fs20 -Cough and cold medicines come in many different strengths.\\~ If you are unsure about the right product for your child, ask a healthcare provider. \\par -If other medicines (over-the-counter or prescription) are being given to a child, the child\\'92s healthcare provider should review and approve their combined use. \\par -Read all of the information in the \\'93Drug Facts\\'94 box on the package label so that you know the \\b\\i active ingredients \\b0\\i0 and the \\b\\i warnings\\b0\\i0 . Follow the \\b\\i directions\\b0\\i0 in the \\'93Drug Facts\\'94 box. \\~Do not give a child medicine more often or in greater amounts than is stated on the package. \\par -Too much medicine may lead to serious and life-threatening side effects, particularly in children aged 2 years and younger. \\par -For liquid products, parents should use the measuring device (dropper, dosing cup or dosing spoon) that is packaged with each different medicine formulation and that is marked to deliver the recommended dose.\\~ A kitchen teaspoon or tablespoon is not an appropriate measuring device for giving medicines to children. \\par -If you DO NOT UNDERSTAND the instructions on the product, or how to use the dosing device (dropper, dosing cup or dosing spoon), DO NOT USE the medicine.\\~ Consult your healthcare provider if you have questions or are confused. \\par -Cough and cold medicines only treat the symptoms of the common cold such as runny nose, congestion, fever, aches, and irritability.\\~ They do not cure the common cold.\\~ Children get better with time. \\par -If a child\\'92s condition worsens or does not improve, stop using the product seek medical advice\\par\\pard\\s0\\ql\\plain\\f1\\fs24\\par}
3456789 5555555-6666666 222222 Williams Robert M 100 Ward Ave Boondocks NC 17821 2 00:00.0 (298) 731-6180 (298) 731-6185 (298) 731-7315 0 Undefined Undefined 3 3 SYSTEM Hospital 3000000000 NPI Doctor Stevens MD 3000000000 3000000000 MD ATTENDING {\\rtf1\\ansi\\ansicpg1252\\deff0\\deflang1033{\\fonttbl{\\f0 Arial;}}{\\colortbl;\\red255\\green0\\blue0;}\\viewkind4\\uc1\\pard\\f0\\fs18 \\pard\\nowidctlpar\\tx288 \\f9\\fs18 The patient is a 2 year, 9 month old monkey who presents with odd symptoms. Symptoms include beating chest and eating bananas. Onset was 8000 day(s) ago. Associated symptoms include scratching (phlegm vomit 6x days ago) and excessive hairiness. Note for \Monkey symptoms\: pt was last seen 2x light years ago. no better\\par\\par}
4567891 7777777-8888888 333333 Jones Michael M 23 Fox Hollow Way Podunk NC 67592 2 00:00.0 (678) 035-7200 (678) 035-7598 (678) 044-7598 0 Undefined Undefined 3 3 SYSTEM Urgent Care 4000000000 NPI President Obama PS 4000000000 4000000000 PS ATTENDING {\\rtf1\\ansi\\ansicpg1252\\deff0\\deflang1033{\\fonttbl{\\f0 Arial;}}{\\colortbl;\\red255\\green0\\blue0;}\\viewkind4\\uc1\\pard\\f0\\fs18 \\pard\\nowidctlpar\\tx288 \\f9\\fs18 The patient is a 31 year, 3 month old IT geek who presents with self-loathing symptoms. Symptoms include sleepiness, orange fingers from dorito eating, carpal tunnel, and lack of social interaction. Onset was 9 year(s) ago. Associated symptoms include poor hygeine (stench / dirty hair ), 'obsesity" and online trolling.\\par\\par}
Raw
raymond.py
# -*- coding: utf-8 -*-
import re
import csv
import unicodedata as udata
from cStringIO import StringIO
from pyth.plugins.rtf15.reader import Rtf15Reader
#from pyth.plugins.plaintext.writer import PlaintextWriter
#from pyth.plugins.xhtml.writer import XHTMLWriter
f_ori = '/Users/drlulz/Desktop/Data Example for DrLulz.csv'
f_new = '/Users/drlulz/Desktop/newer.csv'
def decode_cell(row):
temp_cell = []
for cell in row:
if re.match(r'^{\\\\rtf', cell):
cell_encode = udata.normalize('NFKD', cell.decode('unicode_escape')).encode('ascii', 'ignore')
cell_rtf = Rtf15Reader.read(StringIO(cell_encode))
cell_txt = [x.content for x in cell_rtf.content]
for line in cell_txt:
for l in line:
temp_cell.append(l.content)
combined = [i for sub in temp_cell for i in sub]
return ''.join(combined)
#raise Exception('temp_cell')
def find_rtf(row):
temp_row = []
for cell in row:
if type(cell) == str:
cell = unicode(cell, "utf-8", errors="ignore")
else:
cell = unicode(cell)
if re.match(r'^{\\\\rtf', cell):
cell_matched = decode_cell(row)
temp_row.append(cell_matched.encode('ascii', 'ignore'))
else:
temp_row.append(cell.encode('ascii', 'ignore'))
return temp_row
with open(f_ori, 'rU') as file1, open(f_new, 'wb') as file2:
reader = csv.reader(file1)
writer = csv.writer(file2)
#filtered_row = []
for row in reader:
new_row = find_rtf(row)
writer.writerow(new_row)
#filtered_row.append(new_row)
#print filtered_row[0]
#print filtered_row[1]
#print filtered_row[2]
#print filtered_row[3]
#print filtered_row[4]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment