Transcribing ProPublica podcast with Python and Watson Speech to Text API

EXAMPLE_WATSON_API_README.md

Using IBM Watson Speech to Text API to translate a ProPublica podcast

An example of using the Watson Speech to Text API to translate a podcast from ProPublica: How a Reporter Pierced the Hype Behind Theranos

This is just a simpler demo of the same technique I demonstrate to make automated video supercuts in this repo: https://github.com/dannguyen/watson-word-watcher

The transcription takes just a few minutes (less if you parallelize the requests to IBM) and is free...but it isn't perfect by any means. It doesn't fare super well on proper nouns:

• Charles Ornstein's last name is transcribed as Orenstein
• John Carreyrou's last name becomes John Kerry Roo
• Elizabeth Holmes is Elisabeth homes
• Theranos isn't popular enough to be a proper noun yet, so it is transcribed, variously, as their in house, their nose, and thoroughness
• in one instance, their Palo Alto headquarters is transcribed as they're palatable headquarters

On the other hand, ProPublica, Ken Auletta, David Boies are perfectly transcribed. Anyway, it's decent enough for keyword searching.

The watson-transcribe.py script is a Python 3.x script that:

1. downloads the podcast
2. splits it up into smaller WAV files
3. uploads each WAV file to Watson's API to transcribe
4. saves the JSON response
5. parses the JSON response into two plaintext files

This gist contains what the plaintext files look like:

• raw.txt.md - just the line-by-line transcription from Watson
• words.csv - The word-by-word transcription, including the confidence level and the timestamps of the words.

raw.txt.md

and nbsp walking to the ProPublica podcast I'm Charlie Orenstein and I cover healthcare here at ProPublica

today we're gonna be talking about laboratory testing

it's no secret to many that lab testing has been caught in the dark ages of medicine

or you have to go to your doctor to get a prescription for a lab test

and then your doctor gets the results of the lab test if you're lucky will share with you

and then you'll be back to going to your doctor again if you need a follow up

a company called their in house aim to change that

founded in two thousand and three by Elisabeth homes

a nineteen year old drop out of Stanford University this company and to really shake up laboratory testing

it received fawning coverage from many mainstream media outlets including The New York Times

but John Kerry Roo a reporter at the Wall Street journal took a more critical look at their in house

and found that behind this glossy surface there were a lot of questions about both its effectiveness and the science that underlie its main products

John is joining us here on the podcast John Markham hi thanks for having so how did you decide to take a look at their house

well I had read Ken Auletta as profile of Elizabeth homes in the New Yorker in which I think came out in December of two thousand fourteen and

I found it interesting there were some brief critical %HESITATION sections in there that raised questions for me but I don't think all that much more of it and then as the calendar year turned to %HESITATION two thousand fifteen a couple weeks later mid two thousand fifteen I got a tip from someone not a primary source who had any primary information but someone who is a relating to me

some you know third hand information that that things might not be exactly as they seemed at this company thoroughness

so it's interesting because I think a number of reporters got tips along the way and and you were the one to pursue them I personally heard from somebody who said that his lab results didn't match the lab results he got from a different lab so it doesn't seem like this was a huge secret that there were questions out

right and shortly after I began looking into the company in and looking into this tip a Stanford on a medical school professor I believe %HESITATION put out a critical opinion piece in the journal of the American medical association

and he didn't really have any any information other than the say this is a company that was

making very bold statements about its breakthroughs in the science of laboratory testing saying that it could test for a number of conditions off just a drop of blood **** from the finger and yet it hadn't really done what you usually do in medicine which is a peer review

and he was questioning %HESITATION this new trend of making all these assertions about what you've invented without really proving it and having it vetted by your peers and then there was another %HESITATION professor in Toronto I believe a couple weeks or couple months later who who came out with a critical medical journal and I editorial as well so people are beginning to speak out in the scientific community as I was doing my reporting

so let's just take a step back and put the science aside for a second what is the hope of fairness what it promised people

so the the fair knows invention as Elizabeth homes are

you know announced it in magazines and at conferences was that %HESITATION with a tiny drop of blood **** from a finger with the lancet

they could run the full range of laboratory tests and get you back results on all these tests within hours

and at a low cost to the latter part is true they charge very low prices and then when you look at the assertion that they can do the full range you you ask laboratory experts what that means and they say can mean anywhere from several hundred to several thousand tests

so the claim was quite bold in this hadn't been done before from just a drop of blood being able to run the full gamut of tests and get results back to the patient's very quickly %HESITATION it did sound like a real scientific breakthrough and third for listeners who may not be aware of what is it usually take as far as blood to run that gamut of test what weirdest blood come from and how much blood is needed

well if you get tested for say a a comprehensive metabolic panel which is a typical panel that you might not get a prescription for your doctor for which is about a half dozen tests

you'll go to request or a lab Corp or a hospital lab in they will draw your blood with a needle odd that they put in your arm and they'll draw about five tubes of blood so quite a bit of blood is typically %HESITATION required to to run you know half dozen tests

so Liz with homes was incredibly successful I think getting it hundreds of millions of dollars four hundred million dollars %HESITATION of invest more money and more than us more than that so that value the company at nine billion dollars or so

yeah we we went back and found some %HESITATION some regular

for filings and were able to calculate that they raised at least seven hundred and fifty million dollars most of it more than six hundred million dollars was raised in two thousand fourteen and that last fundraising round valued the company at about nine billion dollars

which is a a huge of valuation that that's our moral less what quest and LabCorp are each valued at and these companies have been around for a long time and they have a huge revenues in huge profits

are in this laboratory upstart that was founded by a college drop out ten years ago suddenly valued at at the

the same valuation as those two huge companies alright let's stop there because I think listeners Arabic %HESITATION why like how did this unicorn

get so much money what did people see it

that isn't clear what people saw in it other than %HESITATION Elizabeth homes as pitch

that she had made this scientific breakthrough is not clear because I've heard that and and pretty much ascertained during my reporting that the company did not offer really any information

about the science and about how the technology works about how it's

laboratory %HESITATION instrument worked

or about its financials so are investors who were putting up this money were for the most part going in blind

but at a star studded board right

they did they had %HESITATION Henry Kissinger

and now George Shultz and Sam Nunn and %HESITATION

Phil frets they had a bunch of older statesman some exit some retired military commanders

it was a %HESITATION heavy duty board a lot of big names

in %HESITATION the military and and former cabinet members

%HESITATION incident seemed impressive at first glance

so your first piece ran last year and you began raising questions what were the key questions that you found

so the first thing was to

look at whether they were or were not running the full range of laboratory tests

on other proprietary technology

and after a lot of reporting in talking to former employees who were in a position to know exactly what the reality of that was

I was able to determine dead of the more than two hundred and forty blood tests that they offered consumers at their blood draw centers in Walgreens stores

that at the very most at the end of two thousand fourteen fifteen of them were run on their

pride Terry lab instrument which by the way they called the Edison after the inventor Thomas Edison pretty clever out why does it matter though that who cares which instrument that tests were run

well

if you're asserting that you that you made a scientific breakthrough that it enables you

to run the full range of laboratory tests off just

a drop of blood and it turns out that you can only do fifteen tests then

you're you're probably hyping where you are in and your ability and so then the question arises whether you've told the truth to investors whether you've told the truth to the public

but then there's the the other issue that I discussed in my first piece which was published in October which is what whether or not

the testing for those fifteen tests and and that the testing for all the other test

was accurate and produced good results and and I had ex employees telling me

that %HESITATION they questioned the accuracy of the Edison machine and that their endless was also doing things like diluting small blood samples in order to create a bigger volume to run them on commercial analyzers

that also created problems with accuracy

and I would travel to Arizona and talk to patients and doctors there and came up with anecdotal evidence of

a test that didn't seem to square with comparative tests done at other laboratories and so all of that made me realize that it wasn't just about potentially company that it over hyped

it's science and its breakthroughs but it might also be a a matter of public health

right and there are no Swiss briefly aggressive it seems our interns responding to your story both before it ran an after ran Elizabeth homes

actually speaking at a Wall Street journal event %HESITATION was critical of you and this is continued sort of including into this year what is their perspective

the company has been incredibly aggressive I in dealing with our reporting from the first moment that I asked an interview %HESITATION with Elizabeth homes in April after the publication of the story and and told pretty much today in addition to sending threatening letters to some of the people have spoken to me and to %HESITATION trying to convince doctors who had given me accounts of a patient cases and trying to convince them to change their stories they've also are systematically put out

statements

saying that that are reporting was inaccurate and and we of course have very much stood by our reporting and I think the sequence of events that unfolded are since my first story has shown that there was very much something to what we were reporting

so you mentioned threats tell us a little bit about those threats

well one of my sources at one of the people who went on the record how was the widow of a %HESITATION top scientists at their nose %HESITATION Rochelle Gibbons whose husband Ian Gibbons

had worked at the company for eight years he was a an accomplished British biochemist who

before working at their nose had worked at several companies in Silicon Valley and been in Silicon Valley for

several decades

he came with their nose into thousand five and was one of their top scientists if you look at the patents that their nose has he's on all the early patents

as they are co inventor and what happened is %HESITATION Ian committed suicide and I believe it was may of two thousand thirteen

and so I talked to his widow Rochelle and as she relayed to me that I he had a repeatedly told her

and before he died that things were not really working

in the are indeed apartment and that they weren't being successful in implementing this vision that Elizabeth had at least four

%HESITATION some of the the tests

and so before we included it in October fifteenth article we ran it by a fair no sin what it there Ernest do there are no scent or a hat rather had a one of its lawyers from boys Schiller David Boies is firm send a

a threatening letter to Rochelle telling her essentially to stop talking

and Rochelle very much interpreted that letter is an attempt to intimidate her into silence for

so let's talk about that sequence of events %HESITATION maybe starting with the most significant in recent event which is a letter that came from the centers for Medicare and

it services

two there are no switch said that there was an immediate jeopardy to patient health and safety are out of there Norwalk California facility the Newark California lab which is about basically right across San Fransisco bay from there headqu headquarters

and now out to put things in context this is their main laboratory there first laboratory they originally I had it in side they're palatable headquarters and then they moved it to a building across the bay of their two labs they have another lab in the Phoenix area %HESITATION it's the first one that they created and it's also the one that %HESITATION contains their proprietary devices there Edison family of devices that also contains commercial devices but unlike the Arizona laboratory this is where %HESITATION their purported proprietary technology is

so when you learn that the inspectors for the centers for Medicare and Medicaid services one in there and discovered %HESITATION major deficiencies

and some of which are presented immediate jeopardy %HESITATION for patients

then that makes you a wonder about the accuracy of the testing that was done and whether %HESITATION the accuracy of the testing is in question for both the commercial analyzers they have in that lab but also for the

proprietary analyzers the Edison's they have their

since October what else has has happened that sort of lends credence to some of the questions you've raised

well the the FDA also %HESITATION inspected %HESITATION thoroughness is headquarters in Palo alto in the Newark lab

I believe in late August and September

first half of September and came away with the judgment that these tiny little vials that there are no excuses that that Terry calls nano trainers to collect the blood from finger pricks

that those were medical devices that the agency should have cleared and was not given the opportunity to clear

%HESITATION and so it essentially put pressure on their our nose

stop using those little vials

for all but one test a test to detect herpes which the FDA had approved in July

so once the agency essentially took away the nano Tanner their nose had to stop

drawing blood from fingers in the hole their nose proposition which is a tiny drop of blood **** from a finger about is no longer really a reality in the field

and in fact at their nose phlebotomy us at the wellness centers inside Walgreen's stores are now of for the most part almost entirely drawing blood with syringes from the vein and filling you know several tubes of blood like any other lab

in addition there been questions or the relationship between Walgreens and there are no says changing right

that's right we are hearing that I things are getting tense %HESITATION between the two partners

%HESITATION Walgreens after our October fifteenth story made it clear that it would no longer expand the thoroughness wellness center is to allow the rest of its stores

%HESITATION some eight thousand stores across the U. S. that it was freezing

ng

a partnership to the are forty stores in Arizona and the one in and Palo alto and then following the CMS a report two weeks ago about the the conditions at that lab that pose immediate jeopardy to patients Walgreens immediately shut down at the wellness center inside its store in headqu and told fairness that it could no longer test any blood samples collected

at the remaining wellness centers in Arizona at that Newark lab

I think it's worth also stepping back and looking at the role that their noses played in in a broader discussion about modernizing laboratory tests %HESITATION Elizabeth homes and their innermost played a role in getting the Arizona legislature to change a law which allowed patients to no longer require a prescription to get a lab test now allowing patients to directly go to a lab and get a test

what is the role in changing sort of the way we look at that testing at their innocence Roland

that was a pretty significant event the %HESITATION Arizona legislature I believe in about July of last year enacted into law a bill on that said that if you were a consumer out there in in Arizona you could just go

and not get your blood tested and not have a a prescription from a doctor you could go in and do it on your own now that was controversial because of their many in the medical profession who say %HESITATION including you know the general practitioners who would usually Abby the middleman who would give you that prescription who say well

how is that progress because once the the patient

you know has those %HESITATION test results

and more likely than not they're gonna need a doctor's opinion to decipher them and and to know what those results mean

and so why not keep the doctor involved from the beginning

so that first of all there is a logic for forgetting the blood tests and then and then there's an expert opinion there when the test results arrive

and to help you read them and interpret them

so you don't have a crystal ball and I don't either

and some not gonna ask you what's going to happen but I I do think it's worthwhile to get your perspective on water things that people should look for as a story evolves in terms of barometers on on where they're in our society

well the the first thing I think people should take a look at is whether %HESITATION fairness I manages to affix these major deficiencies as at its New York facility

our over the next few weeks for the next few months my understanding is that that the company just got another ten day extension it was originally given ten days I'll and on Friday obtained another ten days and it may be able to get several more extensions but at a certain point the clock is going to stop and around late March

if the company hasn't resolved those issues or at least

submitted to the agency to CMS a credible plan of correction then of the agency will make its inspection report which I'm told is lengthy

public and will start imposing sanctions

so that's the first thing toto watch out for

at the second thing is what what is going to happen to this %HESITATION partnership

tween there are no sin and Walgreens because

if Walgreens were to walk away from thoroughness altogether then it's not clear that there are no less

has any ability to generate any more revenues

%HESITATION and so then you have a private start up that's been valued at nine billion dollars

that isn't generating any revenues and so would then be interesting to watch what

investors who put up money in this company have to say about that

well great where you've been listening to John Kerry a reporter at the Wall Street journal talking about his investigative stories about their analysts

as robin

thank you for listening to the podcast we hope you'll tune in again next

mtn

and nbsp

and nbsp

watson-transcribe.py

from pydub import AudioSegment
from glob import glob
from math import ceil
from os.path import basename, splitext, exists
import json
import requests
import csv
WATSON_USERNAME = "YOUR-USERNAME-HERE"
WATSON_PASSWORD = "YOUR-PASSWORD"
WATSON_ENDPOINT = 'https://stream.watsonplatform.net/speech-to-text/api/v1/recognize'
WATSON_DEFAULT_PARAMS = {
    'continuous': True,
    'timestamps': True,
    'word_confidence': True,
}
WATSON_DEFAULT_HEADERS = {
    'content-type': 'audio/wav'
}

# via: http://www.propublica.org/podcast/item/how-a-reporter-pierced-the-hype-behind-theranos/
DOWNLOAD_URL = 'https://api.soundcloud.com/tracks/247345268/download?client_id=cUa40O3Jg3Emvp6Tv4U6ymYYO50NUGpJ'
AUDIO_FILENAME = 'podcast.mp3'
AUDIO_SEGMENT_SECONDS = 300


if not exists(AUDIO_FILENAME):
    print("Downloading from", DOWNLOAD_URL)
    resp = requests.get(DOWNLOAD_URL)

    with open(AUDIO_FILENAME, 'wb') as w:
        w.write(resp.content)
        print("Wrote audio file to", AUDIO_FILENAME)

    # convert to WAV
    audio = AudioSegment.from_mp3(AUDIO_FILENAME)
    xs = 0
    while xs < audio.duration_seconds:
        ys = min(xs + AUDIO_SEGMENT_SECONDS, ceil(audio.duration_seconds))
        fname = str(xs).rjust(5, '0') + '-' + str(ys).rjust(5, '0') + '.wav'
        audio[xs*1000:ys*1000].export(fname, format='wav')
        print("Saved", fname)
        xs = ys


## Transcribe each WAV to Watson
for fname in glob("*.wav"):
    # Download watson's response
    tname = splitext(basename(fname))[0] + '.json'
    if exists(tname):
        print("Already transcribed", tname)
    else:
        print("Transcribing", fname)
        with open(fname, 'rb') as r:
            watson_response = requests.post(
                                  WATSON_ENDPOINT,
                                  data=r,
                                  auth=(WATSON_USERNAME, WATSON_PASSWORD),
                                  params=WATSON_DEFAULT_PARAMS,
                                  headers=WATSON_DEFAULT_HEADERS,
                                  stream=False
                             )
            with open(tname, 'w') as w:
                w.write(watson_response.text)
                print("Wrote transcript to", tname)


# Print out the raw transcript and word csv
rawfile = open("raw.txt", "w")
wordsfile = open("words.csv", "w")
csvfile = csv.writer(wordsfile)
csvfile.writerow(['word', 'confidence', 'start', 'end'])

for fname in sorted(glob("*.json")):
    with open(fname, 'r') as f:
        results = json.load(f)['results']
        for linenum, result in enumerate(results): # each result is a  line
            if result.get('alternatives'): # each result may have many alternatives
                # just pick best alternative
                lineobj = result.get('alternatives')[0]
    #            rawfile.writeline(lineobj['transcript'])
                word_timestamps = lineobj['timestamps']
                if word_timestamps:
                    rawfile.write(lineobj['transcript'] + "\n")

                    word_confidences = lineobj['word_confidence']
                    for idx, wordts in enumerate(word_timestamps):
                        txt, tstart, tend = wordts
                        confidence = round(100 * word_confidences[idx][1])
                        csvfile.writerow([txt, confidence, tstart, tend])


rawfile.close()
wordsfile.close()

words.csv