Skip to content

Instantly share code, notes, and snippets.

@RyanParsley
Last active June 20, 2017 13:35
Show Gist options
  • Select an option

  • Save RyanParsley/f2629488fd71846c3d4dbbf8ebd42eb9 to your computer and use it in GitHub Desktop.

Select an option

Save RyanParsley/f2629488fd71846c3d4dbbf8ebd42eb9 to your computer and use it in GitHub Desktop.
Lessons learned after spending a week of quality time with Watson's Retrieve and Rank service.

name: NLP Crash Course class: middle, center

NLP Crash Course

June 16, 2017


Retrieve and Rank

It's Watson service to, basically, "train" search results.

???

This service is, essentially, a search engine with the ability to add a lot of meta data. In it's untrained state, it does a good job of finding many potential answers to each question. These answers look to share words, but are void of context. Then, a trainer clicks through a 4 star interface to "grade" the relevance of each answer. Train a hundred or more questions and you stop calling it a search engine and start calling it Machine Learning Natural Language Processor.


Example: How Do I pair my Phone?

Note: Make sure to switch on the ignition and the radio. Shift the transmission into park (P) for automatic transmission or first gear for manual transmission.

Note: To scroll through the menus, press the up and down arrows on your audio system.

Make sure to switch on your phone's Bluetooth feature before starting the search. See your device's manual if necessary.

Press the PHONE button.

Select the option for Bluetooth Devices.

Press the OK button.

Select the option to add. This starts the pairing process.

When a message to begin pairing appears in the audio display, search for SYNC on your device.

When prompted on your cell phone’s display, confirm that the PIN provided by SYNC matches the PIN displayed on your cell phone. Your phone is now paired and the display indicates that the pairing was successful. If you are prompted to enter a PIN on your device, enter the PIN displayed on the screen. The display indicates when the pairing is successful.

SYNC may prompt you with more cell phone options. Your cell phone may also prompt you to give SYNC permission to access information. For more information on your cell phone’s capability, see your cell phone’s manual and visit the website.


R&R: The Good

The API is pretty easy to use and the node module looks to cover a lot of scenarios.

???

I went from not knowing what Retrieve and Rank was, to using it and leveraging it from both the command line and an Alexa skill in less than a week and it probably took less than 100 lines of code not counting training content


R&R: The Bad

The developer console strikes me as unreliable and inscrutable.

???

  • I was unable to log in for half of a day
  • An incident that spanned 2 days made it impossible to export training data
  • I need a map to get through their UI (too many services tucked in places that are not obvious)

R&R: The Bad

  • The html that I saw for a data source unusable due without fixing semantics.
  • Even after I "fixed it", it wasn't structured enough to give answers I feel are appropriate for text to speech

Hand tailored solution

response structure

???

With Retrieve and Rank, the output will always have the same tone as your answer data. There's an alternative approach where we piece together responses much like a MadLib.


What's the data look like for that?

sample database

???

This can offer a much richer response, when requests contain sufficient nuance and you take the time to craft response data to take advantage.


Recap

  • If the source you feed into Retrieve and Rank has the right tone and is well structured, the robots can do a lot of heavy lifting for you.
  • If you have sufficiently complicated questions and answers, you can get more personality by crafting granular entities.

Bonus Round: Alexa

An Alexa skill that serves as a simple conduit to Watson's Retrieve and Rank service was pretty straight forward to set up.

The biggest problem being most of the answers are too long winded for a good voice experience.

???

Once I had a node app that could interface with Watson's API, turning it into an alexa skill was fairly trivial. The biggest problem being most of the answers are too long winded for a good voice experience.


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment