Skip to content

Instantly share code, notes, and snippets.

@shubham0204
Created March 23, 2019 12:28
Show Gist options
  • Save shubham0204/f1fdad7a910fe17064752affaef474cb to your computer and use it in GitHub Desktop.
Save shubham0204/f1fdad7a910fe17064752affaef474cb to your computer and use it in GitHub Desktop.
import os
import yaml
dir_path = 'raw_data'
files_list = os.listdir(dir_path + os.sep)
questions = list()
answers = list()
for filepath in files_list:
stream = open( dir_path + os.sep + filepath , 'rb')
docs = yaml.safe_load(stream)
conversations = docs['conversations']
for con in conversations:
if len( con ) > 2 :
questions.append(con[0])
replies = con[ 1 : ]
ans = ''
for rep in replies:
ans += ' ' + rep
answers.append( ans )
elif len( con )> 1:
questions.append(con[0])
answers.append(con[1])
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment