Yuxiang (Jimmy) Wu yuxiang-wu

Model Name:

RoBERTa + CSPT (single model)

Model Description:

We first train a generation model to generate synthetic data from ConceptNet. We then build the commonsense pre-trained model by finetuning RoBERTa-large model on the synthetic data and Open Mind Common Sense (OMCS) corpus. The final model is finetuned from the pretrained commonsense model on CSQA.

Experiment Details:

Commonsense Pre-training:

epochs: 5
maximum sequence length: 35
learning rate: 3e-5

Ubuntu 16.04 Developer Tools Installation

First things first !

sudo apt update
sudo apt upgrade

Standard Developer Tools

sudo apt-get install build-essential git

	# NOTE:
	# You can find an updated, more robust and feature-rich implementation
	# in Zeno Build
	# - Zeno Build: https://github.com/zeno-ml/zeno-build/
	# - Implementation: https://github.com/zeno-ml/zeno-build/blob/main/zeno_build/models/providers/openai_utils.py

	import openai
	import asyncio
	from typing import Any

	#!/usr/bin/env python3

	import sys
	from argparse import ArgumentParser
	from collections import defaultdict

	def parse_args():
	p = ArgumentParser('Converts word to integer using byte-pair encoding.')
	p.add_argument(
	'--input',