Nick Doiron mapmeld

Code: https://colab.research.google.com/drive/1vltPI81atzRvlALv4eCvEB0KdFoEaCOb?usp=sharing

Can these scores be improved? YES!

Rerunning with more training data, more epochs of training, or using other libraries to set a learning rate / other hyperparameters before training.

Experimenting with epochs - when I doubled the number of epochs, MuRIL improves only slightly (69.5->69.7 on one task)

The point of a benchmark is to run these models through a reasonable and identical process; you can tweak hyperparameters on any model to improve results.

Measuring Gender Bias in Spanish Language Models

Presenter

Nick Doiron, Tufts University / Independent Research

GitHub: https://github.com/mapmeld ; LinkedIn: https://www.linkedin.com/in/nickdoiron/

Context

Related Research

Towards Debiasing Sentence Representations (Liang et al., 2020) English BERT and ELMO

Measuring Bias in Contextualized Word Representations (Kurita et al., 2019) analyzes English BERT with word-association https://aclweb.org/anthology/W19-3823/

Releasing Hindi ELECTRA model

This is a first attempt at a Hindi language model trained with Google Research's ELECTRA. I don't modify ELECTRA until we get into finetuning, and only then because there's hardcoded train and test files

CoLab: https://colab.research.google.com/drive/1R8TciRSM7BONJRBc9CBZbzOmz39FTLl_

Additional background: https://medium.com/@mapmeld/teaching-hindi-to-electra-b11084baab81

It's available on HuggingFace: https://huggingface.co/monsoon-nlp/hindi-bert - sample usage: https://colab.research.google.com/drive/1mSeeSfVSOT7e-dVhPlmSsQRvpn6xC05w

	t5.data.TaskRegistry.add(
	"byt5_ex",
	t5.data.TextLineTask,
	split_to_filepattern={
	"train": "gs://BUCKET/train_lines.txt",
	"validation": "gs://BUCKET/validation_lines.txt",
	},
	text_preprocessor=[
	functools.partial(
	t5.data.preprocessors.parse_tsv,

	from simpletransformers.classification import ClassificationModel

	# set use_cuda=False on CPU-only platforms
	model = ClassificationModel('bert', 'monsoon-nlp/dv-wave', num_labels=8, use_cuda=True, args={
	'reprocess_input_data': True,
	'use_cached_eval_features': False,
	'overwrite_output_dir': True,
	'num_train_epochs': 3,
	'silent': True
	})

	# pip install gdal
	import json
	from osgeo import ogr

	# depends on your shapefile
	target_shapefile = 'tl_2010_sample_shapefile.shp'
	fips_id = 'GEOID10'

	saveblocks = json.loads(open('savefile.json', 'r').read())

	# pip install requests
	import time, json
	import requests

	api_key = "API_KEY_STRING"

	# look up FIPS for state and county:
	# https://www.nrcs.usda.gov/wps/portal/nrcs/detail/national/home/?cid=nrcs143_013697
	state = '12'
	county_fips = ['086']

	# -- coding: utf-8 --
	"""
	Class definition of YOLO_v3 style detection model on image and video
	"""

	import colorsys
	import os
	from timeit import default_timer as timer

	import numpy as np