Skip to content

Instantly share code, notes, and snippets.

View JohnGiorgi's full-sized avatar

John Giorgi JohnGiorgi

186 followers · 37 following
View GitHub Profile
@JohnGiorgi
JohnGiorgi / run_seq2seq_qa.py
Last active April 15, 2022 14:24
Adapts the HF Transformers run_seq2seq_qa.py script to be used with HotpotQA
#!/usr/bin/env python
# coding=utf-8
# Copyright 2021 The HuggingFace Team All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
@JohnGiorgi
JohnGiorgi / mixed_precision_trainer.py
Last active February 28, 2020 16:01
A trainer for AllenNLP that supports automatic mixed precision (AMP) training with NVIDIA Apex.
import datetime
import logging
import math
import os
import re
import time
import traceback
from typing import Dict, List, Optional, Tuple, Union, Any
from apex import amp
@JohnGiorgi
JohnGiorgi / bert_snli.py
Last active July 14, 2021 19:32
The accompanying gist for the blog post A Dead Simple Example of Fine-tuning BERT with AllenNLP.
import logging
from typing import Dict
from allennlp.data.dataset_readers import SnliReader
from allennlp.data.dataset_readers.dataset_reader import DatasetReader
from allennlp.data.fields import Field
from allennlp.data.fields import LabelField
from allennlp.data.fields import TextField
from allennlp.data.instance import Instance
from allennlp.data.token_indexers import TokenIndexer
@JohnGiorgi
JohnGiorgi / word_dropout.py
Created August 20, 2019 13:07
A torch module for word dropout. Will randomly replace some words with a specified word (e.g. an UNK token).
import torch
from torch.nn.modules.dropout import _DropoutNd
class WordDropout(_DropoutNd):
"""During training, randomly replaces some of the elements of the input tensor with
`dropout_constant` with probability `p` using samples from a Bernoulli distriution. Each channel
will be zerored out independently on every forward call.
Input is expected to be a 2D tensor of indices representing tokenized sentences.
@JohnGiorgi
JohnGiorgi / biaffine_classifier.py
Last active November 2, 2023 04:58
PyTorch implementation of the biaffine attention operator from "End-to-end neural relation extraction using deep biaffine attention" (https://arxiv.org/abs/1812.11275) which can be used as a classifier for binary relation classification. If you spot an error or have an improvement, let me know!
import torch
class BiaffineAttention(torch.nn.Module):
"""Implements a biaffine attention operator for binary relation classification.
PyTorch implementation of the biaffine attention operator from "End-to-end neural relation
extraction using deep biaffine attention" (https://arxiv.org/abs/1812.11275) which can be used
as a classifier for binary relation classification.
@JohnGiorgi
JohnGiorgi / readme.md
Last active December 20, 2018 18:18
Collection of repositories that implement large language models for fine-tuning
@JohnGiorgi
JohnGiorgi / setting_up_environment_for_fastai.md
Last active November 24, 2018 15:21
@JohnGiorgi
JohnGiorgi / using_compute_canada_resources.md
Last active November 10, 2023 22:44
Helpful tips/tricks for using Compute Canada resources

Using Compute Canada Resources

Basics

You ssh into the systems using:

ssh <username>@<login node>

E.g., to login to Niagra for user example:

@JohnGiorgi
JohnGiorgi / gpu_enabled_tensorflow_enviornment_on_ubuntu_18.04.md
Last active June 5, 2019 16:52
Instructions for setting up a GPU-enabled Tensorflow Environment on Ubuntu 18.04

Setting up a GPU-enabled Tensorflow Enviorment

Setting up GPU support

Training Tensorflow models on a NVIDIA GPU requires NVIDIA drivers, CUDA, and cuDNN. See here for the instructions to install all of these components on Ubuntu 18.04.

Other resources