This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
auto objFromIds = [](similarity::IdType id, const std::vector<similarity::IdType>& input) { | |
return new similarity::Object(id, -1, input.size() * sizeof(similarity::IdType), &input[0]); | |
}; | |
similarity::initLibrary(0, LogChoice::LIB_LOGSTDERR, 0); | |
std::unique_ptr<similarity::Space<float>> space(new similarity::SpaceSparseJaccard<float>()); | |
similarity::ObjectVector data; // doesn't free anything, just a vector of Object* | |
data.push_back(objFromIds(0, {0, 1, 2, 3, 4, 5, 6, 7, 8, 9})); // delete me | |
data.push_back(objFromIds(1, {2, 3, 4, 5, 6, 7})); // delete me |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import numpy as np | |
import sys | |
import yaml | |
import argparse | |
from transformers import BertModel | |
parser = argparse.ArgumentParser(description='Convert Huggingface Bert model to Marian weight file.') | |
parser.add_argument('--bert', help='Path to Huggingface Bert PyTorch model', required=True) | |
parser.add_argument('--marian', help='Output path for Marian weight file', required=True) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Chronic Pain: Nele Has Pain | |
Ever since Nele was seven, her legs and joints have been aching. | |
Every single day. | |
To this day, doctors don't quite understand why. | |
Some believe it's simulated. | |
Someone hits Nele's knee with a hammer, with full force, over and over again. | |
"It's a dull pain," she says, not knowing if "dull" is a good word. | |
She's known this feeling for 26 years. | |
Every hour, every minute, it's there. | |
On bad days, it feels as if a backhoe is driving over Nele's legs. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
use strict; | |
use utf8; | |
binmode(STDIN, ":utf8"); | |
binmode(STDOUT, ":utf8"); | |
sub checkAndPrint { | |
my @doc = @_; | |
my @temp = @doc; | |
shift(@temp); shift(@temp); shift(@temp); shift(@temp); shift(@temp); # remove first 5 lines, seems to be boilerplate and title sections with umlauts |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash -v | |
WORKSPACE=19000 | |
SEED=0 | |
HDFS=/hdfs/$PHILLY_VC/marcinjd | |
MARIAN=$HDFS/bins/marian | |
DATA_DIR=$HDFS/WMT.paracrawl | |
LOG_DIR=$PHILLY_LOG_DIR | |
MODEL_DIR=$PHILLY_MODEL_DIR |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#include <typeinfo> | |
#include <typeindex> | |
#include <type_traits> | |
#include <cstdlib> | |
#include <cstdint> | |
#include <string> | |
#include <tuple> | |
#include <iostream> | |
static constexpr uint32_t crc_table[256] = { |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
""" | |
This is a batched LSTM forward and backward pass | |
""" | |
import numpy as np | |
import code | |
class LSTM: | |
@staticmethod | |
def init(input_size, hidden_size, fancy_forget_bias_init = 3): |