This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from dagster import Out, fs_io_manager, job, op, DynamicOutput,DynamicOut | |
from dagster_aws.s3 import s3_pickle_io_manager, s3_resource | |
@op(out=DynamicOut(io_manager_key="s3_io")) | |
def files_in_directory(context): | |
for f in ["1","2","3"]: | |
yield DynamicOutput( | |
value=f, | |
# create a mapping key from the file name |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import json | |
from pathlib import Path | |
from datasets import load_dataset | |
import random | |
ds = load_dataset("id_panl_bppt") | |
tr_data = [] | |
val_data = [] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
TATOEBA_RELEASED_BT = https://object.pouta.csc.fi/Tatoeba-MT-bt/released-data.txt | |
tatoeba-all-bt: | |
for b in ${shell wget -qq -O - ${TATOEBA_RELEASED_BT} | grep -v '.txt' | cut -f1 -d'/' | sort -u}; do \ | |
s=`echo $$b | cut -f1 -d'-'`; \ | |
t=`echo $$b | cut -f2 -d'-'`; \ | |
echo "${MAKE} -C bt-tatoeba SRC=$$s TRG=$$t fetch-bt"; \ | |
echo "${MAKE} MODELTYPE=transformer-align HPC_CORES=2 HPC_MEM=32g tatoeba-$${t}2$${s}-train-bt.submitcpu"; \ | |
done | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# undo git add | |
git rm --cached -r dir | |
# apply patches | |
git diff master Branch1 > ../patchfile | |
git checkout Branch2 | |
git apply ../patchfile |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import torch | |
from transformers import AutoTokenizer, T5ForConditionalGeneration | |
passages = [['7744105', 'For Earth-centered it was Geocentric Theory proposed by greeks under the guidance of Ptolemy and Sun-centered was Heliocentric theory proposed by Nicolas Copernicus in 16th century A.D. In short, Your Answers are: 1st blank - Geo-Centric Theory. 2nd blank - Heliocentric Theory.'], ['2593796', 'Copernicus proposed a heliocentric model of the solar system â\x80\x93 a model where everything orbited around the Sun. Today, with advancements in science and technology, the geocentric model seems preposterous.he geocentric model, also known as the Ptolemaic system, is a theory that was developed by philosophers in Ancient Greece and was named after the philosopher Claudius Ptolemy who lived circa 90 to 168 A.D. It was developed to explain how the planets, the Sun, and even the stars orbit around the Earth.'], ['6217200', 'The geocentric model, also known as the Ptolemaic system, is a theory that was developed by |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from datetime import datetime, time | |
# FastAPI and Pydantic Related Libraries | |
from time import sleep | |
from apscheduler.events import EVENT_JOB_ERROR | |
from fastapi import FastAPI | |
from pydantic import BaseModel, Field | |
from typing import List |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
https://www.funkyspacemonkey.com/how-to-install-macos-big-sur-with-opencore-on-linux | |
git clone https://github.com/kholia/OSX-KVM.git macOS-Big-Sur | |
./fetch-macOS.py --big-sur | |
3. Install xar. Arch and derivates users can install xar from the AUR. Now we need to extract SharedSupport.dmg from InstallAssistant.pkg. | |
If you’ve installed xar from AUR, run: xar -xf InstallAssistant.pkg | |
4. Install darling-dmg. If you’re running Arch or derivates, you can install the package from the AUR. We will need it to mount SharedSupport.dmg | |
If you’ve installed the package from the AUR, run: darling-dmg SharedSupport.dmg temp | |
5. Now that SharedSupport.dmg is mounted we need to extract BaseSystem.dmg from it. The file we need is located inside a .zip file, so we’ll use 7zip to extract it. To do that, run: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import tensorflow as tf | |
import math | |
from functools import partial | |
import numpy as np | |
def get_shape_list(tensor, expected_rank=None, name=None): | |
"""Returns a list of the shape of tensor, preferring static dimensions. | |
Args: | |
tensor: A tf.Tensor object to find the shape of. | |
expected_rank: (optional) int. The expected rank of `tensor`. If this is |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{"lastUpload":"2020-10-25T13:35:37.349Z","extensionVersion":"v3.4.3"} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# ideavim | |
set visualbell | |
set noshowmatch | |
set relativenumber | |
set nohlsearch | |
set hidden | |
set noerrorbells | |
set tabstop=4 softtabstop=4 | |
set shiftwidth=4 |
NewerOlder