This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| /* | |
| Find and remove outliers per group using the interquartile range with SQLite queries. | |
| */ | |
| -- create table with fake data | |
| CREATE TEMPORARY TABLE test_table (group_id INTEGER, variable REAL); | |
| INSERT INTO test_table | |
| VALUES | |
| (1, 2), | |
| (1, 4), |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import pandas as pd | |
| def anti_join(x, y, on): | |
| """Return rows in x which are not present in y""" | |
| ans = pd.merge(left=x, right=y, how='left', indicator=True, on=on) | |
| ans = ans.loc[ans._merge == 'left_only', :].drop(columns='_merge') | |
| return ans | |
| def anti_join_all_cols(x, y): |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| ## working dir at file location | |
| cd "$(dirname "$0")" | |
| ## train the model | |
| echo "[+] Fitting a glmnet model on labeled data" | |
| Rscript 'learning.R' | |
| echo "[+] Finished fitting the model" | |
| ## read in the ambiguous cases | |
| TO_RECODE=`cat ambiguous.txt` | |
| echo "[+] Recoding ambiguous" |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| """Summary | |
| """ | |
| import logging | |
| from pathlib import Path | |
| import fire | |
| from datasets import Dataset, load_dataset | |
| from tqdm.auto import tqdm | |
| from transformers import AutoTokenizer |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| class countCalls(object): | |
| """Decorator that keeps track of the number of times a function is called. | |
| :: | |
| >>> @countCalls | |
| ... def foo(): | |
| ... return "spam" | |
| ... | |
| >>> for _ in range(10) | |
| ... foo() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from functools import wraps | |
| from typing import Any, Callable, TypeVar | |
| F = TypeVar("F", bound=Callable[..., Any]) | |
| def log_variables(func: F) -> F: | |
| @wraps(func) | |
| def wrapper(*args, **kwargs) -> Any: | |
| if not hasattr(wrapper, "initialized"): |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import sys | |
| from functools import wraps | |
| from typing import Any, Callable, Optional, TypeVar | |
| F = TypeVar("F", bound=Callable[..., Any]) | |
| def log_variables(func: F) -> F: | |
| @wraps(func) | |
| def wrapper(*args: Any, **kwargs: Any) -> Any: | |
| def tracer(frame, event, arg) -> Optional[Callable]: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import inspect | |
| import logging | |
| from functools import wraps | |
| def inject_logger(method): | |
| @wraps(method) | |
| def wrapper(*args, **kwargs): | |
| frame = inspect.currentframe() |