Skip to content

Instantly share code, notes, and snippets.

View cx0's full-sized avatar
🙊

Onuralp cx0

🙊
View GitHub Profile
@cx0
cx0 / fetch_EGA_files.sh
Last active October 1, 2020 07:17
Fetch a single file (EGAF) or download an entire dataset (EGAD) using EGA credentials.
# check the ports
# openssl s_client -connect ega.ebi.ac.uk:8443
# openssl s_client -connect ega.ebi.ac.uk:8052
# pyEGA3 - EGA python client version 3.4.0
# Python version - 3.7.3
pyega3 -cf default_credential_file.json fetch EGAF00001383154
pyega3 -cf default_credential_file.json fetch EGAD00001000440 --saveto Desktop/ega_output/
#!/usr/bin/bash
# ClinVar weekly updates: https://ftp.ncbi.nlm.nih.gov/pub/clinvar/tab_delimited/
# Download assembly-specific variant annotation (Release date: 2022-09-19)
wget https://ftp.ncbi.nlm.nih.gov/pub/clinvar/tab_delimited/variant_summary.txt.gz
awk '{print "\t"$0"\t"}' acmg.filtered.genes.list \|
rg -zf - variant_summary.txt.gz \|
rg 'GRCh38' \|
rg 'single nucleotide variant' \|
@cx0
cx0 / extract_table_from_pdf.md
Created October 31, 2023 07:03
Extract table from paper using Nougat
# install nougat
pip install "nougat-ocr[api, dataset]"
# crop the table from paper (preserve pdf)
# using default 0.1.0-small model
nougat /tmp/2304.08485.table3.only.pdf -o /tmp/" --markdown
\begin{table}
@cx0
cx0 / beautiful_mnist_torch.py
Created November 18, 2023 12:51
Implementation benchmark: PyTorch against tinygrad
# tinygrad implementation: https://github.com/tinygrad/tinygrad/blob/master/examples/beautiful_mnist.py
%time
import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import datasets
import torchvision.transforms as transforms
from torch.utils.data import DataLoader
from tqdm import trange
@cx0
cx0 / layout.md
Last active February 18, 2024 02:00
Configure launchpad grid layout on macOS

Change the number of columns and rows on the launchpad grid.

defaults write com.apple.dock springboard-columns -int 8
defaults write com.apple.dock springboard-rows -int 8
defaults write com.apple.dock ResetLaunchPad -bool TRUE
killall Dock

Use Default to reset.

@cx0
cx0 / copilot_cli.md
Created February 17, 2024 23:01
GitHub CoPilot in the CLI

Use GitHub Copilot in the command line.

pip uninstall gh
brew install gh
gh auth login
gh extension install github/gh-copilot
gh extension upgrade gh-copilot
gh copilot suggest 'read text file line by line and return the total number of alphanum chars in each line'
@cx0
cx0 / turbo.py
Last active April 9, 2024 21:25
GPT4V turbo performance on images with rotated text
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
model="gpt-4-turbo",
messages=[
{
"role": "user",
"content": [
@cx0
cx0 / gpt-4o.ipynb
Last active May 16, 2024 10:33
Live demo of GPT-4o coding assistant and desktop app
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

User prompt:

You have access to source code for torch.distributed.pipelining package and relevant documentation for implementing pipeline parallelism. Write a function that creates an accurate pipeline_order for Schedule1F1B given world_size and n_microbatches such that _format_pipeline_order will return the correct output. Make sure to provide a minimal example to verify function performs as expected.

CoT:

  • Crafting a pipeline: I’m thinking through a function to set up the pipeline order for Schedule1F1B in PyTorch, aiming to efficiently manage forward and backward passes across stages. Progress is steady, ensuring an interleaved pattern that optimizes pipeline stage utilization.
  • Crafting the schedule: I’m working on a 1F1B pipeline schedule, alternating forward and backward pass microbatches at each stage. Ensuring optimal utilization, I aim to refine a formula for this arrangement.
  • Piecing together: I’m using a well-known formula from “GPipe” and “torch.distributed.pipeline”