This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import io | |
| import tarfile | |
| import random | |
| from collections import defaultdict | |
| from lxml import etree | |
| import uuid | |
| from PIL import Image, ImageDraw | |
| from glob import glob | |
| import time | |
| import os |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import sys | |
| fd = open(sys.argv[2], "w") | |
| lines = open(sys.argv[1]).readlines() | |
| for l in lines: | |
| toks = l.split(" ") | |
| t = toks[0] | |
| image_id = t.split("#")[0] | |
| caption = " ".join(toks[1:]) | |
| L = f"{image_id}.jpg,{caption}" | |
| fd.write(L) |
We can't make this file beautiful and searchable because it's too large.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| language,acc1,model,mean_per_class_recall,dataset,pretrained,image_retrieval_recall@5,acc,mean_average_precision,acc5,task,model_fullname,text_retrieval_recall@5 | |
| en,0.18418107200596942,RN50-quickgelu,0.18626162135734856,cars,cc12m,,,,0.5200845665961945,zeroshot_classification,RN50-quickgelu cc12m, | |
| en,0.17945529163039423,RN50,0.1817410995349502,cars,cc12m,,,,0.5139907971645318,zeroshot_classification,RN50 cc12m, | |
| en,0.7211789578410646,ViT-B-16,0.7212942008721658,cars,commonpool_l_basic_s1b_b8k,,,,0.9759980101977366,zeroshot_classification,ViT-B-16 commonpool_l_basic_s1b_b8k, | |
| en,0.8166894664842681,ViT-B-16,0.8155803538003477,cars,commonpool_l_clip_s1b_b8k,,,,0.9907971645317747,zeroshot_classification,ViT-B-16 commonpool_l_clip_s1b_b8k, | |
| en,0.6628528789951499,ViT-B-16,0.6613039344249736,cars,commonpool_l_image_s1b_b8k,,,,0.9621937569953986,zeroshot_classification,ViT-B-16 commonpool_l_image_s1b_b8k, | |
| en,0.6775276706877255,ViT-B-16,0.6744836925516351,cars,commonpool_l_laion_s1b_b8k,,,,0.9602039547319985,zeroshot_cla |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import ast | |
| import json | |
| import logging | |
| import math | |
| import os | |
| import random | |
| import sys | |
| import braceexpand | |
| from dataclasses import dataclass | |
| from multiprocessing import Value |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Disable Laptop Keyboard in Ubuntu | |
| Taken From :- https://blog.hostonnet.com/laptop-keyboard-ubuntu | |
| To disable laptop built in keyboard in Ubuntu, first you need to identify the device id for keyboard. | |
| Now run xinput –list command to get list of all input devices | |
| hostonnet@pc4:~$ xinput --list | |
| ⎡ Virtual core pointer id=2 [master pointer (3)] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| job_id_regexp: "Job Id:(\\d+)" | |
| cmd: "sbatch {sbatch_script}" | |
| check_interval_secs: 600 | |
| partition: booster | |
| account: laionize | |
| experiments: | |
| small: | |
| model_scale: | |
| model: [ViT-B-32] | |
| samples_seen_scale: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash -x | |
| #SBATCH --account={account} | |
| #SBATCH --nodes={nodes} | |
| #SBATCH --gres=gpu:4 | |
| #SBATCH --ntasks-per-node=4 | |
| #SBATCH --cpus-per-task=24 | |
| #SBATCH --time=06:00:00 | |
| #SBATCH --partition={partition} | |
| #SBATCH --output={output_file} | |
| echo "Job Id:$SLURM_JOB_ID" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| checkpoints: | |
| checkpoint_interval: 10 | |
| checkpoints_path: checkpoints | |
| checkpoints_path_is_shared_file_system: false | |
| resume_checkpoint_path: null | |
| save_initial_state: false | |
| data_stages: | |
| - name: Stable Training Stage | |
| start_training_step: 1 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| #SBATCH --nodes=8 | |
| #SBATCH --time=00:20:00 | |
| #SBATCH --ntasks-per-node=1 | |
| #SBATCH --cpus-per-task=48 | |
| #SBATCH --gres=gpu:4 | |
| #SBATCH --partition=booster | |
| #SBATCH --account=transfernetx | |
| #SBATCH --exclude=jwb[0059,0067,0069,0193,0284,0287,0294,0359,0418,0637,0829,0832,0838,0898,0907,0921,0971,1004,1023,1029,1213,1126] | |
| #SBATCH --threads-per-core=1 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| #SBATCH --nodes=8 | |
| #SBATCH --time=00:20:00 | |
| #SBATCH --ntasks-per-node=1 | |
| #SBATCH --cpus-per-task=48 | |
| #SBATCH --gres=gpu:4 | |
| #SBATCH --partition=booster | |
| #SBATCH --account=transfernetx | |
| #SBATCH --exclude=jwb[0059,0067,0069,0193,0284,0287,0294,0359,0418,0637,0829,0832,0838,0898,0907,0921,0971,1004,1023,1029,1213,1126] | |
| #SBATCH --threads-per-core=1 |