I'm working on an image-to-image translation task using aligned/paired images.
The dataset requires preprocessing, including cropping original images into relevant arrays.
nodes: | |
- id: webcam | |
custom: | |
source: https://huggingface.co/datasets/dora-rs/dora-idefics2/raw/main/operators/opencv_stream.py | |
outputs: | |
- image | |
- id: idefics2 | |
operator: | |
python: https://huggingface.co/datasets/dora-rs/dora-idefics2/raw/main/operators/idefics2_op.py | |
inputs: |
""" To use: install LLM studio (or Ollama), clone OpenVoice, run this script in the OpenVoice directory | |
git clone https://github.com/myshell-ai/OpenVoice | |
cd OpenVoice | |
git clone https://huggingface.co/myshell-ai/OpenVoice | |
cp -r OpenVoice/* . | |
pip install whisper pynput pyaudio | |
""" | |
from openai import OpenAI | |
import time |
So you generated 50000 images for computing FID or whatever, and now you want to upload those samples to HF.
You try, but one of the filetransfers fails, and you lose all your progress.
I mean it'd be nice if HF could just… fix this… like, put retries into huggingface-cli upload
instead of just discarding tens of gigabytes of progress… but we live in the world in which we live.
So let's make it easier. instead of 50k small files, let's upload 50 big files. Collate 'em into .tars.
I'm not sure this makes a valid WDS, but it's close; I think you would need to rename the files to 000000.img.png
if you wanted that.
#!/bin/bash | |
# Copied and modified from https://github.com/Technica-Corporation/Tegra-Docker | |
# Copyright (c) 2017, Technica Corporation. All rights reserved. | |
NV_LIBS="/usr/lib/aarch64-linux-gnu \ | |
/usr/local/cuda/lib64 \ | |
/usr/local/cuda \ | |
/usr/src/tensorrt \ | |
/usr/local/cuda-10.0 \ | |
/usr/include \ |
This is an Keras implementation of ResNet-101 with ImageNet pre-trained weights. I converted the weights from Caffe provided by the authors of the paper. The implementation supports both Theano and TensorFlow backends. Just in case you are curious about how the conversion is done, you can visit my blog post for more details.
ResNet Paper:
Deep Residual Learning for Image Recognition.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun
arXiv:1512.03385