izikeros

Text Splitter

A Python script for splitting text into parts with controlled (limited) length in tokens. This script utilizes the tiktoken library for encoding and decoding text.

Table of Contents

• Introduction
• Installation

qxcv

# from_gym.py adapted to work with Gymnasium. Differences:
#
# - gym.* -> gymnasium.*
# - Deals with .step() returning a tuple of (obs, reward, terminated, truncated,
#   info) rather than (obs, reward, done, info).
# - Also deals with .reset() returning a tuple of (obs, info) rather than just
#   obs.
# - Passes render_mode='rgb_array' to gymnasium.make() rather than .render().
# - A bunch of minor/irrelevant type checking changes that stopped pyright from
#   complaining (these have no functional purpose, I'm just a completionist who

MihailCosmin

#!/bin/bash

### steps ####
# verify the system has a cuda-capable gpu
# download and install the nvidia cuda toolkit and cudnn
# setup environmental variables
# verify the installation
###

### to verify your gpu is cuda enable check

endes0

This script will create an audio file for each page of a PDF, reading it trought Fastspeech2 using fairseq framework. Perfect for creating audiobooks. It also reads the PDF table of contents and groups the files by the top level charapters. Finally it creates a playlist.

Tested on Ubuntu 22.04.1 and Python 3.10.6

Usage

$ pdfToAudio.py --pdf <your pdf file here>

Requeriments

You should have the conmand pdftotext from poppler-utils installed:

wassname

"""
Sometimes we want to use quantiles loss in machine learning, but the outputs are not ordered. This is sometimes called the quantile crossover problem.
Surely it would help to impose the constraint that the quantiles must be ordered?

What's the best way to do this? 
Well it seems me that we should predict differences from the median,
and apply a softplus to make sure the differences are only in one direction. 

Note this will NOT work for very small target values. Because we are using a softplus the model must output very large
logits to get very small numbers. This means it will have difficulty with small y values. 

danesherbs

# This hook is particularly useful when ablating layers

class ContextHook:
    def __init__(self, layer):
        self.layer = layer

    def __enter__(self):
        self.handle = self.layer.register_forward_hook(self.hook)
        return self

wassname

"""
Implements on disk caching of transformed dataframes
Used on a function that returns a single pandas object,
this decorator will execute the function, cache the dataframe as a pickle
file using the hash of function and subdirectory, and the arguments and filename.
The next time the function runs, if the hashes match what is on disk, the decoratored function will simply load and return
the pickled pandas object.
This can result in speedups of 10 to 100 times, or more, depending on the
complexity of the function that creates the dataframe.

wassname

For your IAM user you get a csv of credentials like this

User name,Password,Access key ID,Secret access key,Console login link
USERNAME,PASSWORD,ACCESS_KEY,SECRET_KEY,https://0123456.signin.aws.amazon.com/console

If your region is sydney (ap-southeast-2) in keepass you enter:

Title: USERNAME/COMPANY

bdsaglam

import pathlib
import hydra

hydra._internal.hydra.GlobalHydra().clear()

config_dir = pathlib.Path('/path/to/configs/')
hydra.experimental.initialize(config_dir=config_dir)

cfg = hydra.experimental.compose(config_file='config.yaml', overrides=[])
print(cfg.pretty())

chausies

import torch as T

def h_poly_helper(tt):
  A = T.tensor([
      [1, 0, -3, 2],
      [0, 1, -2, 1],
      [0, 0, 3, -2],
      [0, 0, -1, 1]
      ], dtype=tt[-1].dtype)
  return [