trojblue / wacom_widd_decode.md

Created January 20, 2025 03:14

Toy code to read wacom-exported widd files with python, for intuos pro fine-tip pens

code v1:

import json
import struct
import base64
import matplotlib.pyplot as plt
from collections import Counter

def read_widd_file(file_path):

trojblue / llm_lecturer_prompt.md

Last active January 15, 2025 20:52

Let ChatGPT-o1 explain subject matters and complex topics, in an engaging and easily digestible way.

Lecturer Prompt for LLMs

Let ChatGPT-o1 explain subject matters and complex topics, in an engaging and easily digestible way.

To use the prompt, you can either:

Save this this in "Custom instructions" part of ChatGPT / Claude,

Or in a new chat:

Copy the prompt in front, using prompt structure similar to this:

trojblue / argilla_custom-field_s3_access.md

Created October 10, 2024 12:37

Label S3 images with Argilla [WIP]

edit dockerfile to allow host gateway access:

version: '3'
services:
  argilla:
    image: argilla/argilla-server:latest
    ports:
      - "6900:6900"  # Existing Argilla port
    extra_hosts:

trojblue / comfyui_api_example.md

Created June 30, 2024 19:37

enable "dev usage" in comfy settings to export api_workflow.json:

https://github.com/comfyanonymous/ComfyUI/blob/master/script_examples/basic_api_example.py

use the script:

import websocket #NOTE: websocket-client (https://github.com/websocket-client/websocket-client)
import uuid
import json

trojblue / pandas_memory_optimize.md

Last active June 19, 2024 08:07

用几种办法来减少dataframe占用的内存:

去掉信息重复的columns
提前去掉不需要的行
转换数字到最小精度(-50%)
转换Python string (objects)为pyarrow str (-30%)
转换date string为pd datetime (-85%)
转换大量重复出现的string为category (-95%)

trojblue / kedro_dynamic_pipeline.md

Last active June 12, 2024 23:44

using functools.partial to pass in real arguments into kedro:

from functools import partial, update_wrapper
from kedro.pipeline import Pipeline, node

from .nodes import process_todo, DemoMerger


def create_wrapped_partial(func, *args, **kwargs):

trojblue / plot_quadratics.md

Created March 28, 2024 18:26

import numpy as np
import matplotlib.pyplot as plt

def plot_quadratic_coefficients(coefficients):
    """
    Plots y = ax^2 + bx + c for each set of coefficients within specified x and y ranges.

    Parameters:
    - coefficients: dict, a dictionary of coefficient sets with 'a', 'b', and 'c' for each key.

trojblue / visualize_tag_counts.md

Last active March 12, 2024 20:41

用来数出df里某列 tag counts数量, 然后可视化的代码:

def safe_split_tag_str(tag_str, separator=","):
    """
    Splits a tag string into a list of non-empty, whitespace-stripped tag strings.
    """
    if not tag_str:
        return []

trojblue / parquet_splitter_usage.md

Last active March 10, 2024 14:54

(pixiv-data-process/yada/13_pixiv_streamlined.ipynb)

输入一个(本地或者s3地址), 返回包含了所有文件的列表, 上传图片-meta的关系到s3:

(没那么多数据的时候可以直接这么用:)

# https://github.com/troph-team/build-it/blob/f996fe55a6fd2beda9e62a6624be0f0fe2a05848/buildit/sagemaker/parquet_splitter.py#L13
import os
from dataproc3.sagemaker import ParquetSplitter

trojblue / lambda_h100_setup.md

Created December 10, 2023 23:04

nd setup, works on lambda h100 pcie:

conda:

cd ~/ && mkdir -p miniconda3 && wget https://repo.anaconda.com/miniconda/Miniconda3-py310_23.5.2-0-Linux-x86_64.sh -O ./miniconda3/miniconda.sh --no-check-certificate && bash ./miniconda3/miniconda.sh -b -u -p ./miniconda3 && rm ./miniconda3/miniconda.sh && ./miniconda3/bin/conda init bash && source ~/.bashrc  && python -m pip install unibox ipykernel jupyter poetry && python -m ipykernel install --user --name=conda310

nd:

yada trojblue

Lecturer Prompt for LLMs