This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import datetime | |
| import re | |
| # https://regexlib.com/REDetails.aspx?regexp_id=3344&AspxAutoDetectCookieSupport=1 | |
| # http://stackoverflow.com/questions/3143070/javascript-regex-iso-datetime/7221570#7221570 | |
| re_iso8601date = r'^(?:(?=[02468][048]00|[13579][26]00|[0-9][0-9]0[48]|[0-9][0-9][2468][048]|[0-9][0-9][13579][26])\d{4}(?:(-|)(?:(?:00[1-9]|0[1-9][0-9]|[1-2][0-9][0-9]|3[0-5][0-9]|36[0-6])|(?:01|03|05|07|08|10|12)(?:\1(?:0[1-9]|[12][0-9]|3[01]))?|(?:04|06|09|11)(?:\1(?:0[1-9]|[12][0-9]|30))?|02(?:\1(?:0[1-9]|[12][0-9]))?|W(?:0[1-9]|[1-4][0-9]|5[0-3])(?:\1[1-7])?))?)$|^(?:(?![02468][048]00|[13579][26]00|[0-9][0-9]0[48]|[0-9][0-9][2468][048]|[0-9][0-9][13579][26])\d{4}(?:(-|)(?:(?:00[1-9]|0[1-9][0-9]|[1-2][0-9][0-9]|3[0-5][0-9]|36[0-5])|(?:01|03|05|07|08|10|12)(?:\2(?:0[1-9]|[12][0-9]|3[01]))?|(?:04|06|09|11)(?:\2(?:0[1-9]|[12][0-9]|30))?|(?:02)(?:\2(?:0[1-9]|1[0-9]|2[0-8]))?|W(?:0[1-9]|[1-4][0-9]|5[0-3])(?:\2[1-7])?))?)$' | |
| for yyyy in range(1000, 3001): | |
| for mm in range(1, 13): | |
| for dd in range(1, 32): |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <!DOCTYPE html> | |
| <html lang="en"> | |
| <!-- | |
| - Modeled after PayPal Giving Fund donation form: https://www.paypal.com/fr/fundraiser/charity/4864486 | |
| - Supports preset donation amounts and custom amount | |
| - Contains two HTML forms because cannot put in a single form radio preset amount input and text custom amount input | |
| - Inlines in two copies of business email, currency code, item name (for https://russie-Libertes.org) | |
| - Verification/tracking pixel must be used for method="post" | |
| - Default preset amount is highlighted with a heart symbol | |
| - Unique form ids must be used for several forms |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # git clone https://gist.github.com/vadimkantorov/07a421f676f8af94457732f8e9127dde | |
| # cd 07a421f676f8af94457732f8e9127dde | |
| # uv venv | |
| # uv sync | |
| # source .venv/bin/activate # uv run python | |
| # https://github.com/pytorch/pytorch/issues/160875#issuecomment-4183789164 | |
| # https://github.com/vllm-project/vllm/issues/24218 | |
| [build-system] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # python html2xhtml.py test.html test.xhtml | |
| import html.parser | |
| import xml.dom.minidom | |
| class HTMLToMinidom(html.parser.HTMLParser): | |
| # https://developer.mozilla.org/en-US/docs/Glossary/Void_element | |
| def __init__(self, void_elements = ('area', 'base', 'br', 'col', 'embed', 'hr', 'img', 'input', 'link', 'meta', 'param', 'source', 'track', 'wbr')): | |
| super().__init__() | |
| self.doc = xml.dom.minidom.Document() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| alias cmdline='ps --no-headers -o comm,args -p' | |
| cmdline 1 | |
| # can produce JSON, but without process args | |
| alias notbrokenjson='ps --no-headers -o '\''{"pcpu": %C, "group": "%G", "PPID": %P, "user": "%U", "comm": "%c", "rgroup": "%g", "nice": "%n", "pid": %p, "pgid": %r, "etime": "%t", "ruser": "%u", "time": "%x", "tty": "%y", "vsz": %z}'\'' -p' | |
| notbrokenjson 1 > notbroken.json | |
| # does not remove escape sequences or quotes in args - this will break JSON format for some processes | |
| alias brokenjson='ps --no-headers -o '\''{"pcpu": %C, "group": "%G", "PPID": %P, "user": "%U", "args": "%a", "comm": "%c", "rgroup": "%g", "nice": "%n", "pid": %p, "pgid": %r, "etime": "%t", "ruser": "%u", "time": "%x", "tty": "%y", "vsz": %z}'\'' -p' | |
| brokenjson 1 > brokenjson.json |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # git clone https://gist.github.com/vadimkantorov/fe63f8628ff6cad460e934e1d7ed650b | |
| # cd fe63f8628ff6cad460e934e1d7ed650b | |
| # uv venv | |
| # uv sync | |
| # https://github.com/vllm-project/vllm/pull/20358#issuecomment-3247178818 | |
| # https://github.com/vllm-project/vllm/issues/9244 | |
| # https://github.com/astral-sh/uv/issues/8082 | |
| # https://github.com/vllm-project/vllm/issues/24126 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # https://stackoverflow.com/questions/71944328/how-to-implement-json-format-logs-in-python | |
| import json | |
| import logging | |
| # https://docs.python.org/3/library/logging.html#logrecord-attributes | |
| formatter = type('JsonFormatter', (logging.Formatter, ), dict(format = lambda self, record: json.dumps(dict(time = self.formatTime(record), level = record.levelname, message = record.getMessage(), module = record.module, lineno = record.lineno)) ))() | |
| # simpler version below does not escape quotes in message and does not delete newlines in message | |
| # formatter = logging.Formatter('{\"time\": \"%(asctime)-s\", \"level\": \"%(levelname)-s\", \"message\": \"%(message)s\", \"module\": \"%(module)s\", \"lineno\": %(lineno)d}') |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| echo '{"site": {"hello": "world"}}' > foo.json | |
| echo 'Hello {{ site.hello }}' > foo.txt.j2 | |
| bash j2.sh -t foo.txt.j2 -i foo.json -o foo.txt | |
| cat foo.txt | |
| source function.sh | |
| j2 -t foo.txt.j2 -i foo.json -o foo.txt | |
| cat foo.txt | |
| unset -f j2 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Extracted and simplified the two-level aggregation approach (first, parallel aggregation in blocks, then final sequential aggregation) from https://github.com/volcengine/verl/blob/main/verl/utils/kernel/kernels.py | |
| # Examples of single-level sequential, online aggregation approaches: | |
| # - https://github.com/linkedin/Liger-Kernel/blob/main/src/liger_kernel/ops/cross_entropy.py | |
| # - https://github.com/Dao-AILab/flash-attention/blob/main/flash_attn/ops/triton/cross_entropy.py | |
| # logsumexp_torch has some eager pseudo/code in PyTorch which emulates what Triton does, except that BLOCK_SIZE_M equials to M | |
| # tl.program_id(axis=0).to(tl.int64) is used for https://arxiv.org/abs/2410.10989 and https://github.com/linkedin/Liger-Kernel/blob/05b43a14913ced3776aa3fc50020089b8c0d63c1/src/liger_kernel/ops/cross_entropy.py#L77-L79 | |
| # sample_verl.pt is derived from the inputs (logits = torch.matmul(hidden, weights) uploaded by @WindowsXP-Beta in https://github.com/volcengine/verl/issues/2656#issuecomment-3131136498 ) | |
| # created for |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| nvidia-smi -q -x | grep "</pid>" | tr -d "</pid>\t" |
NewerOlder