mutaguchi

llama.cpp serverを使って、入力文字列を翻訳するスクリプト。

使用方法

python LLM_Translator.py 
[-h] 
[-i 入力ファイルパス] [-o 出力ファイルパス] [-p プロンプトファイルパス] [-d 辞書ファイルパス] 
[-u llama.cpp serverのURL] [--quiet] [--no-outfile] [--include-input] [--min-context-count 最小保持ターン数]

mutaguchi

function Filter-Duplication
{
    [CmdletBinding()]
    param (
        [Parameter(ValueFromPipeline)]
        [object[]]
        $InputObject
    )
    begin
    {

mutaguchi

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

models = {
    "stablelm": "stabilityai/japanese-stablelm-base-gamma-7b",
    "chatntq": "NTQAI/chatntq-ja-7b-v1.0",
    "mistral": "mistralai/Mistral-7B-v0.1",
    "starling": "Nexusflow/Starling-LM-7B-beta",
    "antler": "Elizezen/Antler-7B",
}

mutaguchi

How to avoid PSAMSIMethodInvocationLogging. In the previously introduced method, a cmdlet is dynamically generated to execute a method by reflection. In this method, the method is executed by reflection by defining a type for the method invocation and having the Hashtable cast to that type. In the example, with Windows Defender real-time protection enabled, a process that would take up to 35 seconds to complete would take about 1 second using this method.

mutaguchi

from transformers import GPTJForCausalLM, AlbertTokenizer
import torch

model = 'AIBunCho/japanese-novel-gpt-j-6b'
tokenizer = AlbertTokenizer.from_pretrained(model, keep_accents=True, remove_space=False)
model = GPTJForCausalLM.from_pretrained(
    model, 
    load_in_4bit = True,
    torch_dtype = torch.bfloat16, 
    device_map = 'auto')

mutaguchi

Mitigating the Impact of PSAMSIMethodInvocationLogging on PowerShell Performance: An Exploration of the Invoke-Method Cmdlet

Method execution in PowerShell 7.3 and later has been slowed down due to PSAMSIMethodInvocationLogging. This feature executes the AMSI's Logging method by specifying the method information and arguments to be executed just before method execution. Originally, this was an experimental feature, but it has been promoted to an official feature.

The problem of slow operation has been partially solved, but still, with Windows Defender's real-time protection enabled, especially when the argument size is large, the execution of methods within a loop is extremely slow. PowerShell/PowerShell#19431

(Measure-Command{$a='x';$b='a'*1000;foreach($i in (1..1000000)){$y=$a.Contains($b)}}).TotalSeconds

mutaguchi

function Test-All
{
    [CmdletBinding()]
    param(
        [scriptblock]
        $Predicate,
        [Parameter(ValueFromPipeline)]
        [PSObject]
        $InputObject
    )

mutaguchi

# based on StableLM chat
# https://huggingface.co/spaces/stabilityai/stablelm-tuned-alpha-chat

import gradio as gr
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline, StoppingCriteria, StoppingCriteriaList, TextIteratorStreamer
import time
import numpy as np
from torch.nn import functional as F
import os

mutaguchi

japanese-gpt-neox-3.6b-instruction-sftを使ったチャットサンプル

https://huggingface.co/rinna/japanese-gpt-neox-3.6b-instruction-sft

python rinna_chat.py で実行できます。

rinna_chat.py -p test.json のように会話設定JSONを指定することもできます。

mutaguchi

$env:OPENAI_API_KEY = "your api key"
Import-Module PowerGenerativeToolkit
$answers = echo 夜に駆ける ハルジオン あの夢を追って ツバメ たぶん 怪物 ミスタ 群青色 三原色 ラブレタ 優しい彗星 セブンティーン ハルカ もう少しだけ アイドル 海の真ん中に ハルジオン アドベンチャー 大正浪漫 好き 夜に走る ハルジオン あの夢を踏んで ツバメ 多分 怪物さん ミスター 群青 3原色 ラブレター 優しいすい星 セブンティーン 春風 もう少しの間  アイドル  海のまにまに ハルジオン 冒険 大正ロマン 好きだ

$titles = echo 夜に駆ける あの夢をなぞって ハルジオン たぶん 群青 ハルカ 怪物 優しい彗星 もう少しだけ 三原色 ラブレター 大正浪漫 ツバメ ミスター 好きだ 海のまにまに アドベンチャー セブンティーン アイドル

$list = $answers | foreach {
    if ($_ -in $titles) { [pscustomobject]@{"raw" = $_; "title" = $_; "isCorrect" = $true } }
    else { [pscustomobject]@{"raw" = $_; "title" = ""; "isCorrect" = $false } }
}