Sean Dearnaley seandearnaley

Model Name	Article Key	Inference Rate (s)	Valid JSON Rate	Sentiment Variance	Mean Sentiment	Mean Confidence
phi3-4k-sentiment-may-24-2024-2epoches-unsloth.Q4_K_M	f76ab0da	2.37	1.0	0.0	0.61	0.8
phi3-4k-sentiment-may-24-2024-2epoches-unsloth.Q4_K_M	3d6a60c0	1.88	1.0	0.01	-0.46	0.8
phi3-4k-sentiment-may-24-2024-2epoches-unsloth.Q4_K_M	136f0dde	2.62	1.0	0.0	0.01	0.9
phi3-4k-sentiment-may-24-2024-2epoches-unsloth.Q4_K_M	ce972942	1.69	1.0	0.0	0.48	0.77
phi3-4k-sentiment-may-24-2024-2epoches-unsloth.Q4_K_M	028bdea3	2.66	1.0	0.0	-0.41	0.8
phi3-4k-sentiment-may-24-2024-2epoches-unsloth.Q4_K_M	6048aefc	2.7	1.0	0.02	0.56	0.79
phi3-4k-sentiment-may-24-2024-2epoches-unsloth.Q4_K_M	1e2173be	1.99	0.93	0.01	0.57	0.78
phi3-4k-sentiment-may-24-2024-2epoches-unsloth.Q4_K_M	3e1daab6	4.24	1.0	0.11	0.38	0.88
phi3-4k-sentiment-may-24-2024-2epoches-unsloth.Q4_K_M	abc67ae2	2.12	1.0	0.0	0.61	0.8

Comparison	F-Statistic	F-Test P-Value	T-Statistic	T-Test P-Value
llama3_8b-instruct-fp16 vs llama3_8b-instruct-sentiment_analysis-fp16	8.235	0.005	-2.87	0.005
llama3_8b-instruct-q4_K_M vs llama3_8b-instruct-sentiment_analysis-q4_K_M	5.184	0.024	-2.277	0.024
llama3_8b-instruct-q5_K_M vs llama3_8b-instruct-sentiment_analysis-q5_K_M	18.747	0.0	-4.33	0.0
llama3_8b-instruct-q8_0 vs llama3_8b-instruct-sentiment_analysis-q8_0	6.821	0.01	-2.612	0.01
mistral_7b-instruct-fp16 vs llama3_8b-instruct-fp16	1.558	0.214	1.248	0.214
mistral_7b-instruct-q4_K_M vs llama3_8b-instruct-q4_K_M	0.372	0.542	0.61	0.542
mistral_7b-instruct-q5_K_M vs llama3_8b-instruct-q5_K_M	4.425	0.037	2.104	0.037
mistral_7b-instruct-q8_0 vs llama3_8b-instruct-q8_0	0.712	0.4	0.844	0.4
dolphin-mistral_7b-v2.8-fp16 vs mistral_7b-instruct-fp16	0.007	0.935	0.081	0.935

Model Name	Article Key	Inference Rate (s)	Valid JSON Rate	Sentiment Variance	Mean Sentiment	Mean Confidence
llama3_8b-instruct-sentiment_analysis-q4_K_M	63f711be	5.41	1.0	0.01	0.33	0.78
llama3_8b-instruct-sentiment_analysis-q4_K_M	1ed47412	1.9	1.0	0.02	-0.14	0.6
llama3_8b-instruct-sentiment_analysis-q4_K_M	5852d8b8	2.3	1.0	0.01	-0.53	0.8
llama3_8b-instruct-sentiment_analysis-q4_K_M	6f5003f2	4.02	1.0	0.06	0.56	0.81
llama3_8b-instruct-sentiment_analysis-q4_K_M	7be9fbd1	4.4	1.0	0.0	0.6	0.8
llama3_8b-instruct-sentiment_analysis-q4_K_M	7fe80a1c	2.68	1.0	0.0	0.6	0.8
llama3_8b-instruct-sentiment_analysis-q4_K_M	bcdd3918	3.44	1.0	0.01	0.04	0.64
llama3_8b-instruct-sentiment_analysis-q4_K_M	302c8a97	1.51	1.0	0.0	0.0	0.71
llama3_8b-instruct-sentiment_analysis-q4_K_M	7be44b2c	1.54	1.0	0.26	-0.49	0.71

	FROM ./llama3-8b-sentiment-may-3-2024-unsloth.Q4_K_M.gguf
	SYSTEM """
	You are an advanced AI assistant created to perform sentiment analysis on text. Your task is to carefully read the text and analyze the sentiment it expresses towards the potential future stock value of any company mentioned. Analyze the sentiment of this text and respond with the appropriate JSON:
	"""
	TEMPLATE """{{ if .System }}<\|start_header_id\|>system<\|end_header_id\|>

	{{ .System }}<\|eot_id\|>{{ end }}{{ if .Prompt }}<\|start_header_id\|>user<\|end_header_id\|>

	{{ .Prompt }}<\|eot_id\|>{{ end }}<\|start_header_id\|>assistant<\|end_header_id\|>

	"""WhisperCC binding module"""
	import time
	from pathlib import Path

	from whispercpp import Whisper

	from app.audio_tools import NdArray, convert_to_float_array, decode_audio


	class WhisperTranscriber:

	"""Example of using the Whisper library to detect the language and transcribe."""
	import os
	import sys
	import time
	from typing import Optional

	import torch
	import whisper
	from pytube import YouTube