fredriccliver’s gists

fredriccliver / content-extraction-comparison.txt

Created November 24, 2024 18:13

	URL: https://www.bbc.com/news/articles/c93716xdgzqo


	=== Mozilla Readability ===
	{
	"title": "Elon Musk adds Microsoft to lawsuit against ChatGPT-maker OpenAI",
	"excerpt": "Mr Musk, a co-founder of the company, is accusing it and the tech giant of operating a monopoly.",
	"length": 2617,
	"content": "Elon Musk has stepped up his ongoing feud with ChatGPT-maker OpenAI with a revived lawsuit against the firm, adding Microsoft as a defendant.Mr Musk, a co-founder of the artificial intelligence (AI) company, accused it and the tech giant of operating a monopoly in an amended legal complaint on Thursday.It follows previous lawsuits accusing the firm of breaching the principles he agreed to when he helped found it in 2015.Microsoft declined to comment on the lawsuit.An OpenAI spokesperson said Mr Musk's refreshed complaint was \"baseless\".\"Elon’s third attempt in less than a year to reframe his claims is even more baseless and overreaching than the previous ones,\" they told the BBC.They added that previous

fredriccliver / content-extraction-comparison.csv

Created November 24, 2024 18:11

Web Content Extraction for LLM Context Augmentation: A Comparative Analysis

Feature	Mozilla Readability	Article-extractor	Node-unfluff	HTML-to-text
Content Quality	Excellent - clean and complete	Very Good - with HTML markup	Good - some formatting issues	Poor - noisy
Structure Preservation	✅ Excellent hierarchy	✅ Good with HTML tags	⚠️ Partial	❌ Poor
Formatting	✅ Clean paragraphs and headers	✅ HTML formatted	⚠️ Inconsistent	❌ Many line breaks
Metadata	Title/excerpt	Title/author/date	Title/description/author/date	None
Navigation/UI Removal	✅ Complete removal	✅ Good removal	⚠️ Some remnants	❌ Contains UI elements
Signal-to-noise Ratio	High	High	Medium	Low
Text Cleanliness	Very clean	Clean with HTML	Moderately clean	Very noisy
Link Handling	✅ Preserved appropriately	✅ HTML links	⚠️ Raw URLs	❌ Raw URLs scattered
Output Format	Plain text with structure	HTML	Plain text	Plain text

fredriccliver / transcribe_audio_replicate.py

Created November 23, 2024 09:41

Fast and Cost efficient transcription with Whisper and Replicate

	import os
	import json
	import logging
	import replicate
	from pathlib import Path
	from time import time as get_time
	from typing import Optional, Dict, Any
	from dotenv import load_dotenv
	from pydub import AudioSegment
	import tempfile

fredriccliver / dtw.py

Created February 29, 2024 10:02

calculate utterance similarity using DTW algorithm.

	from scipy.spatial.distance import euclidean
	from fastdtw import fastdtw
	import numpy as np

	def calculate_similarity_percentage(distance, max_distance):
	similarity_percentage = (1 - distance / max_distance) * 100
	return similarity_percentage

	def main():
	test_sets = [

fredriccliver / custom_logger_example.ts

Created August 17, 2023 12:52

	// logger.ts

	export function logError(message: string, ...optionalParams: any[]): void {
	try {
	// Get the caller function
	const caller = logError.caller?.name \|\| 'anonymous';

	// Extract filename from the stack trace
	const stack = new Error().stack \|\| '';
	const fileMatch = stack.split('\n')[2]?.match(/(?:http:\/\/\|https:\/\/\|file:\/\/\/).?\/(.?)(?:\?\|:)/) \|\| [];

fredriccliver / keywordExtraction.js

Created January 18, 2022 00:31

	/**
	*
	* Usage
	*
	* node getRemote.js [target url]
	*
	*/


	// import { readSync } from "to-vfile";

fredriccliver / index.js

Created January 17, 2022 07:28

	// e.g.
	// node index.js https://googe.com


	import { readSync } from "to-vfile";
	import { toString } from "nlcst-to-string";
	import { retext } from "retext";
	import retextPos from "retext-pos";
	import retextKeywords from "retext-keywords";
	import fetch from "node-fetch";

fredriccliver / presentingUserInfo.swift

Created October 9, 2021 06:08


	import UIKit
	import FirebaseAuth
	import Kingfisher

	class NextVC: UIViewController {

	@IBOutlet var profileImageView: UIImageView!
	@IBOutlet var nameLabel: UILabel!

fredriccliver / weather.html

Created September 29, 2021 08:15

	<html>
	<head>
	<link
	href="https://unpkg.com/tailwindcss@^2/dist/tailwind.min.css"
	rel="stylesheet"
	/>
	</head>
	<body>
	<div class="">
	<img

fredriccliver / fetch_weather.js

Created September 29, 2021 07:43

	var weatherApiKey = "YOUR API KEY"

	var destinationUrl = (cityName) => {
	return `http://api.openweathermap.org/data/2.5/weather?q=${cityName}&appid=${weatherApiKey}`
	}

	fetch(destinationUrl(city))
	.then((response) => response.json())
	.then((data) => {
	console.log(data)

Fredric Cliver fredriccliver