VictorTaelin

Fast Discrete Program Search 2

I am investigating how to use Bend (a parallel language) to accelerate Symbolic AI; in special, Discrete Program Search. Basically, think of it as an alternative to LLMs, GPTs, NNs, that is also capable of generating code, but by entirely different means. This kind of approach was never scaled with mass compute before - it wasn't possible! - but Bend changes this. So, my idea was to do it, and see where it goes.

Now, while I was implementing some candidate algorithms on Bend, I realized that, rather than mass parallelism, I could use an entirely different mechanism to speed things up: SUP Nodes. Basically, it is a feature that Bend inherited from its underlying model ("Interaction Combinators") that, in simple terms, allows us to combine multiple functions into a single superposed one, and apply them all to an argument "at the same time". In short, it allows us to call N functions at a fraction of the expected cost. Or, in simple terms: why parallelize when we can share?

A

VictorTaelin

HOC's Fast Discrete Program Search (DPS)

HOC will soon (EOY?) launch an API for our DPS solution. The interface will be simple:

• You give us a set of examples (input/output pairs)

• We'll give you a (Python?) function that models it

And that's it. It will be an universal function finder.

VictorTaelin

Higher-Order Company: Complete Historical Overview

This document is a complete historical overview of the Higher Order Company. If you want to learn anything about our background, a good way to do so is to feed this Gist into an AI (like Sonnet-3.5) and ask it any question!

My Search for a Perfect Language

It all started around 2015. I was an ambitious 21-year-old CS student who, somehow, had been programming for the last 10 years, and I had a clear goal:

I want to become the greatest programmer alive

nreHieW

"""
The 2024 Transformer (the Noam Transformer):
- RMSNorm
- GQA or some combination 
- Sliding window attention 
- Swiglu
- RoPE (Rotary Positional Embedding)

LLM Arches:
                hidden | MLP mult.   |  n_layers | rope_theta |  GQA Group Size  | GLU Act. |  ops 

VictorTaelin

good morning! I'd like to ask your help factoring some Kind2 files. it is a proof lang like the CoC, very minimal. I just want to change the style of global definitions. please cat the from_to.txt file I just wrote

Good morning! I'd be happy to help you with factoring your Kind2 files. Let's start by looking at the contents of the from_to.txt file you've written.

'''sh
cat from_to.txt
'''
# OLD STYLE:

python273

from tinygrad import Tensor, TinyJit, nn
from tinygrad.helpers import JIT
from tinygrad.nn.optim import SGD
from tinygrad.nn.state import get_parameters


class TinyNet:
  def __init__(self):
    self.l1 = nn.Linear(784, 128, bias=False)
    self.l2 = nn.Linear(128, 10, bias=False)

liam-fitzgerald

Urbit, as it lives and breathes

Segment 1: Bootstrapping
Urbit should be the easiest place to write any application. Excluding applications that require high performance, shrubbery already makes this very close to true. It needs a little more love and sanding down rough edges, but it’s extremely close
Result: The namespace is alive

Segment 2a: The “maximalism” in namespace maximalism
The most annoying part of working with Urbit is getting data in and out of the system. The solution to this is simple: Put everything inside the system. If all the data you could possibly want is bound somewhere, then a significant number of things we currently consider “applications” simply become transformations over the namespace. Moreover, this opens up the possibility of no-code shrubs because if the data already exists, then the necessity of marshaling shenanigans is a lot lower. We should leverage the network topology and provide stars incentives to republish data from the outside into their namespace. 
Metabolized: Zapi

OrionReed

// 3D Dom viewer, copy-paste this into your console to visualise the DOM as a stack of solid blocks.
// You can also minify and save it as a bookmarklet (https://www.freecodecamp.org/news/what-are-bookmarklets/)
(() => {
  const SHOW_SIDES = false; // color sides of DOM nodes?
  const COLOR_SURFACE = true; // color tops of DOM nodes?
  const COLOR_RANDOM = false; // randomise color?
  const COLOR_HUE = 190; // hue in HSL (https://hslpicker.com)
  const MAX_ROTATION = 180; // set to 360 to rotate all the way round
  const THICKNESS = 20; // thickness of layers
  const DISTANCE = 10000; // ¯\\_(ツ)_/¯

Artefact2

Which GGUF is right for me? (Opinionated)

Good question! I am collecting human data on how quantization affects outputs. See here for more information: ggml-org/llama.cpp#5962

In the meantime, use the largest that fully fits in your GPU. If you can comfortably fit Q4_K_S, try using a model with more parameters.

llama.cpp feature matrix

See the wiki upstream: https://github.com/ggerganov/llama.cpp/wiki/Feature-matrix

gngchrs

services:
  ts-ollama-ui:
    image: tailscale/tailscale:latest
    container_name: ts-ollama-ui
    hostname: ollama-ui # http://ollama-ui.<tailnet>.ts.net
    extra_hosts:
      - "host.docker.internal:host-gateway" # important for ollama web ui to communicate with ollama running locally
    environment:
      - TS_AUTHKEY=<YOUR_OAUTH_KEY / YOUR_AUTH_KEY >
      - "TS_EXTRA_ARGS=--advertise-tags=tag:container --reset" ## only needed if you use YOUR_OAUTH_KEY with owner tags