This is a report on my experience pair programming with Bard on a neural network task that challenged it to its current limits.
Bard now has the ability to program, or put another way Google has removed the gating that blocked it from trying.
All the code in this article is basically 99% produced by Bard. I either prompted it to refactor things or I just tweaked one line or two lines of every 100.
Note: I used gpt-4 a little bit too, for the training part, but this is mostly Bard.
Could an LLM end up being the core part of a dangerous computer worm?
How would we neutralize such a thing if this happened?
There is a hilarious story from https://users.cs.utah.edu/~elb/folklore/xerox.txt about an early computer virus called robin hood and friar tuck. This was basically just two programs running on a UNIX system that would look out for each other and reboot the other process if it was killed. It's interesting to note that since computer programs run thousands of times faster than humans, a human can't type kill -9 robinhood then type kill -9 friartuck in time. The computer is faster so it always wins if you try this. To defeat this you need to take a different approach than speed.
The pytorch (neural network library) examples include a script to try out the training process for MNIST digit recognition data set: https://github.com/pytorch/examples/tree/main/mnist
This builds up a convolutional neural network that takes one of these pictures and processes it down to 10 neurons. The training process uses two sets of labelled data (examples of pictures of digits and which of the 10 possible digits they are): One training set and one testing set. The training set is used to manipulate all of the "weights" inside the neural network by moving in the (very high dimensional) direction of fastest descent, aiming to get the output neurons to produce the intended label given the input picture. The testing set is used as a metric to say how well the neural network is doing.
I ran this, creating mnist_cnn.pt with 99% accuracy on the test data set.
Then I wanted to see if it worked, so I drew images of all 10 digits. There was no way to try this out so I wrote the attach
This blog post is about the Linear Least Squares Problem. This method is credited back to Legendre and Gauss, some of my favorite mathematicians. Why are they such inspiring people? Here is a passage from a post that goes into more depth about Gauss's application of least squares:
The 24-year-old Gauss tackled the orbit problem, assuming only Kepler’s three laws of planetary motion, with his newly discovered error distributions and his method of least squares for three months. He spent over 100 hours performing intensive calculations by hand without any mistakes (and without the luxury of today’s computers!). He had to estimate the six parameters of the orbit (as shown in Figure 7) from only 19 data points, subject to random measurement errors. He even invented new techniques such as the Fast Fourier Transform for interpolating trigonometric series, which produced efficient numerical approximations o
I have included working code examples that can be run throughout, as well as graphs. I hope this helps make this easier to understand in a more hands on way.
Suppose you know that there are 10 balls in an urn, some are red and some are blue. So there are 11 different possible models for this situation:
Take a number, square it, the result is non-negative. Because positive * positive = positive and negative * negative is positive. Or
But someone wanted to take square roots of negative numbers, so they did, and called it 'i'.
The solution, to put imaginary and complex numbers on a solid foundation is something called a ring quotient. What you do is you start with the ring (meaning number system) of polynomials over the real numbers
The target audience for this is people who are beginners at software engineering and using linux. A lot of the information here may be obvious or already known to you. The language involved is C but you do not need to know any C to read this tutorial. I used mg to write this blog post. I used vs code to edit the source code.
This post is also available on gopher://tilde.team:70/0/~river/tweak-free-software
If you use a piece of free software and it's 99% perfect but there's just this one thing it does that annoys the hell out of you.. you can in theory just fix it! Here's a look at what doing that is like. Hopefully it inspires you, or you pick up a could tricks on the way!
