Skip to content

Instantly share code, notes, and snippets.

@navicore
Last active April 3, 2023 03:24
Show Gist options
  • Save navicore/616951da7d6acdb4291e07853e321959 to your computer and use it in GitHub Desktop.
Save navicore/616951da7d6acdb4291e07853e321959 to your computer and use it in GitHub Desktop.
Thoughts on LLMs March 2023

Thoughts on LLMs March 2023

Exciting times!

Fantastic tools I use myself constantly as of December '22.

I agree this is world changing on the scale of the PC and the the Internet.

But

However, drilling into the GUI analogy, remember all real computer work that mattered was done in text screens for 20 more years after it was productized at scale. 3270 and VT100. No real work was done off of the mainframe by general purpose computers (windiws, *nix) until post 2000, banking, insurance, air travel, government systems (IRS), things that made the world turn. Because correctness is hard - working systems are hard and complex working systems are rare and are never made deliberately all at once.

And Scary

Our first encounter with new technology of this significance nearly killed us twice. The nuke and social media. We should be very afraid after seeing how the social media first contact enounter went.

But Not Scary

It is not scary because of AGI. This is nonesense. The concern that we are about to make an intelligent being smarter than we are is complete nonesense.

It is scary because people won't know how to use it and will mistake it for something that has reasoning and goals - but there is no prolog basis here, no motivation for anything - it is mearly predicting tokens based on statistical measures of what humanity has shared on the internet in the past, hence the halucinations and confidence / gas-lighting.

The insistance that we're months away from AGI is no different than religious explanations and "intelligent design / irriducibly complex" nonesense based on "I don't understand it so ... 'god'". The fact is we do understand why LLMs work and that we are surprised by how good they are isn't any sign of AGI. We know from how we built them that they are not deterministic - so it isn't "explainable AI" the way self-driving cars and other kinds of AI will need to be but we do know how they work.

See Prasanta's post for more.

Also, google "Stochastic Parrots".

FWIW, I do think any programmer can be much more productive keeping a chatGPT session going to get another POV and to flatten learning curves where the time spent context switching in training materials vs real work is flattened. For now, you still have to learn the language or API but don't have to translate context from your own work to a widget factory example. In the near future many of us will be able to do real work with just prompts but not just yet.

Corporate IT is so comically inefficient I don't think there is any immediate threat to enterprise coders who ignore LLMs but I don't think any green field work can be done in a competative way w/o coders embracing this new tool.

Back to the AGI nonesense - this is some dark bullshit that is serviing the AI hype machine and distracting us from the real danger of our amping up these systems. We need regulations from the public sector (while there still is one) to bring transparency to how we use this stuff and to addrss the stealing of IP and to watermark the synthetic so we can know what's real.

Addendum

from chatGPT4 itself:

While it is true that artificial neural networks (ANNs) are inspired by the structure and function of biological neural networks in the human brain, the similarities are quite limited. There are several reasons to think that computer scientists might be overstating the correlation between ANNs and the human brain:

Complexity: The human brain contains around 100 billion neurons and an estimated 100 trillion synapses, while even the largest ANNs contain only a fraction of that number. Additionally, the interactions and functions of biological neurons are far more complex than those in ANNs.

Learning mechanisms: ANNs primarily rely on backpropagation and gradient descent to learn from data, which involves adjusting the weights of connections between artificial neurons. In contrast, the human brain learns through a variety of processes, including synaptic plasticity, homeostatic plasticity, and neuromodulation, which are not yet fully understood or replicated in ANNs.

Connectivity: In the human brain, neurons are connected in intricate and highly specific patterns, whereas connections in ANNs are often more uniform and less specialized.

Neuronal diversity: Biological neural networks exhibit a wide variety of neuron types and specialized cells, each with unique properties and functions. ANNs typically employ a much more homogeneous and simplified architecture.

Energy efficiency: The human brain is remarkably energy-efficient, consuming about 20 watts of power, while ANNs often require vast amounts of computational resources and energy.

Adaptability and plasticity: The human brain is capable of adapting to new situations and recovering from damage, whereas ANNs are generally less flexible and adaptive.

Embodiment: The human brain is integrated with and influenced by the body and sensory inputs, while ANNs are often disembodied and rely on preprocessed data.

Despite these differences, ANNs have proven to be powerful tools for solving complex computational problems, and their development has led to valuable insights into human cognition and neuroscience. However, it is important to recognize that ANNs are a simplified model of the human brain and not a direct representation of its full complexity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment