yoavg

Some remarks on Large Language Models

Yoav Goldberg, January 2023

Audience: I assume you heard of chatGPT, maybe played with it a little, and was imressed by it (or tried very hard not to be). And that you also heard that it is "a large language model". And maybe that it "solved natural language understanding". Here is a short personal perspective of my thoughts of this (and similar) models, and where we stand with respect to language understanding.

Intro

Around 2014-2017, right within the rise of neural-network based methods for NLP, I was giving a semi-academic-semi-popsci lecture, revolving around the story that achieving perfect language modeling is equivalent to being as intelligent as a human. Somewhere around the same time I was also asked in an academic panel "what would you do if you were given infinite compute and no need to worry about labour costs" to which I cockily responded "I would train a really huge language model, just to show that it doesn't solve everything!". We

Spuffynism

Solving a Dungeons & Dragons riddle using prolog

Bringing back the magic of Christmas using the magic of prolog

As part of a holiday D&D one-shot session where Santa Claus's toy factory had been sabotaged, our dungeon master presented to us, a group of Christmas elves, a riddle to solve.

9 cards, labeled with the names of Santa's reindeer were presented to us. The instructions indicated that we had to find the order reindeer were in, according to this riddle:

Vixen should be behind Rudolph, Prancer and Dasher, whilst Vixen should be in front of Dancer and Comet. Dancer should be behind Donder, Blitzen and Rudolph. Comet should be behind Cupid, Prancer and Rudolph. Donder should be behind Comet, Vixen, Dasher, Prancer and Cupid. Cupid should be in front of Comet, Blitzen, Vixen, Dancer and Rudolph. Prancer should be in front of Blitzen, Donder and Cupid. Blitzen should be behind Cupid but in front of Dancer, Vixen and Donder. Rudolph should be behind Prancer but in front of Dasher, Dancer and Dond

veekaybee

ChatGPT Resources

Context

ChatGPT appeared like an explosion on all my social media timelines in early December 2022. While I keep up with machine learning as an industry, I wasn't focused so much on this particular corner, and all the screenshots seemed like they came out of nowhere. What was this model? How did the chat prompting work? What was the context of OpenAI doing this work and collecting my prompts for training data?

I decided to do a quick investigation. Here's all the information I've found so far. I'm aggregating and synthesizing it as I go, so it's currently changing pretty frequently.

Model Architecture

Gaelan

ChatGPT passes the 2022 AP Computer Science A free response section

For fun, I had ChatGPT take the free response section of the 2022 AP Computer Science A exam. (The exam also has a multiple-choice section, but the College Board doesn't publish this.) It scored 32/36.

Methodology

• For each question, I pasted in the full text of the question and took the response given.
• I tried each question once and took the response given: no cherry-picking. For readability, I've added indentation in some cases, and included method signatures where they were provided in the question and ChatGPT only provided a body. I've added question numbers; any other comments are ChatGPT's.
• Many questions have examples containing tables or diagrams; because those don't translate well to plain text, I excluded those tables/diagrams and any text that referenced them.
• I excluded the initial instructions at the top of th

jacob-ebey

Remix Deferred

Remix Deferred is currently implemented on top of React's Suspense model but is not limited to React. This will be a quick dive into how "promise over the wire" is accomplished.

SSR + Hydration

It isn't rocket science, but a quick recap of how frameworks such as react do SSR:

1. Load data
2. Render the app

jdarcy

I've commented a few times about some issues I see with the scalability of ActivityPub - the protocol behind the Fediverse and its best-known implementation Mastodon. A couple of folks have asked for more elaboration, so ... here it is.

First, let me add some disclaimers and warnings. I haven't devoted a lot of time to looking at ActivityPub, so there might be some things I've misunderstood about it. On the other hand, I've brought bigger systems - similar node counts and orders of magnitude more activity per node - from broken to working well based on less study of the protocols involved. So if you want to correct particular misconceptions, that's great. Thank you in advance. If you want to turn this into an appeal to authority and say that I'm wrong only because I haven't developed a full ActivityPub implementation or worked on it for X years ... GTFO.

What

What is ActivityPub? It's an HTTP- and JSON-based protocol for exchanging information about "activities". An activity could be many things.

eyeseast

I have an updated version of this on my blog here: https://chrisamico.com/blog/2023-01-14/python-setup/.

Python

This is my recommended Python setup, as of Fall 2022. The Python landscape can be a confusing mess of overlapping tools that sometimes don't work well together. This is an effort to standardize our approach and environments.

Tools and helpful links:

• Python docs: https://docs.python.org/3/
• Python Standard Library:  - Start here when you're trying to solve a specific problem

DougGregor

A Possible Vision for Macros in Swift

As Swift evolves, it gains new language features and capabilities. There are different categories of features: some fill in gaps, taking existing syntax that is not permitted and giving it a semantics that fit well with the existing language, with features like conditional conformance or allowing existential values for protocols with Self or associated type requirements. Others introduce new capabilities or paradigms to the language, such as the addition of concurrency or comprehensive reflection.

There is another large category of language features that provide syntactic sugar to eliminate common boilerplate, taking something that can be written out in long-form and making it more concise. Such features don't technically add any expressive power to the language, because you can always write the long-form version, but their effect can be transformational if it enables use cases that would otherwise have been unwieldy. The synthesis of Codable conformances, for ex

ssrihari

An opinionated list of excellent Clojure learning materials

These resources (articles, books, and videos) are useful when you're starting to learn the language, or when you're learning a specific part of the language. This an opinionated list, no doubt. I've compiled this list from writing and teaching Clojure over the last 10 years.

• 🔴 Mandatory (for both beginners and intermediates)
• 🟩 For beginners
• 🟨 For intermediates

Table of contents

1. Getting into the language

dxdxdt

What to do when Gmail marks all the mails from your server as spam

If you're self-hosting your services and having trouble getting your emails through Gmail and infuriated by Google's non-existent support, you're not the only one. I'd like to share my experiences trying to get it sorted out.

• https://support.google.com/mail/thread/171517615?hl=en&msgid=172576102

I'm the author of the post above. You can tell how arrogant Google employees are from all the previous posts he made in the past.