idevnull SwathiMystery

Some remarks on Large Language Models

Yoav Goldberg, January 2023

Audience: I assume you heard of chatGPT, maybe played with it a little, and was imressed by it (or tried very hard not to be). And that you also heard that it is "a large language model". And maybe that it "solved natural language understanding". Here is a short personal perspective of my thoughts of this (and similar) models, and where we stand with respect to language understanding.

Intro

Around 2014-2017, right within the rise of neural-network based methods for NLP, I was giving a semi-academic-semi-popsci lecture, revolving around the story that achieving perfect language modeling is equivalent to being as intelligent as a human. Somewhere around the same time I was also asked in an academic panel "what would you do if you were given infinite compute and no need to worry about labour costs" to which I cockily responded "I would train a really huge language model, just to show that it doesn't solve everything!". We

Create Postgres / Postgis Server EC2 Box

Launch Ubuntu instance

Click on Launch Instance` button
Choose Ubuntu Server 14.04 LTS (HVM), SSD Volume Type
Configure Security Group
In step 6 Configure Security Group add the rule:
Type: PostgreSQL
Protocol: TCP
PortRange: 5432 (default postgres port)

title	sidebar
Segment Event Tracking for Shopify	Shopify

Segment makes it simple for Shopify merchants to integrate analytics, email marketing, advertising and optimization tools. Rather than installing all your tools individually, you just install Segment once. We collect your data, translate it, and route it to any tool you want to use with the flick of a switch. Using Segment as the single platform to manage and install your third-party services will save you time and money.

The guide below explains how to install Segment in your Shopify store. All you need to get up and running is copy and paste a few snippets of code into your theme editor. (You don't have to edit the code or be versed in JavaScript.) The following guide will show you how, step by step.

Gist

This is a quick tutorial explaining how to get a static website hosted on Heroku.

Why do this?

Heroku hosts apps on the internet, not static websites. To get it to run your static portfolio, personal blog, etc., you need to trick Heroku into thinking your website is a PHP app. This 6-step tutorial will teach you how.

Basic Assumptions

Stevey's Google Platforms Rant

I was at Amazon for about six and a half years, and now I've been at Google for that long. One thing that struck me immediately about the two companies -- an impression that has been reinforced almost daily -- is that Amazon does everything wrong, and Google does everything right. Sure, it's a sweeping generalization, but a surprisingly accurate one. It's pretty crazy. There are probably a hundred or even two hundred different ways you can compare the two companies, and Google is superior in all but three of them, if I recall correctly. I actually did a spreadsheet at one point but Legal wouldn't let me show it to anyone, even though recruiting loved it.

I mean, just to give you a very brief taste: Amazon's recruiting process is fundamentally flawed by having teams hire for themselves, so their hiring bar is incredibly inconsistent across teams, despite various efforts they've made to level it out. And their operations are a mess; they don't real

---------- Forwarded message ----------

From: Mark S. Miller <[email protected]>
Date: Tue, Nov 16, 2010 at 3:44 PM
Subject: "Future of Javascript" doc from our internal "JavaScript Summit"
last week
To: [email protected]


	def load_pool3_data():
	X_test_file = 'X_test_20160212-00:06:14.npy'
	y_test_file = 'y_test_20160212-00:06:14.npy'
	X_train_file = 'X_train_20160212-00:06:14.npy'
	y_train_file = 'y_train_20160212-00:06:14.npy'
	return np.load(X_train_file), np.load(y_train_file), np.load(X_test_file), np.load(y_test_file)

	def batch_pool3_features(sess,X_input):
	"""

	# Note – this is not a bash script (some of the steps require reboot)
	# I named it .sh just so Github does correct syntax highlighting.
	#
	# This is also available as an AMI in us-east-1 (virginia): ami-cf5028a5
	#
	# The CUDA part is mostly based on this excellent blog post:
	# http://tleyden.github.io/blog/2014/10/25/cuda-6-dot-5-on-aws-gpu-instance-running-ubuntu-14-dot-04/

	# Install various packages
	sudo apt-get update

	package com.jelies.spring3tomcat7.config.quartz;

	import org.quartz.spi.TriggerFiredBundle;
	import org.springframework.beans.factory.config.AutowireCapableBeanFactory;
	import org.springframework.context.ApplicationContext;
	import org.springframework.context.ApplicationContextAware;
	import org.springframework.scheduling.quartz.SpringBeanJobFactory;

	/**
	* This JobFactory autowires automatically the created quartz bean with spring @Autowired dependencies.

	// Reference: http://stackoverflow.com/questions/4822471/count-number-of-lines-in-a-git-repository

	$ git ls-files \| xargs wc -l