adivekar-utexas

The Situation

Suppose you've got a team of 2+ people who are working on an AI project (research, business, etc). and they need a common Jupyter machine where they can run notebooks. You have two common options:

• A common JupyterLab: this has the issue that everyone uses the same UI, leading to conflicts. Example: Amazon Sagemaker Notebooks.
• A JupyterHub install: this needs a bit more setup, but gives much better separation.

The rest of this post is going to discuss how to setup JupyterHub in a way that:

1. Everyone gets their user on the machine and their own home directory, which only they have permissions to.
2. No one can access any one else's home directories or files.

adivekar-utexas

import pandas as pd
import hvplot
import holoviews as hv
import hvplot.pandas
hvplot.extension("bokeh")
from bokeh.themes.theme import Theme
hv.renderer('bokeh').theme = Theme(json={
    'attrs': {
        'Figure': {
            'background_fill_color': '#2F2F2F',

adivekar-utexas

@media print {

    /* ---------- wrap long code lines ---------- */
    pre,
    /* standalone code blocks            */
    code,
    /* inline snippets <code>like this</code> */
    pre code {
        /* <pre><code> … </code></pre> combos   */
        white-space: pre-wrap !important;

adivekar-utexas

(Reproduced from https://mscshub.com/)

Background: I work as an ML Scientist at FAANG. I started the UT Austin MSCSO program in Fall'2020, and this review was written in Dec'2023. This review is very specific, so a motivated person can probably find me. Let me save you the trouble, here is my LinkedIn: linkedin.com/in/ardivekar/ . Feel free to reach out with questions. I'm usually also on the MSCSO Slack.

I applied for UT Austin's MSCS Thesis in Dec'2022, and completed the Thesis over 3 semesters (Spring'23 to Spring'24). My thesis was in the NLP domain, during 2023 when AI went mainstream via ChatGPT, and new LLMs were released every single week. As you can expect, this makes my experience unique. But I expect everyone's thesis experience will be unique since it depends on your personal strengths and interests.

Another thing which I did different from other students is that I framed my own research problem. This is recommended only if you're really dead-set on one idea, like I was. Otherwise it's okay t

adivekar-utexas

🚨 (Update June 2025) 🚨 This little post has gained a huge amount of interest, for which I am very grateful. However, it's been overwhelming to keep up with 40-50 reachouts per week. To make things manageable, I am selectively focusing on referrals for full-time Applied Scientist and Data Scientist roles only. The rest of this post still contains tips for other tech roles (Software Engineer, Data Engineer, BIE, etc) and details on the hiring process in general; I hope these are valuable for you. Good luck on your journey!

---

I am always happy to provide referrals for folks applying to Amazon for a variety of roles. Amazon is a FAANG, so I know that landing a job there can make a big difference to someone's career, and I am happy to spend the time to provide a referral.

If you are interested, please read the points below very carefully before reaching out (I only accept requests via LinkedIn). This is to save time on both sides 😄 I usually check around

adivekar-utexas

Submission Steps

1. Download source code from Overleaf if you use it: menu -> download -> source.

2. Strip comments and combine all tex files (f01-main.tex, f02-intro.tex, etc.) into one file arxiv_main.tex.

# Replace f01-main.tex with the main tex file in your overleaf project
latexpand --empty-comments f01-main.tex > arxiv_main.tex

adivekar-utexas

from typing import *
import ctypes
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor

def stop_executor(
        executor: Optional[Union[ThreadPoolExecutor, ProcessPoolExecutor]],
        force: bool = True,  ## Forcefully terminate, might lead to work being lost.
):
    if executor is not None:
        if isinstance(executor, ThreadPoolExecutor):

adivekar-utexas

"""A collection of utilities to augment the Python language:"""
from typing import *
import time, traceback, random, sys
import math, gc
from datetime import datetime
from math import inf
import numpy as np
from threading import Semaphore
import multiprocessing as mp
from concurrent.futures._base import Future

adivekar-utexas

"""
REALNEWS is a big dataset of several million news articles obtained from Common Crawl.
It was used to train the Grover news generation language model. 
Details here: https://arxiv.org/abs/1905.12616

In this script, we download it following instructions from https://github.com/rowanz/grover/tree/master/realnews
(please make sure to fill in the survey in the link above!)
After downloading, the file is a .tar.gz containing an enormous .jsonl file. 

To split it into multiple small .parquet files, I've written the script below.