Kevin Denamganai Near32

Debugging Python within Docker

Often you need to run debugger within docker. It many cases it looks like this:

import pdb; pdb.set_trace()

Or:

Overview

Internet connection and DNS routing are broken from WSL2 instances, when some VPNs are active.

The root cause seems to be that WSL2 and the VPN use the same IP address block, and the VPN routing clobbers WSL2's network routing.

This problem is tracked in multiple microsoft/WSL issues including, but not limited to:

ruvllm_sparse_attention: Subquadratic Sparse Attention for Edge LLM Inference

What This Is (Simple Version)

AI language models are slow on small computers — not because of the model weights, but because of attention: the mechanism that lets every word look at every other word in the text. When you double the text length, attention gets four times harder, not twice.

ruvllm_sparse_attention fixes this by teaching the model to be selective. Instead of every word looking at every other word, it looks at:

The words closest to it (recent context)
A few anchor words at the start (global signals)

	'''
	I am reading this amazing series(https://uvadlc-notebooks.readthedocs.io/en/latest). I always struggle with revisiting
	my old code that has a lot of tensor manipulation. Experimented with reimplementing their MultiHeadAttention layer using
	einops syntax that feels more human readable.
	'''
	import torch
	import torch.nn as nn
	import torch.nn.functional as F
	import einops
	import math