gistfile1

14:38 <%mr_daemon> I really want to put all my (rather terrible) programming skills in Python right now, even if that means giving up Java momentarily.
14:39 <%mr_daemon> Because I fucking love Python, and I don't get to use it nearly enough to develop great skills
14:40 <@KniPhone> Python is so kawaii.
14:40 <@KniPhone> EVERYTHING SHOULD BE PYTHON
14:40 <@KniPhone> EVEN REALITY
14:41 <%mr_daemon> Well it's not suitable for some things
14:41 <%mr_daemon> but with ardeo I will be trying to blatantly ignore that
14:41 <@KniPhone> SUITABLE FOR EVERYTHING
14:41 <@KniPhone> :(
14:43 <%mr_daemon> It has a GIL
14:43 <%mr_daemon> So it kinda sucks for heavy threading
14:43 <@KniPhone> Gil?
14:43 <%mr_daemon> But IT CAN BE WORKED AROUND HOPEFULLY
14:43 <%mr_daemon> global interpreter lock
14:44 <%mr_daemon> This means it will not do proper SMP support :(
14:45 <%mr_daemon> Like, it becomes impossible to scale to multiple processes
14:45 <%mr_daemon> It's sort of like the Big-Lock in older-ish Unix-derivatives
14:46 <@KniPhone> Oh :(
14:47 <%mr_daemon> Where processes sending interrupts from user space assumed there was only one CPU and that it was shared
14:47 <@KniPhone> oh wat
14:48 <@KniPhone> D:
14:48 <@KniPhone> I don't
14:48 <@KniPhone> Understand :(
14:53 <%mr_daemon> Well, let me think for a moment. Uh...
14:53 <%mr_daemon> Well, if you think about it, a CPU can only do one single thing at a time
14:53 -!- O [[email protected]] has joined #glasnost
14:53 <@KniPhone> yes
14:54 <%mr_daemon> Like, while the CPU is accepting my keyboard input, it's not doing anything else -- it's not processing the database query, it's not playing my music it's not doing shit.
14:54 <@KniPhone> ya
14:55 <@KniPhone> brb processing mu-OH GOD KEYBOARDS
14:55 <%mr_daemon> So at some interval, every fraction of a second, a CPU does a context switch and processes some other thing. Basically computers today emulate multitasking by switching really quickly between tasks
14:55 <@KniPhone> yeah~
14:56 <%mr_daemon> The thing is, (and this is a problem faced by every operating system at some point), the OS usually divides CPU usage into time slices
14:56 <@KniPhone> ya
14:56 <%mr_daemon> Like, an allocated slice of time for a CPU to do a task.  So it may work on one task until there it's either done, or the timeslice is over
14:57 <@KniPhone> yeah
14:57 < O> Is this a conversation on iphones and multitasking.
14:57 <@KniPhone> Python and threading.
14:58 <%mr_daemon> The scheduler will decide how to allocate that shit all over the place and decide who gets to run what and when. However, if there is something really important coming up, the kernel usually allows the runnig process to be interrupted, or, in more familiar terms, /preempted/.
14:58 < O> I see.
14:59 <%mr_daemon> Hence the word preemptive multitasking. Now this is all from userland. But the kernel also runs there, you know.
15:00 <%mr_daemon> While it's not a process per se, it's still running and faces problems and challenges that normal processes don't -- I mean processes don't worry about scheduling, do they?
15:00 <%mr_daemon> So if the CPU is doing something like, processing that porn you're viewing
15:01 <%mr_daemon> In the meantime a packet arrives on your ethernet card. The kernel must process it! (let's ignore offloading for now)
15:01 <%mr_daemon> (and in fact pretty much forever)
15:02 <%mr_daemon> The original idea was that adding /another/ processor could allow things to run /concurrently/, like your ethernet card could receive a packet and the kernel could process it, while it's writing to the filesystem at the same time.
15:03 < O> Mozart.
15:03 <%mr_daemon> But the problem that you face when you try to implement this is that the kernel has no clue what process running on cpu0 is doing in relation to cpu1
15:04 <%mr_daemon> Like, what if one CPU allocates memory for your porn, and the other CPU allocates memory to do a filesystem operation?
15:04 <%mr_daemon> And the memory CPU2 allocates is, sadly, the same CPU1 decided to use?
15:04 <@KniPhone> Oh D:
15:04 <@KniPhone> oh dear!
15:04 <%mr_daemon> Yeah that's the main problem. So initially most OSes just declared the kernel to be nonpreemptive
15:05 <%mr_daemon> i.e the kernel itself cannot be pre-empted.
15:05 <@KniPhone> ya
15:05 <%mr_daemon> So shit becomes deterministic. When a part of the kernel allocates a bit of memory, it can be damn sure it's going to be there again when it executes the next instruction
15:05 <%mr_daemon> And that nobody filled it with dickgirl porn
15:05 <%mr_daemon> Instead of the filesystem buffer
15:06 <@KniPhone> hahaha
15:06 -!- PEPPERONI [[email protected]] has quit [Ping timeout]
15:06 <%mr_daemon> So, if you take BSD, the first attempt at SMP was this: The Giant Kernel Lock
15:06 -!- PEPPERONI [[email protected]] has joined #glasnost
15:07 <@KniPhone> wat was tat
15:08 <@KniPhone> also 9% battery left :(
15:08 <%mr_daemon> Processes would be sprinkled around on CPUs, and there was a giant ass fuck lock on the kernel. So before a CPU would try to run something that was kernel-based, it would check if the kernel lock was available in the first place. If it was free, it took the lock and began running it. If it isn't free, the CPU now knows that the kernel is busy being ran elsewhere on another cpu
15:08 <@KniPhone> oh okay that makes sense!
15:08 <%mr_daemon> So the kernel would then know for sure its shit wasn't being run elsewhere being its back
15:09 <%mr_daemon> So essensially
15:09 <%mr_daemon> *essentially
15:09 <%mr_daemon> The kernel would always run on ONE cpu at a time, just like it had always done.
15:09 <@KniPhone> that is somewhat inefficient~
15:09 <@KniPhone> but makes sense!
15:10 <%mr_daemon> That kinda worked okay for two CPUs or so. But now start adding like 8 CPUs and the shit hits the fan because the system will spend most of its time waiting for the fucking kernel lock
15:10 <@KniPhone> yeah D:
15:11 <%mr_daemon> (FreeBSD "fixed" the giant lock by dividing it into a bunch of smaller locks for every single function of the kernel)
15:11 <@KniPhone> That's better but way more complex.
15:12 <@eMb> hi
15:12 <%mr_daemon> (Originally for core functions like the scheduler, the network stack, I/O, etc etc, and then slowly more locks were added to various components. So each subsystem can run independently)
15:12 <@KniPhone> ya
15:12 <%mr_daemon> And they run as, wait for it
15:12 <%mr_daemon> threads
15:12 <@eMb> delicious dumplings
15:12 <@KniPhone> WHAT A TWIST
15:12 <%mr_daemon> yes
15:13 <%mr_daemon> And each tiny lock has their own kind of locking mechanisms
15:13 <@KniPhone> :3
15:13 <@KniPhone> aaaah hold on
15:13 <@KniPhone> Brb smoke
15:13 <%mr_daemon> like, spinlocks, mutexes, sx, rw, semaphores, etc etc
15:13 <%mr_daemon> Okay :D
15:14 <%mr_daemon> SO TO SUM IT UP
15:14 <@p> hi
15:14 <@p> .w 55455
15:14 <@p> .weather 55455
15:14 <@p> worthless
15:15 <@p> 14:15 < heenbot> Currently at Minneapolis, Minnesota: 31.9ºC / 89.5ºF and  Scattered Clouds with a Heat-Index of 37.2ºC / 99ºF
15:15 <%mr_daemon> KniPhone: In essense, in order to take advantage of multi processoring, FreeBSD had to first break the giant kernel lock into smaller locks, with all the implied complexity and trickiness of doing it absofuckinglutely right. And shit's hard.
15:15 < O> mr_daemon: Nicely explained
15:15 < O> You should write bsd documentation
15:16 <@eMb> Is Linux also able to run on several cores?
15:16 < Drizt> No
15:16 <@p> when did this discussions tart
15:16 <@p> start
15:16 <%mr_daemon> KniPhone: This then allowded them to thread some kernel shits and have them running concurrently on multiple CPUs. Python just faces the same problem because the interpreter has a giant lock itself. So everything is stuck at the interpreter level, And since it's a lone userland process, it can't SUDDENDLY GO RUN A THREAD On another CPU without this shit.
15:16 <@p> I want to read it all