OTC CatchUp #127 Summary

Rishit talked about how ChatGPT, GPT-3.5 and GPT-4 are Engineering marvels rather than research marvels. They keep adding more data and parameter-handling capabilities to the models.

Harsh brought up AWS CodeWhisperer and other AWS AI announcements and Rishit said that these new products are all battles of 'Who has more data and compute?' rather than pure AI innovations.

Harsh brought up Artificial General Intelligence (AGI) and Rishit said that a lot of very reputed professors are claiming that AGI is nowhere near and might not be discovered through these Large Language Models (LLMs) that people claim are going to lead to AGI.

Harsh talked about how one of his professors, Dr. Gabriel Kaptchuk is using LLMs in his Steganography research project Meteor to hide text in text.

Rishit talked about how he is learning a lot from working with his professors Dr. Jimmy Ba and Dr. Geoffrey Hinton, who have both done immense work in AI.

Rishit shared his research paper Astroformer: More Data Might Not be All You Need for Classification that got published! 🎉

Rishit also shared his personal page on the University of Toronto web site.

Rishit told us how all ML models fundamentally use gradients to try to get closer to the correct answer from the answer it generated, and how doing away with gradients would presumably make models more efficient and easier to train.

Rishit added that there is loads of amazing research going on in AI that is currently not in the limelight.

User space vs Kernel space

How does a Kernel figure out that a single core processor is in an infinite loop? There is an OS timer tick (APIC timer) that the processor uses to generate a regular interrupt and that is where it can be figured through usage monitoring that something is in an infinite loop.

What if there’s a bug in the Operating System’s Scheduler or if the hardware is hung? The machine has to be reset, but before a cold reset, a hardware interrupt called the Non-maskable interrupt (NMI) is initiated through the hardware’s control plane (that is usually not hung), that no software can trump. NMI is useful because it logs information before carrying out its resetting tasks, which aids in debugging the problem, which a cold reset would’ve made extremely difficult.

Gospels for Kernel level code

Linux’s bind() that binds (assigns) a name (address) to a socket.

The nice command that runs a program with modified scheduling priority.

How does the Linux Out-of-Memory (OOM) Killer decide which process to kill first? (There is no right answer because there is a loss, but the OOM Killer has to be 'fair enough'.)

We’re wondering whether it is okay to mention that programming language’s name in the point above after the stringent (and weird?) trademark policy draft.

Along similar lines, another restrictive licensing issue is the DeWitt Clause that came up with Oracle’s database. (Why’re we not surprised that it’s Oracle?)

Rocket is a web framework for 'the programming language with a stringent trademark policy we mentioned above'.