Writing
19 posts on how things actually work.
Notes on data engineering, Git, networking and the systems behind software. Pick a topic and read it in order, or browse by date.
Apache Spark
What Spark is doing while you wait: nulls, jobs, stages, tasks and where the driver runs.
- Handling Nulls in SparkSix of ten movies have no rating, and filtering for == None returns zero rows with no error. How nulls behave in Spark, and the functions that handle them properly.
- Under the hood of a Spark jobTwo DataFrames with 5 and 6 partitions, one join, and the Spark UI shows 200 tasks. Where that number comes from, and how Spark splits work into jobs, stages and tasks.
- Spark Execution ModesSubmit a Spark job from your laptop, close the lid, and the job dies. Change one flag and it keeps running. Cluster, client and local mode come down to where the driver runs.
- The Life Cycle of a Spark Application ( Outside )You run spark-submit and, a while later, get a result. In between, four things happen and none of them are in your code: the request, the launch, the execution and the shutdown.
Git
From the pendrive problem to what actually lives inside the .git folder.
- Why Version Control Exists - The Pendrive ProblemOne pendrive, two developers, and a folder called Final_v2_DO_NOT_DELETE. What sharing code looked like before version control, and why Git exists.
- Git for Beginners: Essential Commands You Need to KnowYou save your game before the boss fight. Git lets you do the same with code. The core Git commands, explained as save points.
- Inside Git: How It Works Under the HoodChange one file, commit, and Git stores a new blob and a new tree, then reuses everything else. A look inside .git with cat-file.
Networking
What happens between typing a URL and the first byte: DNS, TCP, UDP and cURL.
- How DNS Resolution WorksYou type google.com and the page starts loading almost instantly. In that time a resolver may have asked three servers where google.com lives, and none of them knew the whole answer.
- DNS Record Types ExplainedPointing a domain at your website takes one DNS record. Receiving email takes another. Proving to Google you own it takes a third. A, AAAA, CNAME, NS, MX and TXT records, and the job each one does.
- Networking Basics: Comparing TCP and UDP ProtocolsBefore a webpage sends one byte over TCP, there's already been a full round trip. A DNS lookup over UDP is one packet out and one back. TCP vs UDP is a trade between guarantees and speed.
- Getting Started with cURLWhen an API call fails in the browser, all you see is 'Something went wrong'. One terminal command sends the same request and shows you everything the server sent back.
- TCP Working: 3-Way Handshake & Reliable CommunicationBefore your browser sends a byte of HTTP, it trades three packets with the server, and none of them carry data. They exist to agree on one thing: where to start counting.
JavaScript
Operators, control flow, the global object and what “this” really points to.
- JavaScript Operators: The Basics You Need to KnowYou already know how operators work: you've used a calculator. JavaScript's version adds a compare mode, a logic mode and memory buttons, plus one button that trips up almost everyone.
- Global vs GlobalThiswindow.myVar works in the browser. Run the same file in Node.js and it crashes: window is not defined. Same language, same global object, different names, and the one name that works everywhere.
- The magic of call(), apply(), bind()user.greet() prints 'Hi, alice'. Pass the same function to setTimeout and it prints 'Hi, undefined'. Why this changes, and how call, apply and bind put you in control of it.
- Control Flow in JavaScript: If, Else, and Switch ExplainedYour bank app blocks a payment when your balance is too low. Your game says Game Over when your health hits zero. Every decision a program makes comes down to if, else and switch.
From real projects
Bugs and fixes from things I have shipped: row locks, sessions and CI/CD.
- Automating Unit Tests and Deploying AWS Glue & Lambda Python Jobs with CI/CDMost Glue and Lambda code is tested by deploying it and reading CloudWatch logs. A Jenkins pipeline that catches bugs on the pull request instead, with uv, ruff, pytest and Terraform.
- Preventing Double Bookings in Postgres with SELECT FOR UPDATETwo members, one spot left. How a gym-booking app stops overbooking with Postgres row locks, the race reproduced without them, and two lock mistakes found along the way.
- Why Better Auth Kept Logging Users Out While They Were ActiveA 15-minute idle timeout in Better Auth logged users out mid-click. The cause: a sliding session renewed in the database but never in the browser cookie.