We Engineers Suck at Task Estimation
Why? Because we almost never account for everything: time to build, test, deploy, people falling sick, holidays, code reviews, iterations, changes in requirements, delays from dependent teams...
This edition of the newsletter contains
one quick write-up that will help you grow faster in your career
a video I posted
a paper I read
I have also shared 3 super-interesting articles to read over the weekend. Thank you once again for reading this edition of my Newsletter. Now, without further ado, let’s jump right in.
By the way, the admissions for my System Design June cohort are open. If you are SDE-2, SDE-3, and above, and looking to build a rock-solid intuition to design any and every system, you will find my course super interesting.
Instead of drawing boxes, we go into the intricate details of every single system and build an end-to-end understanding. The learnings from the course can be applied at your workplace from day 1. So, if you are looking for some real engineering discussions or brainstorming, do check out my course.
Course curriculum and other key details: https://arpitbhayani.me/course
We Engineers Suck at Task Estimation
We engineers are terrible at estimating timelines. Here's a neat trick -
break it into major pieces and estimate the time for each,
make it 1.5x, because, you know, testing
now, double it, because your initial estimates were way off :)
Why? Because we almost never account for everything: time to build, test, deploy, people falling sick, holidays, code reviews, iterations, changes in requirements, delays from dependent teams, integrations breaking, compliance, and so much more.
We almost always underestimate the effort required and overestimate our abilities and situations. One time, I gave an estimate of a task to be one week because it seemed simple, but it took me about 4 weeks to wrap up end-to-end.
By the way,
Being hands-on is the best way for you to learn. Practice interesting programming challenges like building your own BitTorrent client, Redis, DNS server, and even SQLite from scratch on CodeCrafters.
Sign up, and become a better engineer.
Here's the video I posted
I published a video - Build a robust Payments service using Idempotency Keys
One common thread that stitches all payment systems like Razorpay, Stripe, etc is Idempotency; and knowing how they implement it is essential.
We retry whenever there is a failure. However, in the payment system, auto retries can lead to double payments. Every single payment system leverages Idempotency Keys to avoid double payment.
I published a quick video explaining idempotency keys, how they work, and, more importantly, the code and implementation nuances.
Paper I read and would highly recommend
I spent some time reading The Chubby lock service for loosely-coupled distributed systems
When we start any API server at Google, we see a log message containing the phrase "Chubby lock ..." and to understand what goes behind the scenes, this weekend, I am reading the Chubby paper.
Chubby is Google's distributed lock service that was built way back in 2006 and is still functional today. The paper covers the nuances of building a distributed lock and what it takes to run it in production, used by hundreds of servers concurrently, and at Google scale.
Some time back, I read this paper for the second time, and I can vouch that by reading this paper, you will also understand the nuances and practical application of leader election, metadata consistency, and failover. It is one of those systems that prioritizes availability and reliability over throughput.
By the way, implementing this paper was our assignment in the Distributed Systems course, so not super tough, but definitely a good one.
You can download this and other papers I recommend from my papershelf.
Three interesting articles I read
I read a few engineering blogs almost every day, and here are the three articles I would recommend you to read.
Thank you so much for reading this edition of the newsletter 🔮 If you found it interesting, you will also love my courses
I keep sharing no fluff stuff across my socials, so, if you resonate, do give me a follow on Twitter, LinkedIn, YouTube, and GitHub.