May 16th, 2024 (1 day ago) • 2 minutes

Fire everywhere

I don't know if it's just me, but incidents are actually exciting problem to work on 🫨.

Think of it this way - instead of working on your mundane day to day tasks, you are working on a high stake, time constrained problem that requires you to think on your feet.

You get roped into meetings then a bunch of you just sit down, brainstorm ideas, suggest solutions, and then execute them. It's like a mini project that you can finish in a day or two.

It also tests you on whether you have enough knowledge on the system you are working on. Surface level knowledge won't cut it. You need to dig deep into the system, research on tiny details that you might have missed before, and then connect the dots to find the root cause of the problem.

You also have to be able to communicate your findings in a timely manner, and in a way that everyone can understand.

All around, I think that's why I like working on incidents. It's a break from the usual routine, and it's a good way to test your knowledge and skills.

Today's problem was related to some spark job running over 45 hours which then cause data to be staled downstream. It was obvious at first, but we realize that the problem was the node that are storing the intermediary data (extracted from Snowflake) was killed before the job loads the data into DocDB (our destination). This caused the job to re-run the same data over and over again and thus the long running time.

We figure this out really by diving into the logs and trying to make sense of the timeline and job history from Spark's History Server.

May 15th, 2024 (2 days ago) • 2 minutes

Huh time travel?

Daylight saving was a hot topic about a couple months ago. I remembered talking to my friend on what the heck is it, why do we need it and how does it work. It was confusing but now the same question popped up in my brain again.

So here's my notes for future me.

What? Daylight saving time (DST) is the practice of setting the clock forward by one hour during the warmer months of the year, so that evenings have more daylight and mornings have less.

Since we can't change the sunrise/sunset time, we change the clock so it appears to the human brain that sunrise happened at 9am instead of 8am for instance and sunset at 8pm instead of 7pm.

Why? Mainly to help save energy and offset the amount of time people spend indoors during the winter. It was primarily introduced for the benefit of farmers, who needed daylight hours to get the harvest in.

The Confusion. Some parts of the world don't experience Winter so DST doesn't make sense for them. Even in the US, some states don't follow DST like Hawaii and Arizona. In Australia, only 3 states follow DST out of 6.

The Hard Part. Since DST is a human concept, it's hard for physical clock / software clock to follow. At work, this meant that time sensitive jobs like collecting loans from users need to be adjusted/monitored closely twice yearly.

References

• Spot on video
• Falsehoods about time

May 14th, 2024 (3 days ago) • 1 minute

Instead of thinking about the worst case scenario, think about what you can do to make things better, to increase you luck surface area.

Think that you can make it instead of worrying you can't. The former makes you do something, while the later just plays in your head.

Do things that will increase your chances of succeeding.

Think positive and act on it.

May 13th, 2024 (4 days ago) • 1 minute

Today I went to a co-working space at Sri Petaling to have some focus time since I am WFH. The place was quite nice - it was quiet and there's free flow of snacks and drinks.

I got myself some coffee to keep my system going and to start working.

I got a Dirty Latte with double shot at first, then followed by a Piccolo few hours later.

That's a total of 4 shots.

As I am going through my second cup, I felt so hyper and jittery that I couldn't focus on my work. I was just typing random stuff and my heart was beating so fast. My legs were constantly shaking and I just felt like walking and running around.

Well, coffee is supposed to make you more alert and focused, but I think I was too energytic instead.

I guess a double shot is my limit and this is a reminder for my future self.

p.s Abid, my colleague is an avid coffee drinker and I am not sure how he can handle so many shots in a day. I should ask him about it.

May 12th, 2024 (5 days ago) • 5 minutes

tl;dr The fish are always surrounded by water. So they never think about it. But they should because they're always looking at things through the water and it affects everything about how they perceive the world.

It’s easy to become obsessed with a goal or a person or a job or a dream – a lot of his work is about obsession – but we should also be aware of how these obsessions can surround us, and even drown us.

This morning I listened to David Foster Wallace's commencement speech to the graduating class of Kenyon College. This was back in 2005, but little did he know that his words would resonate with us in 2024.

3 takeaways:

1. Be aware of arrogance

Our personal point of view is not the only one. It's often scoped by our upbringing, surrounding, and experiences. We should be aware of this and be open to other perspectives. Default from I'm always right to I'm maybe right.

Allowing our own thoughts to be wrong is a sign of growth. It forces us to question our baseline and experience a richer world.

2. Rejecting your default

Adult life is full of boredom, routine and petty frustration. The point is to choose how to react to this. You are not the only that is feeling this, everybody else is just going through as hard as you.

Don't assume you are the center, instead allow your compassion and empathy to develop your world view. Choose to look at things differently even though it's hard.

3. Finding freedom

Ignore your default. True freedom is found in choosing how to think and what to think about.

May 11th, 2024 (6 days ago) • 2 minutes

A follow up from yesterday's blog, I also want to pen down some criteria that I thought of when looking for a new job. This was something I discussed with my friends Abid and Jeremy while we are on a roadtrip.

In short, I boil it down to 4 criterias:

• Having a lot of smart people in the team. Having people who are ahead of you can serve as guidance and mentorship. These people are usually the one that you aspire to be, expert in their field or have so much experience that you just want to learn off them.

  I am fortunate enough to have met some of these people at MoneyLion. If you are reading this, Abid, Jeremy, Arnold, Nazmi, thank you for shedding light on my short journey there.

• Idea/Industry. I want to work on something that I am passionate about. And right now that's fintech or AI. These field is lucrative in itself but also difficult because there's so much regulation and competition. That's what makes it interesting.

  Also because that's where the future of money is.

• Location. I love to travel and work at the same time. So being able to work in a remote country attracts me at this point in life. If I am older and planning to settle down, this point might change.

  Right now, those cities are San Francisco, Japan, and Hong Kong. Why? Strong culture and city life.

• Compensation. This is the least important to me (for now, if I have more commitments then I will move this up). I think if you are good at what you do, money will come. But I also understand that money is important to sustain a living.

  I choose the job which will provide me a good enough compensation that I can save yet spend without constantly looking at the price tag (or contemplating my lifestyle choices).

May 10th, 2024 (7 days ago) • 2 minutes

Today, a colleague of mine asked me about some resources to help him strengthen this fundamentals as an engineer. We briefly talked about interview prep, leetcode, how to become a staff engineer, and so on. I thought it would be a good idea to write a blog post about this topic (since I will be prepping soon as well 🥲)

I did Neetcode 150 the last time round for interview prep. But apart from that, I think this is a good list to go over to help you to learn and understand the most common form of data structures and algorithms.

Practice a lot of this. Regardless of discipline, you will be asked to solve problems in interviews.

I found lanki.xyz for spaced repetition of leetcode problems. I haven't try it out yet but this seems good.

Watch some real-life interview on interviewing.io to get a sense of how the interview goes

Go through some of these books to help you understand the bigger system.

System Design

• Designing Data-Intensive Applications - Notes available courtesy of San Diego Machine Learning Club
• System Design Interview - both book 1 and 2
• System Design Primer
• Grokking the System Design Interview

Databases

• Database Internals

Only if you are prepping for backend roles.

Backend questions

• Backend SWE interview questions

General guides

Interview Guide

• Interview Guide
• Tech Interview Handbook

May 9th, 2024 (8 days ago) • 1 minute

A reminder that there are so many things that I can do with myself to make life more attractive and interesting.

A reminder to take time off from the daily grind and just enjoy world as it is.

Travel to a new place.

Learn a new skill/language.

Read a new book.

Watch a new movie.

Be friend with a stranger.

May 8th, 2024 (9 days ago) • 1 minute

Today is unusually tiring and un-motivational. I am not too sure why?

The feelings sucks because I want to spend time doing something but I am just roaming around the house. Occasionally, taking a nap and waking up quickly after. I thought this helped but it didn't.

I have been on med for the past 2 weeks, taking daily doses of Acutane and Antibiotics. This stuff helped a ton with skin acne but I am definitely experiencing the side effects now. I also cannot each a huge bunch of things because of it. It's like a love-hate relationship with the med.

I feel like a zombie 🧟 Aimless with energy draining off quicker than your iPhone.

I hope this feeling goes away soon.

Some things I am looking forward to for the coming weeks:

• Going out for meals to catch up with friends and colleagues before;
• Leaving to SF
• Resting at home after quitting
• Trip to Bangkok
• Building Onyx (holy, summarization is hard - probably this is a good PhD topic I would do.)
• Time to read and write more

May 7th, 2024 (10 days ago) • 1 minute

"I fear not the man who has practiced 10,000 kicks once, but I fear the man who has practiced one kick 10,000 times." - Bruce Lee

This is very much applicable to career as well. You will be average if you try to learn everything and be the best in everything. Time is unfortunately finite.

Of course, you don't want to be a 1 trick pony because that limits your options and opinions. Try a few things and identify your T shaped skills.

Choose thoughtfully, Act decisively.

p.s I used to want to be a one man army. It had helped me of course to know a bit of everything, so I can cross relate things outside of my domain. But this didn't scale well.

Roadmap from 2022 where I try to make myself a 10x engineer for any startups :)