Making (Business) Sense of Microservice Architectures

Every major cloud technology conference these days has multiple presentations about moving to a microservices architecture, so I did some research and tried to answer basic questions like what microservices are, what benefits and costs they provide to businesses, and when it might make sense to think about using them. My findings are summarized below; I think others may find it useful.

1. What Are Microservices?
The Netflix story is a great one for getting a basic understanding of microservices. Back in 2008, the Netflix website was one giant application, also referred to as a monolith in microservices lingo. The Netflix monolith had a lot of problems. For example, a simple error such as a missing semi-colon in the code base could bring down the entire Netflix website for several days. Also, developing new features was pretty slow as code changes (such as adding a new column to a database table) needed cross-functional teams to review and approve. Finally, and maybe most importantly, Netflix needed to move to the cloud so they could scale quickly enough to meet rising customer demand. Forklifting their monolith to the cloud wasn’t really an option, so they spent seven years adopting a Microservice architecture. This meant that they broke down their monolith into smaller, more manageable, and fully decoupled components, exposed through lightweight APIs.

2. Benefits
The benefits of adopting a microservice architecture include:
a. Scaling Efficiently: Teams can easily scale out the application components that need more resources (such as authentication & payments), without having to scale out the entire application.
b. Raising Availability & Fault Tolerance:  The decentralized nature of a microservice architecture eliminates single points of failure and delivers resilient, highly available applications that experience very little outage.
d. Getting Rid of Big Bang Deployments: Application deployment becomes a non-event as teams can deploy their services independently multiple times a day, without having to put the entire application in maintenance mode, or co-ordinate with other teams.
e. Giving Teams Technical Autonomy: Teams can implement their service using whatever technologies they prefer, as long as these services are exposed through standard APIs.

3. Costs
Despite their numerous benefits, microservices come with the following costs:
a. Microservices Imply an Org. Change: At its heart, microservice architectures are about creating the autonomous teams that can build, run and manage independent and loosely coupled services. This usually means a change from the current org. structure, and we all know how painful and emotional org. changes can be.
b. Application Architectures are More Complex: Every microservice architecture is essentially a distributed system, which means that things like monitoring, testing, and end-to-end visibility of the system become really really hard. Also, moving developers between teams becomes more difficult as their new team may work on a completely different stack from their current team.

3. When To Think About Adopting Microservices
According to Martin Fowler, a microservices thought leader, there are a few housekeeping items you need to address before moving your critical applications to a microservice architecture. They include:
a. Rapid Provisioning: You should be able to provision new servers in minutes and not days.
b. Monitoring: You need to know what’s going on within your application; where things are failing and why.
c. Rapid Deployment: Your deployment processes should be automated, and should take minutes and not days.

4. Resources
If you are interested in diving a little deeper into microservices, here are a few resources I found to be super helpful:
a. Blog Post on Why You Can’t Talk About Microservices Without Mentioning Netflix.
b. Video of Microservices Overview by Martin Fowler – Very good overview.
c. Video of Microservices at Netflix Scale: Principles, Tradeoffs, and Lessons Learned by Ruslan Meshenberg (Director of Platform Engineering @ Netflix) – A little more technical depth, but nothing overwhelming. Good presenter.
d. Video of What I Wish I Had Known Before Scaling Uber to 1000 Services by Matt Ranney (Chief Systems Architect @ Uber) – A lot of focus on the people side of microservices.

 

Advertisements

Two Years In Product Management: What I’ve Learned

About this time last year I did a blog post on what I’d learnt after spending a year as a product manager. It turns out that I picked up some additional things in my second year of product management that I believe are also worth sharing. Learning is really supposed to be a lifelong process so this shouldn’t be too surprising right? Okay here’s what I learned:

1. A Product Manager’s Greatest Contribution To their Product Team is Customer Insights: While product managers can wear many hats (UX, project management, analytics, e.t.c), I believe that the greatest contribution that a product manager can bring to their team is customer insights, or knowledge of their product’s buyers and users.
There are two reasons for this. First, customer insights are essential as most great products start with great customer insights. Without a deep understanding of the customer, great engineering, UX and analytical skills will not result in a successful product. Therefore, product managers have to make sure that their product teams never lack this important ingredient. The second reason why a product manager’s greatest contribution to their team is customer insights is that within the product team, the product manager is ultimately responsible for customer insights, just like the software engineer is responsible for delivering amazing code, and the UX designer is responsible for delivering compelling and delightful product experiences.

2. You Cannot Outsource Your Product Strategy To Your Most Important Customer(s): We all know how important it is to listen to customers. What may not be as obvious is that it is possible to listen too much to customers. This happens when product managers develop their product strategy and roadmaps based on customer input alone. This makes sense at some level right? If you want to build what customers want then just go ahead and ask them! What makes this approach risky is that in most cases, the customers that product managers get input from may not adequately represent their product’s current and future customer base. Customers who usually talk to us are tiny segments like the really big customers, the really sophisticated ones, or those who really love or really hate our product. In order to create a holistic product strategy, we need to consume and distill additional sources of customer insight such as actual product usage data, competitive analyses, and – where applicable – feedback from customer support and sales.

3. Data-Driven Product Development is Hard: A few months ago I did a blog post on data-driven product development: a systematic way for ensuring that product investments actually deliver the right outcomes to customers and the business. What I’ve learned from helping other product teams become more data driven, and from the data-driven journey of my own product team, is that this stuff is really hard to do. Few things suck as much as working on a new, “game changing” feature for months, and then discovering that the feature did not impact any of your key business metrics after release. The very real temptation then is to go back to the old days of celebrating the release of a feature without checking the impact of that feature on your business. This will definitely guarantee lots of good feelings all around within your product team, but it will not guarantee the future of your product, your team, or your current job. 🙂

That’s what I’ve learned this past year. What have you learned in your own role?

 

Three Things A Professional Poker Player Taught Me about Failure and Risk Taking

A few weeks ago I had the opportunity to hear a presentation on risk taking and decision making by a guy called Caspar Berry. If you’ve never heard of him before I’m not surprised; I also had never heard of the guy. What piqued my interest was the fact that he was a professional poker player, and was actually the poker adviser for the James Bond movie, Casino Royale. Pretty cool huh? 🙂

Caspar’s presentation was very entertaining. He has this uncanny ability to use a fast sequence of images on the screen to pass his message across while keeping his audience highly engaged. Here’s a short video clip of him I found on YouTube. If you get a chance to listen to him, I highly recommend it.

Caspar shared some good insights on failure and risk taking that resonated really well with me. Three interesting insights from his presentation are as follows:

1. Find Success By Taking Lots of Acceptable Risks
In business and life, some of us try to find success by looking for ‘sure things’. Unfortunately, these low risk opportunities usually have low potential rewards. Instead of looking for sure things, Caspar’s recommendation is that we look for great opportunities tied to acceptable risks. What qualifies as an acceptable risk? A good rule of thumb is that it should be outside your comfort zone, but before your zone of unacceptable consequences. For example, going to a happy hour with lots of people late in the evening is outside my comfort zone, while being unable to pay my rent is at my zone of unacceptable consequences. Somewhere in between these two examples is my version of acceptable risk; yours will be different.

2. Embrace Failure as an Important Ingredient for Success
When we talk about success, we usually mean an increase in a metric that we value, such as net worth, quality of relationships, or health. Similarly, when we talk about failure, we usually mean an increase in a metric that we don’t value, such as feeling rejected, incompetent, or uncertain. Where things get interesting is the fact that you cannot increase the metrics you value, without risking an increase in the metrics you don’t value. For example, you cannot improve the quality of your relationship with another person without a real risk of being misunderstood or rejected. As a data guy, this idea of thinking of success and failure in terms of metrics got me really excited. 🙂
In the end, the question we all need to answer is this: how much more failure am I willing to tolerate, so that I can have a shot at increased success?

3. Focus on Long Term Success (and Ignore Short Term Failures)
Bill Gates once said that “most people overestimate what they can do in one year, and underestimate what they can do in ten years.” A compelling long term goal provides the momentum needed to power through the inevitable setbacks that will occur along the way. Therefore, as you strive towards your long term goals, remember that you don’t have to win every single battle; you just have to win the war.

I think Casper’s insights on failure and risk taking can be summarized by a quote from Mark Zuckerberg: “I have more fear in my life that we aren’t going to maximize the opportunity that we have, than that we mess something up and the business goes badly.”  With this mindset, one can continuously take calculated and manageable risks towards the achievement of their life and career goals.

Image by http://www.solverde.pt

Make “Disagree and Commit” Work for You

The ‘Disagree and commit’ principle seems to have been created by Scott McNealy at Sun Microsystems, and adopted by other leading tech companies such as Intel. However, it was Amazon that brought the principle mainstream by making it one of its 14 leadership principles. The core message of disagree and commit is simple: conflict within a team is highly beneficial during the early stages of decision making as it helps the team to arrive at a better solution. However, conflict that persists after a decision has been made is quite harmful and prevents the team from executing effectively. In cases where you disagree with your team’s decision, you should still commit to helping your team execute flawlessly.

 Challenges with Disagree and Commit

The disagree and commit principle makes sense and can really help teams remain fast and nimble. However, adopting this principle is not easy. Two challenges I encounter as I try to disagree and commit are:
1. Disagreeing with Powerful People: I still find it difficult to disagree with more senior or influential team members even when I know doing so might help us get to a better solution. If this challenge resonates with you, check out Amy Gallo’s HBR article on how to disagree with powerful people. It’s pretty good.
2. Committing while I am Still Unconvinced: I personally value clarity and logic, so I find it really difficult to commit to something that doesn’t yet make sense to me. In an ideal situation, I’d like to wallow in the debate a little more so I can become convinced, but in cases where there’s no time, I realize I have to commit but that’s really hard for me.

Make Disagree and Commit Work for You

I did a little bit of research and arrived on five steps for improving my ability to disagree and commit. They are:

Step 1 – Ask for Permission To Disagree: This is quite useful in instances where you need to disagree with someone more powerful than you. According to Amy Gallo, a simple statement such as “I have some concerns with that idea and I’d like to lay out my reasoning. Is that okay?” gives the person a chance to become open to your idea and also helps to make you calm enough to engage in a difficult conversation.

Step 2 – Make Them Feel Heard: Paraphrase the original idea as a way to make the person you are disagreeing with feel heard. Another benefit of doing this is that it helps ensure that everyone is talking about the same things. You’d be surprised how often people find themselves talking about different topics within the same conversation.

Step 3 – Share your Thoughts in the Form of a Question: These questions should be about aspects of the original idea that you disagree with, or alternative ideas that you have. An example of a right way to do this is “Have you thought about this risk? or that alternative?”. An example of a wrong way to do this is “Can’t you see how shallow your idea is?”.

Step 4 – Ask for More Time To Research Alternatives: Sometimes, we are quick to assume that important decisions must be made immediately, but this is not always the case. If you disagree with the current direction of an important decision, ask for more time to come up with something better.

Step 5 – Take a Leap of Faith: This is where the rubber hits the road. If after steps 1 – 4, your team is still taking a decision that you’re opposed to, you need to take a leap of faith and commit. Some things to think about that may help this process are:
1. You aren’t always right, and this might be one of the instances where you are wrong.
2. You can give your team members the benefit of doubt because you believe they are smart, competent people working together for the good of the team. If you don’t believe this then you should find a new team.  🙂
3. The more you disagree and commit to other people’s ideas, the more likely they are to disagree and commit to your own whacky ideas in the future.

Any other ideas on how to effectively disagree and commit?

Notifications Are Killing Your Productivity. Kill Notifications Instead.

Have you ever picked up your phone to perform a task (send an email, reply an SMS, e.t.c), but found yourself struggling to remember what this task was as soon as you actually opened your phone?
This happens to me all the time, and while there may be other reasons for my strange, forgetful behavior, the key culprit I have found so far are the notifications pushed to me by my favorite apps such as Facebook, WhatsApp, and LinkedIn (I love LinkedIn 🙂 )

Notifications are helpful since they let you know where to hunt for your next media high. A notification from Facebook might mean that your friend has liked your post, created her own post, or replied to a post that you commented on. Lots of possibilities for connection and sharing! However, despite their usefulness, notifications can and do hurt your productivity in significant ways. Here’s two ways notifications kill productivity:

1. Notifications Increase Your Rate of Context Switching: Like Pavlov’s dogs which always salivated at the sound of a bell, we all have learnt to prioritize responding to our notifications in order to quickly receive the variety and pleasure that they promise. The fact that notifications now sit at (or close to) the top of the list of things clamoring for our attention means that we are constantly switching between them and everything else going on in our lives.
What is the cost of all this context switching? An article published by Inc. states that it takes us about 25 minutes to resume a task after being interrupted. Notifications constantly interrupt us, which means that they add lots of 25-minute resumption times to our already overflowing schedules.

2. They Distract You Whether You Respond To Them or Not: Your phone buzzes while you are in a meeting, and you resist the urge to pull it out of your pocket to see what’s up. Congratulations! But guess what? You spend the rest of that meeting wondering what type of media goodness is waiting for you once you leave the room. Whether you respond to them or not, notifications reduce your ability to stay be fully engaged during interactions with others, or to carve out time to do deep, meaningful work.

Taking Charge of Your Notifications
From my personal experience, and the experiences of others, I can think of two options for dealing with notifications and the threat they pose to our productivity:

1. Turn Them Off: Last week, I turned off notifications on all my favorite apps. This turned out to be the most productive thing I did all week as it significantly reduced my rate of context switching. Turning off all notifications can bring about a serious case of FOMO (Fear of Missing Out). However, my experience so far has been that other internal triggers such as boredom and curiosity keep you going back to your favorite apps with the frequency required for you to not really miss out on anything important.
Also, to remain accessible to close friends and family, I chose to keep receiving notifications from one chat app that almost no ones uses these days (don’t ask me which one ;)). You too can create your own secret communications channel!

2. Find An App For That: As with everything else, apps have started springing up to solve the notifications problem. I haven’t really tested any one of them so I can’t provide any feedback, but according to Trello, examples of apps trying to solve the notifications problem are Offtime, Flipd, and Moment.

Any other thoughts around how to manage notifications effectively?

I Want To Use Data To Improve My Product. Where Do I Start?

Most technology professionals will probably agree that data-driven business and product decisions deliver better outcomes than simply going with your gut, or worse: with the HiPPO. However, an important challenge remains for product teams trying to become data-driven: where do they start?

Below is a series of steps for navigating the path to data-driven product management. These steps could be applied to an entire product, or to a feature existing within a larger product. Also, while it is focused more on technology products, I believe parts of it could be applied outside the technology industry as well. Here goes:

1. Does your product work?: An awesome app that crashes every single time it’s opened is a worthless app. Therefore, the first set of data and metrics to be collected and analyzed should revolve around product health and operational intelligence. This will help you answer questions like how many hours in a day your product is available, what your crash rate is, and what your product’s performance is like during peak times.
Examples of publicly available product health and operational intelligence tools include Splunk and the elastic stack, or ELK.

2. Is it popular?: Once you get going with analyzing your product’s health, the next step is product and user analytics. This means collecting the data and developing the metrics needed to figure out how many people are using your product, and whether that number is trending up, or down. You also want to analyze your user base to identify your product’s user segments, and to begin understanding which segments are valuable, and what the valuable segments care about so you can tailor your product roadmap and marketing programs accordingly. This is not easy stuff, but getting it right pays huge dividends.
Publicly available tools for enabling product and user analytics include Localytics, Mixpanel, and Amplitude.

3. Are product updates changing key metrics?: Every update made to your product that results in no change in any of the metrics that you care about is probably a worthless update. On the other hand, increases or decreases in your business and product metrics may be as a result of external factors such as the weather or U.S elections. 🙂 One way to improve your certainty around what effects your updates are having on your key metrics is by running experiments. In very simple terms, this involves releasing product updates to a subset of your product’s user base, and then comparing the subsequent behavior of the users who  received the update with those who didn’t receive any product updates.
Publicly available tools for enabling experimentation include Optimizely and Unbounce.

4. Is your product Intelligent?: Artificial intelligence (AI) is all the rage today, as it promises to unleash a new wave of tech-enabled productivity on consumers and businesses. However, a critical ingredient for enabling AI scenarios is data. For example, the reason why Amazon can tell you that people who bought item X also bought item Y is because it has huge amounts of transaction data on both items. For your product to be able to support such intelligent scenarios, you need to collect and organize the relevant data sets, and then employ the right algorithms.
The process of enabling AI scenarios within a product usually involves custom development by software engineers. However, publicly available tools that can facilitate this process include Amazon ML and Azure ML.

Image by www.qubole.com

A Year in Product Management: What I’ve Learned

A few weeks ago, I celebrated my first year as a product manager. Much to my disappointment, I didn’t get a whole lot of congratulatory messages. Only LinkedIn remembered (thanks LinkedIn!).

Self reflection is something that I truly value, so I decided to take some time to think about what I’ve learned about the product management discipline during the past year. I came up with three key findings:

1. Wearing Multiple Hats is the Name of the Game
Before I became a product manager, my thoughts about the role can be summed up in this beautiful Venn diagram from the pm heels blog:

pm-ven-diagram

Now that I have a year’s worth of product management experience, I’ve modified this diagram to become something like this:

 

2. It is Important to Always be Working on the Most Important Things

From my product management Venn diagram, you can see that there is an infinite number of tasks that a product manager can be involved in. Yes those tasks can have UX, business, or  technology components. However, they can also involve random stuff like getting pizza for engineers who are working late (I haven’t had to do this yet but I happily will).

This brings us to the problem of prioritization. Since no one can do everything, product managers have to constantly check to ensure that whatever they’re working on has a higher chance of improving their product’s success metrics compared to other things that they could be doing. In case you’re wondering, there most definitely are some cases where the most important thing a product manager can do is get pizza for engineers who are working late. 🙂

3. Relationships Matter

I once heard someone describe product management as “the grease and the glue of product development”. It’s up to us to rally our stakeholders around a common goal, and to ensure overall progress towards the achievement of that goal. This means that we have to constantly influence our peers in design, development, and marketing. As none of these people work for us, it really comes down to influencing without authority.

What I’ve learned in the past year (and from folks like Robert Caldini) is that people who like and respect you are more likely to listen to and understand your ideas. In the same vein, you are more likely to listen to and understand the ideas of people that you like and respect. This insight has led me to always try to make relationship investments up front, before they were needed; by then it was usually too late.