Building an AI Product: Roles and Responsibilities

Machine learning is all the rage today, and lots of people are interested in a machine learning career. When I get a chance to chat with people that “want to do machine learning stuff”, I usually ask what roles they’re interested in, and they usually mumble something like “Data Scientist”, in a very unconvincing way. 🙂

The good news is that in addition to data scientists, lots of other roles contribute to the process of building AI products. To highlight these roles, let’s walk through the process for building an AI product, and see who we need along the way.

According to Yael Gavin, a former Facebook PM (who by the way has a great series of posts on machine learning for PMs), the process for building an AI product (or ML model) is as follows:

1. Ideation: This is when the product team comes together to align on their goals, customer problems they want to solve in order to achieve those goals, and metrics to measure success. While this stage is primarily driven by the product manager, engineering lead, and UX designer, additional contributors include the product analyst, who helps the team to think deeply about metrics, and the data scientist who helps the team understand how machine learning capabilities can help solve the customer’s problem.

2. Data Prep: Here the product team decides how to collect the data necessary for training their ML models, and computing the metrics needed to measure their success. This stage is driven by engineering, with help from the product analyst, who makes sure that relevant metrics can be calculated, and the data scientist, who makes sure that required ML model inputs (also known as features) can be produced from the data.

3. Prototype & Testing: This is the heart of the model building process. Here the data scientist trains the ML models and checks to make sure that the predictions or classifications from the trained models are within acceptable levels of accuracy. This highly iterative process is driven by the data scientist, with some help from the product analyst and engineering. The product analyst tracks model accuracy metrics, while engineering helps the data scientist understand the data, and delivers additional data where necessary.

4. Productization: At this stage the model is doing what it’s supposed to, and is ready for primetime. The product manager, UX designer and engineering lead the process of hosting the model in a scalable manner, and building the delightful user experience that leverages the machine learning model to actually solve the customer’s problem. Additional contributors to this stage include the product analyst, who tracks progression of metrics towards pre-determined targets, and the data scientist who monitors how the ML models perform in production, and makes improvements in an iterative manner.

As you can see, traditional product roles like product managers, software engineers, and so on contribute heavily to the process of building AI products. The reason for this is that your customers don’t care about fancy machine learning models; they care about delightful user experiences that allow them shop, learn, travel, connect and create more, but faster and cheaper.

The proven model for delivering these delightful experiences is still the goal-oriented, cross-functional, and autonomous product team. However, this team has a new member: the data scientist. What this means for other members of the team is twofold. First, educate yourself on the basics of machine learning. You don’t need to become an ML expert, but the UX designer for example should know as much about machine learning processes as he does about software engineering. Second, develop the ability to identify customer problems that were unsolvable in the past, but could now be solved with machine learning. For example, you should be able to tell if a customer problem sounds like a prediction problem, or a classification problem, so you can brainstorm with your data scientist on what is possible solution wise.


Tricks of the Product Management Trade

Every profession has tactics used by experienced practitioners to be more efficient and productive. Product management is no exception, and in the past few years I have picked up a few tricks of the PM trade that I believe are worth sharing. Three of them are as follows:

1. N.I.H.I.T.O: I learned this concept from the product management classes offered by Pragmatic Marketing. NIHITO is an acronym for Nothing Important Happens In The Office. What this means is that as a product manager, you need to get out of the building and frequently engage with customers to identify new problems to solve, and new opportunities to go after.
Product demos, customer visits, and stakeholder management rarely happen from your desk (unless you’re using video conferencing technology). This is why there tends to be an inverse relationship between the number of hours you spend at your desk per week, and your overall productivity as a PM.

2. Date for A Date: At the very early stages of a project, you usually don’t know when it will be delivered as there’s still too many unknowns. Unfortunately, this doesn’t stop your stakeholders from  constantly asking you for a delivery date! They’re not trying to be mean or anything, they just need to make their own plans which happen to be dependent on yours. Instead of telling them that you don’t know, share the date by which you will know, and spend the time between now and then identifying and eliminating the key risks of your project. Once this is done, you can hopefully share a still imperfect, but better informed estimate of when your project will land. This will go a long way to increase your credibility and professional reputation.

3. Meetings Before The Meeting: Your role as a PM might involve selling a complex idea to multiple stakeholders in a high-stakes meeting. If this meeting is the first time your stakeholders are seeing your idea, you will most likely fail. The reason for this is that this setting is actually the wrong one for having the detailed conversations needed to get buy in from your stakeholders.
The way around this problem is that before the big meeting, meet with key stakeholders individually to share your idea and give them the opportunity to raise their concerns. It’s a lot more work of course, but it dramatically improves your chances of success.

I hope you find my tricks helpful. Also, if you have tricks of your own please share! I’m always looking for my next career/life hack. 🙂


My Reflections on Parental Leave

Two weeks ago I returned to work after spending the last 12 weeks on parental leave. Since 2005, this was the longest period of time I had spent without being in school or at work, so I decided to reflect on the experience to see what I learnt during this unique period of my life.
My takeaways from my time spent on parental leave fall into three main buckets:
1. Gratitude
I feel immensely grateful for the opportunity to work at Microsoft, which has a very generous family leave policy. Giving me full pay for 12 weeks spent taking care of my family – in a country with no mandated family leave whatsoever – is quite spectacular. I really applaud our leadership team for deciding to adopt this policy.
In addition to Microsoft’s generous family leave policy, I’m also grateful for my awesome team mates who held the fort while I was away, and made my return back to work pretty seamless. I think the way in which we support each other when one of us needs time off work says a lot about the health of our work environment.
2. Admiration
During my time off  I developed a deep admiration for full time care givers. Before parental leave, I asked myself what I was going to do with all the “extra time”. During my time off, I found myself wishing for more hours in my day! 🙂  I also realized that a lot of the skills needed to care for my daughter were pretty similar to the ones I needed in my day job: planning, figuring out the root cause of an issue (yelling in this case :)), and finding a way to automate or eliminate repetitive tasks. In the end, I learnt that diapers, bottles, and whatever else is needed to take care of a child or elder parent are both time consuming and intellectually demanding. My hat goes off to those who do this full time.
3. Hope
In the past, especially in the culture where I grew up, men did not stay home to take care of their babies or parents. I’m excited to see a gradual change from that mindset, but the reality is that most dads are still unable to take parental leave. We still lack a critical mass of employers and governments willing to support us, and everyone’s personal circumstances is unique. However, for those who can take parental leave, my hope is that you go ahead and do it for three main reasons. First, the returns to your family is quite substantial. The arrival of a child is a huge change for any relationship, but if properly managed, that change can result in a much better relationship; that has definitely been the case for me. Second, if Anne-Marie Slaughter is right, then goals such as investing in the human capital of the next generation (by caring for children), or affirming our common humanity (by caring for older parents) are too important to be left to only 50% of the human population (women). Finally, quoting a dear friend and mentor: “only when men begin to consistently take advantage of parental leave policies at work will we see more parity and support evolve overall for talent in the workforce who also have goals of caring for their family through their time and presence”. 

3 Books I Read in 2017: Worth Their Weight in Gold

I decided to take a page from my buddy Rohan Rajiv’s playbook, and reflect on the books I read this year that really resonated with me. They are:

1. Inspired: How To Create Products Customers Love by Marty Cagan: This book is about nine years old, but it still totally nails the contemporary issues that product managers (PMs) face. What should be the nature of the relationship between product management and marketing teams? What about product management and engineering? How can PMs really understand customers, assess new opportunities, or recruit development partners? It’s all addressed in this book from a very practical perspective. There’s a new edition out that came out this month which I’m definitely looking forward to reading.

2. Daring Greatly: How The Courage To Be Vulnerable Transforms the Way We Live, Love, Parent and Lead by Brene Brown: Since I watched her insanely popular TED video, I’ve become a huge fan of Brene Brown and her work around vulnerability. This books dives into how to use vulnerability in key aspects of our daily lives, just like the title says :). A key learning for me was that having the courage to be vulnerable is not a sign of weakness; it is a sign of strength. Also, a popular but counterproductive way for us to deal with our shame is to look for and judge people doing worse than us in that area of our lives (parenting is a great example). Finally, when we take the time to reflect on the reasons why we feel shame, we usually realize that things aren’t as bad as we think. Great book!

3. Scrum: The Art of Doing Twice the Work in Half the Time by Jeff Sutherland and JJ Sutherland: Most product teams today build their products using some variation of scrum, which makes this book quite relevant as  it addresses how to get product development right using scrum. The book contains lots of good war stories about the scrum-adoption journey of very different organizations (I really liked the FBI Sentinel story), but the key insight for me was that great product teams are autonomous, cross-functional, empowered and goal-oriented.

That’s it! Cheers to a New Year, and to continuous learning!

Making (Business) Sense of Microservice Architectures

Every major cloud technology conference these days has multiple presentations about moving to a microservices architecture, so I did some research and tried to answer basic questions like what microservices are, what benefits and costs they provide to businesses, and when it might make sense to think about using them. My findings are summarized below; I think others may find it useful.

1. What Are Microservices?
The Netflix story is a great one for getting a basic understanding of microservices. Back in 2008, the Netflix website was one giant application, also referred to as a monolith in microservices lingo. The Netflix monolith had a lot of problems. For example, a simple error such as a missing semi-colon in the code base could bring down the entire Netflix website for several days. Also, developing new features was pretty slow as code changes (such as adding a new column to a database table) needed cross-functional teams to review and approve. Finally, and maybe most importantly, Netflix needed to move to the cloud so they could scale quickly enough to meet rising customer demand. Forklifting their monolith to the cloud wasn’t really an option, so they spent seven years adopting a Microservice architecture. This meant that they broke down their monolith into smaller, more manageable, and fully decoupled components, exposed through lightweight APIs.

2. Benefits
The benefits of adopting a microservice architecture include:
a. Scaling Efficiently: Teams can easily scale out the application components that need more resources (such as authentication & payments), without having to scale out the entire application.
b. Raising Availability & Fault Tolerance:  The decentralized nature of a microservice architecture eliminates single points of failure and delivers resilient, highly available applications that experience very little outage.
d. Getting Rid of Big Bang Deployments: Application deployment becomes a non-event as teams can deploy their services independently multiple times a day, without having to put the entire application in maintenance mode, or co-ordinate with other teams.
e. Giving Teams Technical Autonomy: Teams can implement their service using whatever technologies they prefer, as long as these services are exposed through standard APIs.

3. Costs
Despite their numerous benefits, microservices come with the following costs:
a. Microservices Imply an Org. Change: At its heart, microservice architectures are about creating the autonomous teams that can build, run and manage independent and loosely coupled services. This usually means a change from the current org. structure, and we all know how painful and emotional org. changes can be.
b. Application Architectures are More Complex: Every microservice architecture is essentially a distributed system, which means that things like monitoring, testing, and end-to-end visibility of the system become really really hard. Also, moving developers between teams becomes more difficult as their new team may work on a completely different stack from their current team.

3. When To Think About Adopting Microservices
According to Martin Fowler, a microservices thought leader, there are a few housekeeping items you need to address before moving your critical applications to a microservice architecture. They include:
a. Rapid Provisioning: You should be able to provision new servers in minutes and not days.
b. Monitoring: You need to know what’s going on within your application; where things are failing and why.
c. Rapid Deployment: Your deployment processes should be automated, and should take minutes and not days.

4. Resources
If you are interested in diving a little deeper into microservices, here are a few resources I found to be super helpful:
a. Blog Post on Why You Can’t Talk About Microservices Without Mentioning Netflix.
b. Video of Microservices Overview by Martin Fowler – Very good overview.
c. Video of Microservices at Netflix Scale: Principles, Tradeoffs, and Lessons Learned by Ruslan Meshenberg (Director of Platform Engineering @ Netflix) – A little more technical depth, but nothing overwhelming. Good presenter.
d. Video of What I Wish I Had Known Before Scaling Uber to 1000 Services by Matt Ranney (Chief Systems Architect @ Uber) – A lot of focus on the people side of microservices.


Two Years In Product Management: What I’ve Learned

About this time last year I did a blog post on what I’d learnt after spending a year as a product manager. It turns out that I picked up some additional things in my second year of product management that I believe are also worth sharing. Learning is really supposed to be a lifelong process so this shouldn’t be too surprising right? Okay here’s what I learned:

1. A Product Manager’s Greatest Contribution To their Product Team is Customer Insights: While product managers can wear many hats (UX, project management, analytics, e.t.c), I believe that the greatest contribution that a product manager can bring to their team is customer insights, or knowledge of their product’s buyers and users.
There are two reasons for this. First, customer insights are essential as most great products start with great customer insights. Without a deep understanding of the customer, great engineering, UX and analytical skills will not result in a successful product. Therefore, product managers have to make sure that their product teams never lack this important ingredient. The second reason why a product manager’s greatest contribution to their team is customer insights is that within the product team, the product manager is ultimately responsible for customer insights, just like the software engineer is responsible for delivering amazing code, and the UX designer is responsible for delivering compelling and delightful product experiences.

2. You Cannot Outsource Your Product Strategy To Your Most Important Customer(s): We all know how important it is to listen to customers. What may not be as obvious is that it is possible to listen too much to customers. This happens when product managers develop their product strategy and roadmaps based on customer input alone. This makes sense at some level right? If you want to build what customers want then just go ahead and ask them! What makes this approach risky is that in most cases, the customers that product managers get input from may not adequately represent their product’s current and future customer base. Customers who usually talk to us are tiny segments like the really big customers, the really sophisticated ones, or those who really love or really hate our product. In order to create a holistic product strategy, we need to consume and distill additional sources of customer insight such as actual product usage data, competitive analyses, and – where applicable – feedback from customer support and sales.

3. Data-Driven Product Development is Hard: A few months ago I did a blog post on data-driven product development: a systematic way for ensuring that product investments actually deliver the right outcomes to customers and the business. What I’ve learned from helping other product teams become more data driven, and from the data-driven journey of my own product team, is that this stuff is really hard to do. Few things suck as much as working on a new, “game changing” feature for months, and then discovering that the feature did not impact any of your key business metrics after release. The very real temptation then is to go back to the old days of celebrating the release of a feature without checking the impact of that feature on your business. This will definitely guarantee lots of good feelings all around within your product team, but it will not guarantee the future of your product, your team, or your current job. 🙂

That’s what I’ve learned this past year. What have you learned in your own role?


Three Things A Professional Poker Player Taught Me about Failure and Risk Taking

A few weeks ago I had the opportunity to hear a presentation on risk taking and decision making by a guy called Caspar Berry. If you’ve never heard of him before I’m not surprised; I also had never heard of the guy. What piqued my interest was the fact that he was a professional poker player, and was actually the poker adviser for the James Bond movie, Casino Royale. Pretty cool huh? 🙂

Caspar’s presentation was very entertaining. He has this uncanny ability to use a fast sequence of images on the screen to pass his message across while keeping his audience highly engaged. Here’s a short video clip of him I found on YouTube. If you get a chance to listen to him, I highly recommend it.

Caspar shared some good insights on failure and risk taking that resonated really well with me. Three interesting insights from his presentation are as follows:

1. Find Success By Taking Lots of Acceptable Risks
In business and life, some of us try to find success by looking for ‘sure things’. Unfortunately, these low risk opportunities usually have low potential rewards. Instead of looking for sure things, Caspar’s recommendation is that we look for great opportunities tied to acceptable risks. What qualifies as an acceptable risk? A good rule of thumb is that it should be outside your comfort zone, but before your zone of unacceptable consequences. For example, going to a happy hour with lots of people late in the evening is outside my comfort zone, while being unable to pay my rent is at my zone of unacceptable consequences. Somewhere in between these two examples is my version of acceptable risk; yours will be different.

2. Embrace Failure as an Important Ingredient for Success
When we talk about success, we usually mean an increase in a metric that we value, such as net worth, quality of relationships, or health. Similarly, when we talk about failure, we usually mean an increase in a metric that we don’t value, such as feeling rejected, incompetent, or uncertain. Where things get interesting is the fact that you cannot increase the metrics you value, without risking an increase in the metrics you don’t value. For example, you cannot improve the quality of your relationship with another person without a real risk of being misunderstood or rejected. As a data guy, this idea of thinking of success and failure in terms of metrics got me really excited. 🙂
In the end, the question we all need to answer is this: how much more failure am I willing to tolerate, so that I can have a shot at increased success?

3. Focus on Long Term Success (and Ignore Short Term Failures)
Bill Gates once said that “most people overestimate what they can do in one year, and underestimate what they can do in ten years.” A compelling long term goal provides the momentum needed to power through the inevitable setbacks that will occur along the way. Therefore, as you strive towards your long term goals, remember that you don’t have to win every single battle; you just have to win the war.

I think Casper’s insights on failure and risk taking can be summarized by a quote from Mark Zuckerberg: “I have more fear in my life that we aren’t going to maximize the opportunity that we have, than that we mess something up and the business goes badly.”  With this mindset, one can continuously take calculated and manageable risks towards the achievement of their life and career goals.

Image by