Assumptions that would otherwise be impossible to make

And other pleasant side effects of hiring & developing the best

Disclaimer: This article represents my personal opinions, experiences & learning and is in no way representative of others at Amazon.com or of Amazon.com itself.

14 months ago, when I was offered an opportunity to work at Amazon, I felt good about making it through the interviews. The interviews were insightful, challenging and engaging. Just like any rewarding intellectual discussion should be. At the time I did not understand why the interviews were designed like that and why all of my interviewers kept talking about leadership principles as if they were powerful tenets. Once I started working here, I learned a lot more about the leadership principles. They really are powerful tenets that break down the decision making process to fundamentals of both personal traits and business. They deserve a separate series of articles by themselves so I will not digress. You can read more about those here.

One of the leadership principles at Amazon is to "Hire and Develop the Best" which is best summarized by this definition.

"Leaders raise the performance bar with every hire and promotion. They recognize exceptional talent, and willingly move them throughout the organization. Leaders develop leaders and take seriously their role in coaching others."

Interestingly enough I find the spirit of this principle even more strongly represented in the opening text to the leadership principles.

"Whether you are an individual contributor or the manager of a large team, you are an Amazon leader."

When I actually started hiring software engineers, technical program managers and web developers; I learned that the hiring bar was definitely higher than where I thought it was. At first I found that to be ridiculously unreasonable. Eventually it started to make sense. It became so obvious that I felt stupid for not always having approached hiring in this way. Combining a high hiring bar with continuously growing and developing the people that you hire creates very interesting and strong ambient forces at the organization. I must share some of the assumptions that this allows management and peers to make simply because I find this very liberating and exhilarating.

People who meet a high bar of leadership principles are natural ownersSo you look for people who can own problems, solutions and customer experience. People who act in the same way whether you are looking at them or not. People who have very high standards for themselves and others. People who are experts at their jobs. Such people are natural owners and do not require supervision. This allows us to create small teams that can work independently and can be trusted to do the right thing.

When builders build then innovation is natural
If you create small independent teams of builders who can build without supervision then you create bubbles of innovation. Every individual and team starts working in a natural state of self-driven problem solving and innovation. This is fundamentally not very different from what drives start ups to innovate and be creative at a pace that is vastly different from large organizations. The same concepts and practical setup allows Amazon teams to work more like start-ups than like 'departments'.

If you can assume trust, transparency and honesty then productivity soarsLeaders who meet a high bar have to be self-critical, aware of their surroundings, willing to learn fast and be transparent. Once you take any grey areas associated with these traits off the table then nobody has to look over their shoulders. That means people, their managers, their managers' managers to the executive leadership can focus their entire energies on doing the right thing instead of supervising and 'keeping an eye' on other people.

A productive and innovative team drives big thinking and great solutionsSo I have a productive team with builders building innovative solution almost all the time. What happens as a natural outcome of that? Big thinking! You start thinking about solving problems bigger than you can otherwise imagine a team of five engineers to be solving. You start having people building 'one click buying', 'cloud computing' and many such amazing solutions. One of the senior engineers explained this to me beautifully.

"I am not really looking for someone who just knows the technical difference between a list, a map and a set. I am looking for someone who reads a customer feedback about wanting to share shopping ideas with their friends and then goes on to actually build a shared shopping list with a scalable and reliable design"

What he was saying was a blend of thinking big, inventing and timely implementation of ideas.

Great people can be developed quickly by giving them more responsibilityThis structure allows for expecting more from people, giving them more responsibilities and allowing them to succeed or fail fast. Coupled with smart peers and honest feedback this allows rapid development of individuals just like that of software in the same environment. Once you give people more responsibilities, you still don't have to supervise them. You can still continue to assume that they will be owners.

And all of that creates another virtuous cycle...This flow of hiring great people, making them owners, allowing builders to build in innovative teams. These people working in an environment of trust, transparency and honesty drive big thinking to solve important problems. This environment allows people to develop fast and take on more responsibilities while delivering great products and services for our customers. This is yet another virtuous cycle that I see around me and that I am still so excited to be a part of. There is obviously constant room for improvement and making it even better but I would absolutely love to see this approach adopted by other large organizations to create an even larger eco-system of amazing innovation!

Building a ship versus building a boat

Understanding the intricacies of designing a large scale distributed system and why being agile is critical...

The industrial revolution made possible the use of new materials and designs that radically altered shipbuilding. The last two decades have been like an industrial revolution for software design and development. Just like the availability of steel, manufacturing processes, education, pre-fabrication techniques and the production line changed the scale of ships; The availability of cloud computing, a global customer base, ready to use components, a broad adoption of service oriented architecture and the Internet have come together to change the scale of software. Just like the difference in building a boat versus building a ship; Building a large scale distributed system is fundamentally different from building a small custom application. They are different processes with different focus areas and different assumptions which just look deceivingly similar.

Not built by one; not built for one

Most software today is not built from scratch and does not have a closed or well defined group of builders or users. Take a moment to think about that. Most boats have the same set of people working on it from its start to its end. They can make assumptions about everyone knowing everything and all of the compromises and what not. Most ships are built over years. People join and leave the builder team. And the ship continues to be tweaked, modified and upgraded years after it sails out of the shipyard. Doesn’t that sound very similar to the way software is ‘shipped’ today?

Software teams today need to start with that assumption that team members may not be around as the software continues to be developed and matures. Establishing that premise from the start enforces stricter contracts between components, reduces dependence on assumptions and drives better design as well as documentation. Moreover any large scale distributed system consists of multiple components spread across teams so following established design patterns and simple, clear contracts is very important

Similar assumptions and premises need to be established for a customer. When designing a large scale distributed system you are generally not targeting one known customer. You are targeting a ‘use-case’. You are targeting a problem that is valuable to solve and that will likely remain valuable over a long term. So while user groups and active feedback are important tools, they definitely fall short of being comprehensive for the modern software world. You have to build on premises of continuous evolution of the use-case and the user. That calls for simplifying and automating the voice of the customer. A/B testing, instrumentation, automated behavioral inputs, and in-flow feedback become more and more important as compared to concerted one-off events.

To make these assumptions requires clarity of the goal and big thinking. Don’t limit yourself to a developer or a customer. Go beyond thinking how your solution will be used in the next 2 years. Your design assumptions and tenets need to be bold. Compromises can be made during implementation as that is bound to be iterative. A bold, long term oriented design which does not assume a closed group of builders or users will naturally enable an iterative, improvement oriented implementation cycle.

Don’t build all of your tools

Builders enjoy building from scratch. It’s fun, rewarding and a great learning experience. It works great for small projects but to design and build anything larger requires a strong adherence to “don’t reinvent the wheel”. No modern large scale project is built from scratch. Ship builders buy entire hulls pre-fabricated by other companies. At the very least a large number of complex components such as electricals, engines, cooling systems etc. are usually built by other companies independent of a particular ship’s requirements. These systems are designed to suit most ships. Similarly modern computing eco-system is replete with reusable components, libraries and complete solutions. Some things we have already started taking for granted; For most web services, you usually don’t have to worry about managing a data center, building a storage system or solving networking problems. The range and scope of things that you don’t have to do is rapidly growing. For example you don’t have to build a messaging or queue based communication system between components. SQS & SNS are existing solutions that you can use off the shelf. You don’t have to build an optimized key-value store and choose from a plethora of options such as MongoDB or DynamoDB. You don’t have to build a scalable cache, you can just use a solution like Elasticache. Going forward you may just be able to write a piece of code that responds to an event without worrying about any of the underlying complexity! (See AWS Lambda). This is not intended to be a list of AWS services but I use them as obvious examples.

The point that I am trying to make is that a lot of design time is and should be spent on deciding on the unique and valuable parts of your solution. I will refer to Guy Kawasaki from his “The art of getting customers”. He says that in today’s market if you want your solution to be a successful one then it needs to be both valuable and unique. That is such a simple and powerful notion. So as a group of designers and developers, you need to maximize the time that you spend on your core valuable and unique solution. For anything that is not core, you need to be strongly biased towards existing components or systems that can be reused. That touches upon a key Amazon leadership principle as well, Invent and Simplify. Simplify. Invent. It is incredibly important to simplify your solution down to its very core. When I hear people say that their website’s value proposition is fast response and data retrieval, I start with the assumption that its not. Those are essential requirements that can be met by existing solutions. That is generally not a unique value proposition.

Don’t boil the ocean; Think big, build small

This write up would be incomplete without touching upon the other end of the spectrum of software design; Implementation. ‘Think big, build small’ is being proven to be the most successful model of development. Design for 10 years but build for just a few months. Why? The answer to that is rooted in understanding that platforms, components and systems are continuously evolving. No modern software system exists in isolation. So if you don’t need a functionality implemented right away, don’t implement it. Chances are that someone else will solve it by the time you need it and you can just reuse what they build. The ecosystem will continue to change as your solution is being built. It will either completely solve, greatly simplify or deprecate the implementation that you are thinking of. So unless it will be used by someone in the very near future, defer development. At any given point of time, investing your key resources in the most important, most immediately needed implementation while adhering to a bold, long term design is as of today the best known approach in my experience. That’s why being ‘agile’ is critical. A lot of developers that I talk to during interviews treat being agile as ‘being aware of constantly changing customer requirements’. That is only part of the picture though. It is equally important to be both externally and internally aware beyond the customer requirements. Knowing the ecosystem and constantly updating the assumptions on what is readily available, what can be reused, what the current strengths of the team and how best to utilize these assets are strong aspects of agile development.

I will close this out with an anecdotal story. A team of 3 developers, one strong in web technologies, one in core computing fundamentals and one mathematician started developing a phone app that managed wish lists. They spend the first 3 months of implementation designing a data model and a custom data store to store dynamic wishlists. Then the mathematician left the team. Another guy joined the team. He was a contributor to MongoDB. The ended up throwing away what they had built in the first 3 months and re-designed a much better data store using existing solutions in 1 week.

When building a ship, optimize for time, resources, long term and constantly updating reality.