The example above is just one among many examples of wrong judgement in interviews. If you run interviews for long enough, you w’ll admit at some point that you recruited some suckers that looked amazing in the interview and rejected some brilliant people that under-performed (at the light of your judgement). There is a lot to be told about this subject and although I’m not any expert in the matter, I will share some of my thoughts about it.
“Clearly, it is unrigorous to equate skills at doing with skills at talking..”
— Nassim Taleb
One of the most common mistakes is to confuse specialists knowledge in some subject with expertise on doing it. Taleb coined the term “Green Lumber Fallacy” in his book Antifragile, to describe the difference between theoretical knowledge from “specialists” about a subject and real expertise in its practice. In the book What I Learned Losing One Million Dollars it is narrated at some point the story of one of the most successful traders to ever buy and sell green lumber (fresh cut wood) actually had no idea what he was trading (he thought the wood was painted green). His ignorance of the product had no impact on his ability to make money trading it. Taleb outlined in his book a similar situation, where a star Swiss Franc trader who’s inability to locate Switzerland on the map didn’t hinder his ability to make money trading its currency.
Despite their ignorance of the product they trade, those two traders understand the risks involved in trading it. That knowledge is more important than knowing European geography or being intimate with green lumber. Ideally we want to find candidates with both types of knowledge, but the truth is that it is very difficult to find those candidates and we don’t want to lose talented people that fail in some not so relevant quiz question.
I call those talented people that for some reason lack the skills to excel in interviews DOERS. DOERS typically don’t talk too much, don’t lose time with elegant formalisms, but get things done. They deliver quality, are accountable and available to the team whenever they need. Many of the most talented people I’ve worked with are DOERS. Their value may not be obvious in a first impression or interview, but it is revealed as soon as someone works with them.
Of course there are also people very talented with talking. They know very well how to express themselves and how to communicate with others. These people easily create empathy with others and can use it, sometimes, in their advantage. I call this type of people TALKERS. People with communication skills have an easier task to create good impression, having advantage on creating a good perception of competence in others.
Besides the communication style and personality traits that influence how well someone communicates there are people with a very strong critical thinking that do a lot of research and reflection about a lot of fields, have intelligent speeches, use logical arguments and commonly bring esoteric subjects to discussion. I call those people INTELLECTUALS.
From the three traits I identified, DOERS, TALKERS and INTELLECTUALS, only one is needed to be a good software developer: the DOER. The other two are optional. Maybe desirable for some more specific roles, but not essential for a software developer role. Of course we all love to work with people that accumulate the three traits but it is very difficult to find it in the very same person. I call the people with the three traits the FEYNMANs, in honor of the charismatic physicist Richard (Dick) Feynman.
“The best hiring for the team”
Some years ago I collaborated with a team that was recruiting new developers and a new member was hired. The newcomer was labeled as rock star. He came from a renowned company with a big salary and a very impressive history of entrepreneurship. The company did a big effort to match his salary and he started as the company top payed software developer. Two years later he left the company after two consecutive negative evaluations and was by far the most under performer of the team. This guy was some kind of INTELLECTUAL TALKER but he lacked the only essential trait to be a successful software developer: to be a DOER. He never delivered working software. The complaints about his work was unfinished work, not working code, not tested code. He never admitted it and pushed complains and excuses around. I call this people with talking skills and some street smarts to fool others with sounding sentences INTELLECTUAL BULLSHITERS. The negative impact of having such person in a team can be huge. Beyond all the damage the person created with the poor quality of the work, you can imagine how other team members, those you depend on to deliver, would feel when they find out that guy earns the company top salary. It will undermine the trust in the management and destroy the company culture. In this case the person left the company on his own feet after two years, the management did nothing to solve it. The problem could be solved at the very beginning as soon as it became obvious the person didn’t match the minimum. Gary Vaynerchuk has a very good summary about how to deal with those failed hirings:
How to not get fooled then? It is important to keep in mind the first idea I presented: talking and doing are two different skills that work independently. This allows to approach the candidates with a broader mindset, accepting the case where a bad interview can give a good candidate and where a good interview can give a flop. This kind of scepticism when judging the candidate is a first step for a very important mindset to keep: we are fallible at judging others. Recognising our misjudgments, we can more easily take measures to deal with it and act as soon as the problem is identified.
Gary’s advice should be taken into account and be used when the culture of the team is at risk. But of course it is not the solution to all recruitment problems, like implementing a process where we accept every candidate and fire those we don’t like. Actually there are companies with that kind of approach. Simon Sinek in his book Leaders Eat Last describe some companies with a toxic culture where the under performers are fired and the top performers get raises and promotions. In those companies there are no culture of collaboration (guess why), workers don’t stand for too long and they are typically time bombs waiting for collapse. Fortunately I don’t know many of those kind of companies in the IT industry. Recognising casting errors and dealing with it is important but we still need to look for a match in interviews, at least most of the times.
“Birds of a feather flock together”
The single best heuristic I know to hire someone is to ask the people you trust for referrals. Top performers like to work with other top performers. They are critical with they own judgement and are typically select with recommendations. When referrals are not possible, looking for the places where the candidate worked before and checking what he accomplished there can give good indicators.
There are lots of public texts from companies that write about hiring processes. Google was known by the demand in the hiring process with lots of interviews with different people but abandoned it in favour of a more lightweight process with more focus on cultural fit. Some companies use tools like codility to perform practical exercises or use some scripts with practical exercises to realise during the interview. Typically in the interview the two main concerns of the interviewer is to understand the cultural fit and the technical capabilities.
Let them sing
The most effective way I know to evaluate someone technical ability is to watch his work. And that is what some companies do, or at least try to simulate in some way. Toptal is known by including in the hiring process a small project where some requirements are defined and the candidate has a deadline to deliver it. VW include in the interview process a full day of working with some members of the team. There are companies that include some trial period where it is given some payed work that can be done after hours or during the weekend. In fact if we look for tools like codility or the exercises provided during the interview, they are trying to replicate some technical tasks in a way they happen in a real project. The difficulty lays in creating a fair judgement in the short interview period, where the candidate is pressed with time and has a poor context and no real world help, like co-workers or stackoverflow. Those companies are trying to create a more real world alike exercise.
Evaluating technical skills is just a matter of looking to the result of the work of the candidate. The problem is that a real measure of work may take several hours or days to perform. The challenge is to find a well balanced trade-off between the recruitment process duration and the quality of the feedback that is possible to obtain.
But even with well designed processes, sometimes we find candidates with a completely different background experience, used to solve problems in a completely different problem space from the one we work with. In my experience those are the majority of the candidates. The software development spectrum is so big that we hardly find someone with the kind of experience we are looking for. Asking questions about distributed services nuances to a candidate that spent the entire career developing stand alone desktop apps is not very useful. Most of the time in the interviews we are struggling to find a common place of expertise to bring the discussion. Many times it is frustrating. Many candidates are not familiar with your problem space and except those INTELLECTUALS that may understand your problems without never worked on it, most of the people will fail to solve your puzzles. The interview is transformed into some kind of intuition room where you spend 50 minutes trying to confirm the impression you made in the first 10 seconds.
Either by interviewing juniors with very few experience or more senior candidates with complete different backgrounds and lack of expertise in the problems you are looking for, most interviews are much more about evaluation the candidate potential than to assessing skill. Designing a bunch of puzzles to evaluate a pre defined set of skills you are looking for is one thing, finding out the potential of candidates to learn to solve those puzzles is another completely different.
A heuristic to access potential is to evaluate the level of expertise that someone has in a subject he is comfortable with. Someone with a very deep understanding of the subjects he works with gives a very good prospect about what he can do. But doing this may not be trivial. It implies to invert the interview logic to focus on the candidate strengths instead of delivering a “standard” set of questions to see how his skills match what the team needs. This inversion in the interview logic creates, theoretically, a more subjective evaluation process, where each interview became more singular in the way it can evolve. It is also much more demanding to the interviewer, sometimes the area of expertise of the candidate is far from reach from the interviewer. I remember a story from a senior developer who worked for a trading company for several years using Java. He went to the job market to find a new job but he had some hard time to pass the HR interviews. The problem was he never worked with the fancy new web frameworks that appear in all job orders. The screening process from HR departments never considerate him. When he finally got a chance for a technical interview he started to draw in the whiteboard the architecture of the trading software he worked on. He carefully explained all the details in the solution, discussed all the important tradeoffs and showed a great understanding of all the underlying principles. The two interviewers where unbelieving about the candidate they got at hands, he was immediately hired. He is now one of the lead architects of the company.
The candidate from the previous story was not only a technical fit for the team but also a cultural one, being promoted latter to a key role in the company. Accessing someone skill or potential may not be an easy task but even if we get it right, that is not the end of the story.
It’s all about the culture, stupid!
Building a culture is that thing that everybody talks about and wants to do but don’t know very well how to do it. But in the end, it’s all about the culture. We all know that story about the rising star that is not a team player. And we all know how the team ended, winning nothing. No single person worths a team, or a company. But sometimes a single person is enough to undermine an entire structure. When someone talks about some new recruitment the first question asked is always: is he/she nice? (if the newcomer is a woman maybe someone will ask first another question). No wonder that companies put a lot of effort to find a cultural fit. Some companies even put more value in cultural aspects than technical skills.
“Clients do not come first. Employees come first”
There are many examples of companies where the culture is a flag. Where taking care of the employees is at the center of the concerns. From all known examples of companies with strong cultures, hardly any company beats the Taj Mahal Palace Hotel story.
On Nov. 26, 2008, terrorists simultaneously attacked about a dozen locations in Mumbai, India, including one of the most iconic buildings in the city, the Taj Mahal Palace Hotel.
For two nights and three days, the Taj was under siege, held by men with automatic weapons who took some people hostage, killed others and set fire to the famous dome of the hotel. (…)
Apparently, something extraordinary had happened during the siege. According to hotel managers, none of the Taj employees had fled the scene to protect themselves during the attack: They all stayed at the hotel to help the guests.
“I was told many stories of Taj hotel employees who made sure that every guest they could find was safely ferreted out of the hotel, at grave risk to their own lives,” Zakaria said on his program.
Often during a crisis, a single hero or small group of heroes who take action and risk their lives will emerge. But what happened at the Taj was much broader.
During the crisis, dozens of workers — waiters and busboys, and room cleaners who knew back exits and paths through the hotel — chose to stay in a building under siege until their customers were safe. They were the very model of ethical, selfless behavior.
To understand why the employees did behavior like that, the Harvard Business Review did a research about the hotel’s HR policies and found the following:
“What was interesting about all those interviews with senior management was that they could not explain the behavior of their own employees,” he told me. “They simply couldn’t explain it.”
First, recruitment. In their search to find maids and bellhops, the Taj avoids big cities and instead turns to small towns and semi-urban areas. There the Taj develops relationships with local schools, asking the leaders of those schools to hand-select people who have the qualifications they want.
“They don’t look for students who have the highest grades. They’re actually recruiting for personal characteristics,” Deshpande says, “most specifically, respect and empathy.”
Taj managers explained to Deshpande that they recruited for traits like empathy because that kind of underlying value is hard to teach. This, he says, is also why recruiters avoid hiring managers for the hotel from the top business schools in India. They deliberately go to second-tier business schools, on the theory that the people there will be less motivated by money.
“India is a country where people are almost obsessed about grades. In order to get ahead, you have to have really high grades. But here is an organization that is doing just the opposite — they’re recruiting not for grades, they’re recruiting for character.”
The Taj is owned by a corporation called the Tata group, which for the past hundred years has been run by an extremely religious family that’s interested in social justice: The company typically channels about two-thirds of its profits into a charitable trust.
But Deshpande says there are also practical reasons for this focus on character. The Taj hotel has made its name on customer service, and they are near maniacal about it, treating it almost like a science.
For example, managers have mapped the number of interactions that happen between customers and hotel employees in a typical 24-hour stay. There are on average 42, often unsupervised, interactions between employees and guests.
Each of these interactions is viewed by the company as an opportunity for employees to delight their customers with their kindness. So everything — everything — about the training and rewards systems set up by the Taj is designed to encourage kindness.
This story provides a very compelling lesson: “Corporate design is absolutely critical, for good, and for evil.”
Finding the right fit for the team and the company is hard. We need to know that doing and talking are two different skills. Struggle to avoid to be fooled by it, in both directions, either by reproving good candidates or passing bullshiters. We need to be humble and admit our poor judgment capacity and how it can hurt our team culture if we don’t act after it. We need to extract the maximum amount of information in a very small time window, ideally by evaluating candidate skills in some real world look alike puzzles. But mostly, we will be frustrated with the gap between the expected experience and the candidate real experience. We will struggle with the software development spectrum to find a commonplace to bring the discussion and find that most of the time we are assessing much more the potential than the skill. Finally we need to frame all this technical concerns within the company culture, because, in the end, the culture is what matters the most.