As a software engineer, I was trained to think about the worst case. What is the worst possible performance for the algorithm used as specified in O(n), “big Oh“, notation? If you had formal education in computer science, you have been asked this question. Given that we are supposed to think about the worst case, why are there so many problems in the software development industry?
One post that got me thinking about this problem comes from Alex Payne, where he demands that people don’t be a hero.
Every team I’ve ever worked on has had a hero. You’ve probably worked with one too: the guy or gal eager and willing to pull all-nighters, work weekends, and take over on-call duty when nobody else wants to…
Because of this type of effort, this person has praise heaped upon them. Sometimes the company may even acknowledge their efforts publicly. Sometimes this praise is a good idea because that person may have save the company millions of dollars. However, should the hero always be praised? Alex does not think so:
The hero is celebrated. The hero is a role model…
Here’s the thing: the hero is the most damaging person on a team, particularly on a team that’s supposed to be writing high-availability or otherwise mission-critical software.
I may not entirely agree that the hero is the most damaging person on a team, but there is a significant problem with the hero. The hero treats the symptoms of the project, like lateness or defects, but the hero ignores the disease. Why are there so many defects? Why is the project so late? In some cases, it could just be a bad team. Sometimes a team just does not work well together or can not be productive. It happens. There may also be unproductive or unskilled members of your team. Typical hiring practices do not always give you an insight to the type of skill someone has or what type of worker they are. The hero will help the late project suddenly get back on schedule. This means that when a project review is completed, people will just remember that heroic effort saying “Without that we would have been late!”
Why Do We Need A Hero?
So, why do we so often need a hero for our projects to be completed successfully? There are some cases where people have severely underestimated a task, or there is a new technology that took longer to learn than expected. In my experience, those issues do arise but not as often as poor scheduling. Poor scheduling comes in many forms. There is the simple bad estimation problem which all engineers have. Sometimes we use different units of work when we estimate as compared to how the work gets done. For example, a task may take 8 hours, so we add it to the plan as taking an engineer one calendar day to complete. In almost all cases, the engineer does not code for 8 hours per day. The real number of coding hours is probably 6 per day or even as low as 4 depending upon the company. Obviously, blindly adding task estimates to a calendar does not provide a good or valid plan.
Estimation and planning has become such an issue for software development that people have written blog posts that were meant to be humorous but contain too much truth to really be funny. I was reminded of one such post by a colleague last week. Liz Keogh wrote about Estimation Anti-Patterns about two months ago. Much of the post is somewhat funny, but I have been involved in projects that used some of these types of estimation methods. In Liz’s post, underestimating tasks is typically due to Fractal Estimation:
Some stories may be considered too large to fit into an iteration, so the team decide to split them into the smallest possible deliverable slices, re-estimating each slice. Of course, this is a bit like measuring the coast of Britain. The epic which was 13 points at project inception becomes 16 points of features, and 23 points of stories by the time it’s done in the sprints.
Another often seen estimation and planning problem is similar to Liz’s Target Estimation:
PM: How many story points can you manage this week?
PM: That’s not good enough. I need you to do 12. How many can you manage this week?
In some cases, you will see this manifest in “Contract Driven Estimation”, where someone sold the new and not yet developed feature, as well as the date when the feature would be in production. Other projects may also miss deadlines due to the “Shortened Due To X Estimate”. X could be the budget available or some other future constraint. The problem is that the work is not lessened, just the project plan changes.
How Do We Avoid The Need For A Hero?
I have not painted a pretty picture of software development up to this point. Software development is hard, and the accompanying tasks of estimation and planning are just as hard. The reason for this is the complexity involved and the lack of physical rules. For example, in mechanical engineering, there are many physical rules that you must follow, so at the basic task level, a majority of engineers will be able to do the same job in approximately the same time. Those engineers can differentiate themselves through design of complex systems or even just finding a better way to do things.
Software engineering does not have those boundaries. Some people say that it is more art that science. Because of the lack of these restrictions, estimation of tasks is not only inexact, it is almost impossible to consistently do well. Agile methods were born out of the frustration of various software development methodologies. Much of this frustration came from the missed deadlines of longer projects. These agile methods do not improve your estimation abilities at the beginning of a project, but they may use the history of the team to more accurately predict how the project is actually progressing. By using the amount of work completed in the most recent iteration, you can better estimate what will be completed in the current and potentially future iterations. Typical planning processes do not give you this ability. Typical planning assumes that productivity is constant, while we know that a single person is not consistently productive at the same level for a long period of time. A team will have ebbs and flows of productivity among the various iterations.
So, in order to leave you with a few concise bullets that could help improve your development projects, I will try to summarize the ideas:
- Personalities are important, so make sure that your team can work well together.
- Software engineers are not replaceable parts, some people are just that much better than others.
- Estimates of tasks are likely not equal to actual calender hours. Ensure you know how they translate in your company.
- Agile planning uses recent team history, “yesterday’s weather”, to determine how the team will perform in current and future iterations.
- Avoid planning back from the deadline. Do not pick a target date and try to figure out how to make the date, it normally will not work.
If you follow this basic advice, it is likely your project will go smoother than it has before. I can not promise better results for your team, because your team is the one doing the work. Also, I know this is not a thorough investigation of why software development projects fail. This is based on personal experience and various things I have read over my career. Are there some major issues or great advice that I missed? Feel free to add your thoughts in the comments.
11 thoughts on “How Can Software Development Avoid The Need For A Hero?”
Your insights about software development apply to project management as well. There almost always seems to be a “hero” who does the work of ten people in less time to get the job done.
The problems are not really specific to software development, but that is where my experience is. Estimation and planning are basic project tasks regardless of the industry. Some industries do it better than others.
[…] This post was mentioned on Twitter by Rob Diana and Ibrahim Saracoglu, twitips MV. twitips MV said: How Can Software Development Avoid The Need For A Hero?: As a software engineer, I was trained to think about the … http://bit.ly/4RjcxE […]
“Avoid planning back from the deadline. Do not pick a target date and try to figure out how to make the date, it normally will not work.”
This is interesting to me. I see what you’re saying, but what about situations where a target date is already set? I’m thinking especially in terms of clients who are doing other campaigns or events. I typically just try to manage their expectations and get them to scale back if necessary so that the deadline is realistic, but should I spend more energy getting them to push the deadline?
As usual, I just was not clear enough. What I was trying to say was you have the list of things you want to do, and you have a deadline, then you try to create estimates that make everything fit together. Like you said, you need to adjust scope in order for this to really work, but sometimes people just refuse to do such a thing. If you can push the deadlines, that would work as well.
your story about the software development i found is really interesting and somewhat related to mine as i also had been a part of many past development.some people become hero by thrashing the all other members of the team..
[…] How Can Software Development Avoid The Need For A Hero? (regulargeek.com) […]
I have ran into a manager that can do milestones by a certain deadline. He estimates a project with that in mind, and seems to ignore when I claim that I can’t make those deadlines. He got to his manager position by being a great programmer, he was even recognized as a “hero” by the company, and I’m usually in the position of not being as experienced as him on various milestones, and find his estimates to be more based on what he can do, not on what I can do. My arguments about pushing back deadlines so I can make them usually get refused. Is there some way I can make him understand that his project estimates are unreasonable for ME? Or should I just find a new job?
While I would not recommend that you find a new job, that is really your decision. One of the things that I always try to do is ensure that estimates are based on who is doing the work. If the people doing the work do not agree with the estimates, that can quickly become a very bad situation for the people doing the work. If your manager and other managers do not see this as a problem, then you may have a decision to make. I believe it was Martin Fowler that said something like “If you can’t change your organization, change your organization”.
Hero programmers are usually a failure of management. Either they expect too much from one individual, or too little from the rest of the team. I was lucky in my career. My first manager was awesome, the kind of manager programmers dream about. She saw that I was morphing into the hero programmer, and pulled me aside to remind me that development is a team effort. She asked a lot of questions, and after our conversation, put in a request to HR for an additional developer. She didn’t stop there; I was spending a lot of time on build related issues too, so she went out of her way to create an entirely new role for a build manager (she new it would take a long time to create this new role so requested both at the same time). And this was about ten years ago before build managers were ubiquitous. Contrast that to a team I was on where a hero programmer who put in 12-16 hour days was promoted to manager. Needless to say the environment was a total nightmare. He couldn’t figure out why people actually left work to go home. He actually told me once that even if I didn’t have work to do, I should stick around late at night to display my dedication to the rest of the team so they’d feel more compelled to stay late themselves. Forget staying late, I left the team. 🙂
I understand where you are coming from. Once you are spoiled by a good manager, bad managers really look even worse than they are. However, asking you to stay later just to show dedication is one thing I will never understand.
Comments are closed.