A few weeks ago I talked about what a software engineer may do for design tasks. Today, we are going to take a step in a slightly different direction and look at how projects are planned and what a software engineer may do during planning. If you are just starting out as a software engineer, you will not be heavily involved in the planning except when the lead engineer asks for an estimate for a specific task. This post is really about what that lead engineer will probably be doing when people say “planning”.
In a perfect world, all of the requirements of a system would be gathered and there would be zero unknowns or missed items. Unfortunately, this never happens no matter how much time is spent gathering requirements. Because of this, part of planning for the software engineer is reviewing the requirements for obvious misses as well as adding any technical requirements. Technical requirements are those functional pieces of the application that the business team is not always aware of. For example, there could be standard auditing requirements that mandate all database access must be tracked. Typically, this type of requirement is not a simple logging effort, but it requires a significant amount of development to track the query executed, parameters of the query, copies of the original rows in an update, copies of the new rows for the update and anything else that needs to be audited. This needs to be added as a technical requirement due to the amount of time it would require to implement. There are other possibilities as well, but this gives you an idea.
Once the technical requirements have been determined, you can start the actual planning and estimation. I will use the terms interchangeably, but there is a significant difference between them. Estimation is the activity of determining a likely amount of time a task will take to complete. Estimates can change based on who does the estimate or who is going to do the work. Estimates are also very fuzzy, as there is no scientific process to determine an estimate. Planning is the activity of scheduling tasks and assigning those tasks to specific people. This also assumes that the estimates are not estimates, but actual planned durations. As you can see, planning and estimating do not really work well together, but it is the typical way projects are completed in a corporate environment.
Typically, the requirements are mapped to high-level tasks, which may be broken into smaller tasks, in order to be able to estimate the requirement. For example, in a user management system, a requirement may be “I want to be able to review and modify user information.” That is obviously a little vague from the developer point of view, so it needs to be translated into approximate tasks. This could be a simple Review User Page and an Edit User Page. These are reasonable tasks to estimate if you have had enough experience doing estimations. From my experience, a mid-level (3-5 years) software engineer will require about 5 days to complete each of these pages. This may seem like a lot of time for a simple page, but there are several pieces to a Review User Page. There is the data object that represents a user, the data access to retrieve the user from the persistent storage, the control logic like a Spring Controller or a Java Servlet to handle some simple logic, the html or jsp page to display the information and unit tests for as much as you can test. For a majority of this work, there is additional configuration that needs to be completed, maybe some database tables to be created and the refactoring that will always be necessary. When you look at the Edit User Page, you have the same concerns in addition to saving the user information and any security concerns about who can modify the data. So, the 5 days for each page is just an average I have seen over many years of these types of applications.
Hopefully, the lead engineer has a decent number of years of experience as that helps determine appropriate estimates. The lead engineer may also ask each software engineer on the team to estimate certain tasks because they will be doing the tasks. The lead engineer will take the lower level estimates and summarize to the appropriate level so that planning for the project can be completed. In some environments, there is much discussion over the estimates because they are typically considered too lengthy or the project needs to be done by a certain date. Disappointingly, some projects will have the estimates changed in order to comply with this preplanned date and the developers will need to live with this. Thankfully, this is not entirely normal, but it does happen far too often. Because good estimation is extremely difficult and preplanned dates do appear, agile processes can be used to manage the development process in a different and more developer friendly way.
Agile Planning and Estimation
In an agile setting, planning is a much different concept where you decide which tasks will be completed during the sprint or iteration. Some agile teams may still estimate a task in time units like hours or days, but other teams use fictitious units like points or even doughnuts. Agile planning, specifically Scrum, uses velocity and “yesterday’s weather” to determine how much work can be completed. The Scrum Alliance defines velocity as:
In Scrum, velocity is how much product backlog effort a team can handle in one sprint.
Yesterday’s weather is the idea that the team should be able to complete the same amount of work in the next iteration as was completed in the previous iteration assuming the team and amount of time worked stays the same. So, if you have a team of 3 people that all worked 40 hours per week in a 2 week iteration, you would have 240 hours worked total. If the team completed 24 points in the previous iteration, then you can plan for another 24 points in the next iteration. This type of planning and estimation allows you to adapt to the current conditions and productivity. So, if you are ahead of the planned schedule, you could add a feature and still be completed on time. If you are behind because some features took longer than expected, you may be able to remove a feature that is not really required for initial release.
Go Forth And Plan
In the agile development world, there are several different ways to complete the planning process. The one described above is very common and is just meant to give you an idea of what agile planning is like. I am not an agile development guru or zealot, and I am not a waterfall bigot or evangelist. Some processes will work better in some environments, and it is also dependent upon the team itself. If the entire team is not familiar with agile processes, it would not be recommended to start using agile planning. If waterfall development has always worked for your team, then please continue to be successful with it. Do not change things unless you can improve it. When planning and estimating, the most important thing is history. Your experience is important and agile planning looks at the most recent experience of the entire team. So, the next time you are planning a project, think about your last project and how long some of the tasks took. It will probably help you determine better estimates and have more confidence in your estimates.