Bad Programmer, Bad Process Or Bad Education

A few articles in the past week prompted some thinking about the industry of software development. In this case, when I say software development, I am talking about developing websites, web applications, commercial software, enterprise software and almost anything else that requires someone who can code. When comparing the industry to other scientific industries, software development is still young, but more importantly software development is one of the few industries that anyone can get into.

First, let’s look at the articles that prompted all of this. On DZone, there was a post asking us to embrace the fact that we are bad programmers. The first paragraph really summarizes the post:

How many developers think they’re good programmers? We’re not. We’re all fairly bad at what we do. We can’t remember all the methods our code needs to call, so we use autocompleting IDEs to remind us. And the languages we’ve spent years, in some cases decades, learning? We can’t even type the syntax properly, so we have compilers that check and correct it for us.

Admittedly, the post does talk about the benefits of unit testing and other good things, but the premise is interesting. Why are we still dependent upon IDE auto-completion? Why do we really need compilers and interpreters to tell us that we typed the syntax incorrectly? Why are we still talking about syntax?

There are really two parts to this problem. The first is the fact that computer aided software engineering (CASE) is mostly a failed segment of the industry. The tools were not that bad, but the problem is that the industry is really not mature enough to properly specify the type of functionality we need in an application. Part of the reason is that software development is hard. It may also be related to the fact that the industry is mostly unregulated. If you look at pharmaceuticals, they are heavily regulated. Architecture and construction are regulated or at least they have rules they must follow. Software engineering has soft guidelines at best. I am not talking about the SDLC processes like Waterfall, XP and Scrum, I am talking about things our code must or must not do. Static analysis tools like PMD or FindBugs may be part of this, but in order for the industry to really mature it feels like something is missing. Another part of this is the certification or licensing required for each of these jobs. Architects typically require an advanced degree to become an architect, and depending upon what you design, you may need certifications or licenses. Scientists at a drug development company require a minimum of an MS degree eventually, and in many cases a PhD. You cannot become an architect or a scientist without having these advanced degrees or certifications. I am not calling for certifications in software engineering, but this is one of the few industries where you can just pick up a book, start coding and get a job. This does not always mean that these developers are good developers, but just that anyone can become a developer.

The second part of this is that the software development industry is that the industry has become heavily focused on the application side of computer science. I believe this was brought on by the industry itself. When I was in college, there was already pressure from industry to ensure that people graduating with a degree in computer science could be immediately productive in a business environment. The standard curriculum will always change due to the rapid pace of change in our industry, but changing the curriculum to fit what the industry wants may not be the answer. You would hear that developers coming out of school did not know how to build an application. I would argue that the point of education is to teach students the basics of a subject and to teach students how to learn more. If we target students to be more marketable to industry when they graduate, we are losing some of the core knowledge we are supposed to give students. Some of the “leading” universities still have programs that teach a breadth of information, but some smaller or more local universities are going the “marketable graduate” route.

The other post that prompted this was from Dave Winer regarding doing more computer science.

I’d like to see young computer science students get up and present to their teachers some new computer science. Some new use of graph theory. Or an old one re-applied to a modern world. We used to do that, when I was a comp sci grad student in the 1970s. I think we got way too caught up in the commercialism.

Why is this a problem? Well, the real issue is that in order to get to the scaling specialist realm of expertise (or any specialization) or even to build one of these fancy social applications , you need to know a lot of information:

  • Basic data structures – What is the difference between a linked list and a hashmap?
  • Basic algorithms – There are several sorting algorithms and many suck. There are also different algorithms if you are memory constrained, CPU constrained or disk constrained.
  • Object Oriented Programming and other programming types – Object oriented programming is not the only type of programming. There is also procedural, functional, logic programming, and other strange paradigms. Knowing some basic information about these can really make you think about a problem differently.
  • Database systems theory – Basic knowledge of how databases work is helpful in understanding why your application is slowing down. The NoSQL movement may not be the same as RDBMS technology, but some of the core concepts still come from standard database theory.
  • Theory of Computation – There is a lot of theory behind the often mentioned Big-O notation. This helps you understand you why your algorithm is so damned slow.
  • Statistics and Probability – Much of computer science theory is based on mathematics. Statistics and probability also helped forge many concepts in machine learning and artificial intelligence.
  • Machine Learning and Artificial Intelligence – This is typically kept outside of the standard curriculum, but many applications are based on recommendations, categorizations or prediction. A basic understanding of what is possible would be enough to give you an idea of where to look next.

This is just a basic list of foundational knowledge. As you dig into the more advanced topics, you can find tons of interesting information. Many graduates of computer science do not take courses in each of these areas. I believe this focus on job readiness has forced us to skip the courses that help us create the next computing paradigms. This is why Dave Winer complains about our need to do more computer science, because less computer science is being done in schools. He talked about the companies at TechCrunch Disrupt recently:

The products were all cut from a small number of molds. Somehow they have fixated on turning the internet into a game.

What does all of this have to do with each other? The lack of tools to formalize the process could be due to our lack of foundational computer science. There is a large breadth of information that needs to go into tools that would give us standard rules of programming. The lack of education in advanced topics also hurts computer science innovation. When computer science innovation lags, there are less opportunities for “world-changing” computing technologies. Even if we do not target world-changing ideas, a greater breadth in education would give you different ideas on what you could accomplish.

It may not be that you are a bad programmer, we all make mistakes. It may not be that you are using a bad process, even though we all know that the Waterfall process has its problems. You may not have a formal education in computer science either. All this being said, it does not excuse us from learning from our mistakes, learning more about the software development process, and learning more about areas of computer science that are unknown to us.

7 thoughts on “Bad Programmer, Bad Process Or Bad Education

  1. One of the frustrations for me as a long-time tech guy who is fairly new to software development is that I realize more and more how time-consuming even small development projects can be to create and later maintain. In a business environment, with close user interaction, I’d say there are lots of opportunities for tiny “one-off” projects to spring up, and these are just the sorts of applications which can later consume a lot of time as they are likely to be built without robust testing, monitoring, and configurability in mind.

    This has led me to (try to) give a lot of thought to the essential differences in the development process which are required for one-off business application types of environments, as opposed to more product-oriented environments.

    Any tips on how to build effective safety nets around the inevitable off-the-cuff errors which can proliferate when building lots of small applications?


    1. Shannon

      The tiny “one-off” projects in larger businesses are typically a sign that something larger may be missing. As you said, the time and maintenance costs for these small projects can be very high. However, if you can look at them as a group you will start to see patterns or commonalities emerge. In every business, many people feel that their requirements are “special” or “unique” to their business or industry. There are a lot of things that are common among many industries, and if you look at the requirements as an outsider or from a group perspective it can be helpful.

      In terms of the process of development, testing and support, the process is not really different. Some of the requirements will change, the durations may change, but the overall process stays the same. The safety nets stay the same as well. Automated testing, in all levels of testing not just unit testing, can provide a great foundation for not breaking the existing functionality. Code coverage tools like Cobertura and static analysis tools like PMD can also help cover some of the gaps in code. Otherwise, programmers make mistakes in their code (and the tests for them) and sometimes requirements are unclear. Currently, there is no way around it.


  2. I am not an ardent fan of programming but some where I am compelled to write this. I live in India. There are more than 3000 engineering colleges here where Computer Science is also taught in most of them. I am a student of one such college and finished my course in computer science. After reading your post, it has made me think about what I really have learnt all these years. The answer is: not much. The intention of our parents to send us to educational institutions is to get a job. Most do not bother how or what they are missing from a normal education.
    Though being computer trained, the most dreadful topics for me are data structures and queing theory. You may laugh at this but, yes I hate to revise the long binary tree algorithms and linked list and get confused with it, and finding the time complexity.
    One may think that only parents are like this but here the teachers also measure the merit of a student by witnessing one’s success in ON Campus Recruitment of fellow students.
    I could goon criticising the education system here. But the thing I wanted to bring this to your notice is that, the future of Programmers is not bright, because from 3000 colleges even if 20 students per college pass out per year than 60,000 computer science graduated (who are supposed to be educated) would be programming for various MNC’s.
    The future lay in the hands of people in R n D.
    All The Best.


    1. Teja

      I understand what you mean. I have worked with Indian consultants and one of my employers had an office in Bangalore that I visited. I learned a lot about the Indian education system at the university level. That being said, sometimes you need to make the most of your education or even start doing projects outside of schooling. Everyone needs to get a job at some point, and I know how hard it is to make yourself marketable. Like you say though, there probably needs to be a change in some of this education to ensure we do not lose our ability to innovate in technology.


Comments are closed.