Often the effort of managing software engineers is compared to herding cats. For those who don't know, herding cats is an idiom referring to the frustrating attempt to control or organize a class of entities which are uncontrollable or chaotic.
Why do software engineers have the reputation of being uncontrollable or chaotic? Are they unmanageable? You might think so--even doing a search on Amazon for software development management leads to Managing the Unmanageable.
Here is what I believe causes software engineers to have this reputation:
- Engineering as part of the job title
- Complexity of software
- Creative, smart employees
- Distrust of management
- Being a great engineer doesn't mean you are a great manager
Engineering as part of the job title. If there ever was a list of mislabeled jobs, Software Engineer would have to be up there. Except for software, every other engineering field has a well defined set of rules, procedures, knowledge and systems. Software engineering has none of that. There are patterns that are discussed (see Gang of Four) but no hard rules that apply. Each software project starts from zero (or should start from zero).
Complexity of software. Software has gotten so complex that there isn't even a consensus on how to measure the complexity. Managers always want to know when the software will be complete but how can a completion date be determined if we can determine the complexity of what is being built? Software engineers are bad at guessing how long something will take. This is not a mark against any engineer but a testament to the fact building software is not really engineering and it is a complex, nasty business.
Creative, smart employees. Every single company is trying to hire the best and brightest software engineers. This means that if a company is succeeding in hiring, then the software engineers are a group of smart, creative, highly-motivated set of individuals. Creative types, even software engineers, don't always do their best work in a 9-5 manner. Smart creative people don't want to just be told what to do, they want to know the why. Sharing of information and context is vital with software engineers, yet it is not a common practice.
Distrust of management. When it comes down to it, software engineers want to write software. That usually means having a long block of uninterrupted time to code. Even a small distraction has a huge impact--due to the complexity of software (see above) it will take a distracted engineer at least thirty minutes to get back to the point of distraction. To most engineers, managers just create meetings and these meetings are horrible distractions and most likely wastes of time. The worst type of manager will be one that doesn't understand the complexity of software and will not understand why a project is behind "schedule" and will often ask for more updates which will add more distractions causing the project to delay even more.
Being a great engineer doesn't mean you are a great manager. Software managers tend to come from top software engineers. There is just a little overlap in the skills necessary to be a top engineer and a top manager. A great software engineer gets a of lot of things done individually while a great software engineering manager gets the team to do a lot of great things. It is hard to transition from having a high individual output to delegating. Having a manager that can't delegate, that can't see the big picture and doesn't provide a clear vision to the team is demoralizing, demotivating and down right dangerous to a team.