Why New Systems Fail: an interview
My friend Phil Simon published a book in 2008 called Why New Systems Fail. The book explores the many reasons why projects, mainly IT projects, fail and what a wise person can do to prevent and recover from these situations. It’s a hard nosed, low to the ground book, which tend to be my favorites. The book did surprisingly well in its first printing, so the book was reissued recently in an updated and revised edition.
Given failure is one of my favorite topics, I interviewed him about all things failure and his take on why some projects work well, but most don’t.
CONTEST: Phil will send a copy of the book to the person who leaves the best comment or question below.
SB: I’m a fan of failure and its value in teaching us things. But to label a book “Why new systems fail” might seem cynical or depressing to some. Why did you choose such a provocative title?
PS: Some have called me (and others who write routinely about failure) cynical. If you look at the statistics on IT project failures, though, you’ll see that more than three in five fail. So, is that really being pessimistic or realistic? I tried to write the book in a way to maximize success but, to me, it’s irresponsible and misleading to pretend that IT projects tend to go well.
More than just whining about problems, I pepper solutions to implementation and project challenges throughout the book. It’s about avoiding failure, not simply stating that things can go awry.
Consider the analogy of going to the doctor’s office. How can you expect to be cured if you don’t talk about what’s ailing you? How can a doctor prescribe treatment?
There are big differences in culture between projects for profit, and IT or infrastructure projects inside companies. The latter are often more bureaucratic and political. Do you think there is a difference? Are IT projects harder or easier?
I completely agree with you. In short, I believe that generally there is a difference between the two (although there are exceptions).
“For profit” software development projects tend to differ substantially from “implementation” projects. Let’s define the latter as those that involve purchasing a software vendor’s prepackaged product, configuring it, testing it, and implementing it. Those systems never face external customers. They’ll be used exclusively by internal folks in accounting, HR, payroll, etc.
Developing a product to be released to the market can unify a company, particularly if it’s a small outfit. If the product’s people are incentivized (through bonuses, stock options, or recognition), then the development can go (relatively) smoothly. People want to make the product a reality. There can be an “all hands on deck” approach because that product is part and parcel of the company’s identify. Employees want to show off its bells and whistles. Human obstacles to progress are more quickly identified and removed if necessary.
On the other hand, implementations rarely go smoothly for all sorts of reasons covered in the book. At a high level, different people and departments often have vastly different agendas, up to and including killing the implementation itself. I’ve seen it many times. Often, this lack of a singular organizational focus ultimately causes IT projects to fail. Some people don’t want the new system; they like the old one just fine.
As to which is harder, it’s hard for me to unilaterally say. I have worked on relatively simple implementations that have gone fairly smoothly, even though the system was quite complicated. People knew that their jobs would change and they just dealt with that.
It’s one thing for something to fail dramatically (unmitigated disaster), like say the Challenger shuttle disaster, where it’s so bad everyone is forced to confront the causes. But in IT it’s often easy to hide, bury or whitewash projects so they don’t seem as bad as they are. Do you think the ability to hide failure contributes to the frequency of failure? What can a manager do to minimize this?
Absolutely. For obvious reasons, organizations and key people aligned with the success of a project have strong incentives to under-report failures. For example, I have seen people call activating a reporting dashboard “a success” even though it contained only one report and no one ever used it. I have also seen people fudge numbers to make overages less obvious.
As for what managers can do, it’s a very tough question because of all of the constraints. What do you do when the CIO essentially tells you that you have a week to do something that should take a month? What do you do when you recognize that a key person isn’t performing yet have no direct authority over him/her? What if your boss won’t publicly cop to it, but he’s told you privately that he wants the project killed or postponed?
These are all people issues that make up the much of the book.
Why New Systems Fail reads like a handbook and checklist for common mistakes to avoid on IT projects. But in many cases the being aware of problems doesn’t mean you have enough power to prevent them. How do you compare political and organizational problems to project management problems in terms of how often they’re the root cause?
As you know, awareness of an issue and the ability to fix it are two very different things. It’s hard to separate PM issues from cultural/political/organizational ones. For instance, it’s possible to have a great PM and team running up against constant institutional issues that prevent the project from being successful. It’s also possible to have a great culture conducive to success and a poorly planned and executed project within that culture.
I have found in my years as a technology consultant that both of these two extremes are unlikely to be found. Typically, organizations with oodles of politics, an aversion to change, difficult end users, and cultural problems tend to run projects poorly. That includes appropriate staffing levels, comprehensive requirements’ gathering, sufficient data validation and testing, acceptance of new technologies, and all of the things that collectively cause projects to fail.
I have said before that Jack Welch himself could not have rescued some of the unmitigated disasters in the book. Companies with challenging cultures have a tough time managing projects and implementing software on time. Is this a PM issue a cultural one? It’s a chicken-and-egg question.
If you like Phil’s line of thinking, which I do, the book is available from Amazon (also kindle edition) but Phil blogs regularly as well and tweets here – check it out.
CONTEST: Phil will send a copy of the book to the person who leaves the best comment (in his opinion :) or question below.
I’m going to offer this as a reflection based on other impressions from reading about failure. It seems that the failure of certain new systems is a Good Thing(TM), that is, not all systems *should* succeed. The world might be a worse place, had they all succeeded. Certainly, we have our own perception of why any of our projects ought to succeed. What level of judgment should we recognize from the wider world (i.e. political judgment) in shutting down, or maiming, some of these projects? Let’s avoid SkyNet ;)
Are there any implications in your research for projects in the non-profit and community benefit spheres? (For instance, for a group starting a new community health clinic for the homeless?)
Going right back to the idea:
What are the most important criteria you would use to judge the viability of a concept system or prototype?
If only new systems could fail, that would be nice! But old systems fail too ! And people are still using them everyday ! Unproductive systems that make the work of people less valuable than work they could have done with a paper and a pen. The true ROI of systems is rarely computed thoroughly and on the long-term.
Fine post Scott!
I’m going for the book here, just to let you know. Might even turn this into a blog post of my own, I’ve got to settle a score on IT failure…
I think there’s one single solution for IT failure, but first, a few thoughts:
I’m (completely) puzzled by the “
@JohnO – I agree that much can be learned from failure; that’s why I wrote the book.
@Jonathan Sullivan – I didn’t focus on non-profits; I have worked in government and non-profits. By and large, they exhibit many of the same characteristics of their for-profit counterparts.
@Jim – Wow…I could write a book about that question. I’d say that business case and organizational culture are huge factors. People issues are much more important than “technology” ones in my view.
@Jerome – Very true. Particularly at organizations whose management has laid off too many people and no one is cross-trained, many systems are on the verge of failing should a key person depart. There are many anecdotes in the book about that.
@Martijn – Great comments. By “for profit” software, I was trying to distinguish between organizations that implement back office systems (read: ERP, CRM, etc) vs. those apps and systems built specifically to generate revenue.
What about the culture of a company that all the things are derived from their top management? So every changes are ‘must’ and if people don’t adjust are make it successful then perhaps their career will be in danger.
Would this kind of company has more success rate than others? If yes, would it be good for all organizations to do the same thing, at least for their internal IT projects?
As a computer engineering student I am constantly told ways and tasks in order to manage a team. Even that, software does not seem getting better over time.
What do you think we can expect with software engineering as a branch in the medium term in order to improve this situation?
(I write again not for the free-copy but to have a change to ask someone from the USA; I’m sure things must not be the same than in here Spain).
We all know that reasons of failure are mostly not technical. But do you think this makes the software engineering and IT any different from this like civil engineering? Is there _really_ any difference with other professions of intangible products such as advertisement and design?
I will definitely order your book (I read Scott’s on project management five years ago, and it had a great impact on me !) and having worked a long time in IT (over 30 years, and associated with over 30 companies, both as an external consultant and internal employee) I have also been greatly interested in the topics of IT project success and failure. More recently I have more the feeling that certain company environments are more conducive to IT success than others. There must be reasons for differences in IT efficiencies. So I have been trying to put together possible factors, signs, patterns, traits that may characterize the differences between environments with high performance IT and those that seem paralyzed and doomed to failure. I will not flood you here with my list of factors that I think play a role here – you may find these buried in my texts under http://global-plm.com but perhaps here just a few points (I guess some maybe already in your book !):
1. cultural-diversity: many companies where I have worked have gone through many mergers/take-overs. Each individual corporate culture remains alive and clashes / missunderstandings (on all levels) are common. Even the introduction of english as common language can have adverse side effects.
2. competent decision making: who makes the critical project decisions ? Even with deployment of something relatively standard like SAP ERP – a few wrong decisions – a few wrong settings in customizing – and you will be really screwed up. Essential is having the right person(s) making th decision – those who understands the technology (the software) and the business. Having the right person is not a question of luck. This is a corporate cultural thing. The same goes for realistic project estimation, scheduling, resource planing. Some corporations seem good at selecting – others not.
3. ratio of internal/external consultants: particularly if the externals are relatively inexperienced and have not worked long in the particular business area. In the worst case this can lead to a situation what may be called in germany as “Jugendforscht” – a bunch of bright youngsters eagerly discovering new solutions to things already understood and solved many years ago.
4. application complexity: too many clever consultants, but the end-users do not have Phds – essential is an understanding of “less is more …” or “Radically Simple IT”.
Hence increasing IT efficiency and improving the chances of success is a people and corporate culture thing – and in my opinion very little to do with technology. Sure some luck is also involved – ensuring the the right mix of people come together in the project – but the risks can also here be controlled.
P.S. if you have never read it, I would strongly recommend my all time favorite in this respect, Robert N. Britcher’s The Limits of Software: People, Projects, and Perspectives.
Maybe a bit dated with respect to today’s IT environment – but an itellectual delight. Britcher uses as example his experiences from the massive FAA Advanced Automation System (AAS) project, an attempt at an improved air traffic control system that ran into trouble in the early 90s after an estimated 6.5 US$ billion was spent. Quoting from Britcher: ” … the greatest debacle in the history of organized work… we learned nothing from it …”
Two things. First, regarding doing things better, that’s pretty much the last chapter of the book. For now, though, let’s just say that listening to consultants, realizing that your organization is probably not appreciably different than others, and reigning in problematic folks are good starting points.
Second, with regard to other fields, I really can’t intelligently comment. Take civil engineering. I’d defer to experts or engineers who (after they’ve read the book) can tell me if I’ve accurately described their worlds.
As Clint Eastwood said in Magnum Force, “A man’s gotta know his limitations.”
This is a timely topic for me. I was part of my organization’s implementation of a CRM system (SME) and just kicked of a project to implement an internal help desk/internal support system (PM). Our primary customers will be in the IT department.
I don’t think it is wise to begin a project focused on failure, but if no time is spent analyzing the ways in which the project could fail, it is very likely that a PM can be caught off guard. My struggle is to get the project sponsors and other team members to recognize the potential for failure. I’m not concerned about being able to successful implement the system, but the perspective of the organization seems to be that the chance of failure ends two weeks after implementation.
A couple of questions that I’ve been pondering:
How do I get people to examine the potential of failure at the beginning of a project without appearing cynical? We often document “measures of success” perhaps we should also begin including “measures of failure.”
In an organization that moves quickly from one project to the next, how can I slow the project down enough to honestly measure success?
The first question that came to my mind is what definition of failure do you use? IT failure appears to be most commonly judged on ability to meet schedule, budget, and scope (requirements), but I believe that real success of I.T. systems and projects should be judged on factors like delivering value to customers/end users and positive ROI.
Especially when initial schedules & budgets are completely unrealistic, how is it reasonable to claim failure when they are not met?
I have two questions (are they covered in your book?):
If we assume modern IT solutions only has a short life span compared to, well, construction building, doesn’t it imply we are going to have a very high failure rate?
Isn’t failure the key to maturing the industry? (How many millennia has it not taken humans to develop our engineering skills?) Thus the question: Can we compare our failure rate to other industries?
Does the exponential demands attract special kind of people that can cope with the extreme technical demands? My personal experience is that some are genius in their field, but lack social or solution oriented skills. Can this contribute to the failure rate? (hiding failure / trust issues / thinking of the bigger picture) Or is this scenario also found in similar industries?
I hope you now see I have no idea what I am talking about! Enlighten me through your book *hint*hint*! I am hungry for knowledge, feed me! :)
Btw, great interview and great blog!
I see projects fail all the time, mostly because of the grandiose plans and requirements set for them ahead of time.
Is a key to avoiding failure just to accept that it will happen? i.e. shoot for 80-90% of the functionality you need, instead of going after everything?
IMHO, to avoid failures you need a bunch of hardcore cynics in your team who are unforgiving in the way they scrutinize the system. We had this guy who used to flood the bugtracker database & constantly question the designs. People didn’t like him much. Although we all loved him secretly !!
Good question. It may be the elephant in the room but you’re smart enough to realize that there’s a significant chance that, despite best intentions, these projects don’t always go as planned. I’d say, “Get out in front of it.”
Your second question is: In an organization that moves quickly from one project to the next, how can I slow the project down enough to honestly measure success?
– Another good question. CRM projects are major endeavors. Perhaps the cost and importance of this project (short-term and long-term) make this “not just another IT project”? As such, doesn’t it behoove the organization to not do a “fly by night” job on it? It sounds like more time is warranted here.
I define multiple types of failure in the book, from “mild” to unmitigated disaster. I agree that value and ROI are important but many organizations don’t undertake those analyses.
As for your second question, you’re right. Start out with a poorly-defined plan and time line and you’re destined to fail. It’s hard to call that a success though, right?
I wouldn’t assume that assume modern IT solutions have a short life span. I’ve seen systems in place for 25 years when that was never the intent.
I really hope that failure isn’t the key to maintaining the industry. Part of the reason that many systems and apps are never retired is the low success rate on new endeavors. Add to that many of the factors and problems that you’ve seen (and I’ve discussed in the book), and it’s no wonder that many organizations attempt to prolong the inevitable.
As for your second query, you’re right. Poor communication among strategy folks, functional consultants and end users, and technical folks is a huge issue covered in the book. It absolutely drives failure.
Yes. One of the tips in the back of the book is to drop non-essential functionality if the risks aren’t worth the rewards. I couldn’t agree more.
It’s funny. Many of my clients have at times had a love-hate relationship with me. I was like the application or system that could talk. They didn’t always like the answer but they knew that I’d at least be honest with them.
I think most IT organizations are very technology-centered, and (echoing a question in another comment here) often don’t develop projects around a deep understanding of what the using community is trying to accomplish, and what is needed to realize that accomplishment. This leads to systems that are difficult to use, don’t really provide the answers that are needed, and that die as a result. This isn’t entirely the fault of the IT organization; many companies have built an impenetrable membrane between IT and the rest of the organization so that IT analysts never get an opportunity to work in and understand other parts of the company.
As those of us who read this type of literature understand, there are many contributing factors that lead to project failure. Many of the questions have focused on several of those factors and those are very important and valid. Thus, they make a great read. My question though is centered on the more intangible factor which may be the bigger elephant in the room. How much of an organization’s culture determines the level of success of any given series of projects? And, if you take that one giant step further, how much of culture in a state, region, country determines the overall success rate based on the cultural expectations/education/attitudes? So, even if you have knowledgeable people in place willing to implement various steps to achieve success, how often does the culture prevent the success when otherwise it may have been achieved?
At the risk of rambling, let me put it this way. In my experience, culture eats strategy for lunch!
Nobody likes change, but everyone wants improvement. I implemented software systems for 10 years as a consultant, what frustration and education. It’s not so much the system itself, it’s whether or not the users are willing to buy into it.
“At the risk of rambling, let me put it this way. In my experience, culture eats strategy for lunch!”, Phil, I think that about sums it up.
Issues can be come from several corners in an IT project. There are technology issues, people/culture issues, external dependencies.
1. Issues from external factors can be “owned” by everyone and we can plan around it.
2. Technical issues, even the thorniest ones have solutions and don’t generally derail projects.
3. The culture and politics in the organization are the major factors in the success/failure of an IT project. Some comments on this topic:
a. Does the person or “core team” that has the ownership for the project also have the authority to take decisions re: the project? Are all affected parties in agreement with this person/core team owning the project? In many organizations, this is skewed to various degrees. In the worst case, each affected party nominates a “leader” and there follows a long drawn out proxy war with the project in the middle (usually as a sandbag) :-).
b. Is the project going to see tangible deliverables in frequent intervals throughout the project and is there enough effective change control in place so everyone understands the cost of decisions? Funny, but every project I have seen succeed in various degrees also has conversations around tangible deliverables (instead of percentages on a status report) at various points in the project from early on. There is also usually a hard-ass project manager in the middle who can talk about the impact of each “decision” on the project.
c. How pragmatic the culture of the organization is. You have a better chance of success in a hard-nosed organization that’s driven by numbers. Its just the nature of the IT beast.
d. How well do various stakeholders understand the nature of risk and the need for contingency? If there are no open conversations about this, your planning is brittle and has little chance of succeeding.
I could go on, but after the first 5 odd years of my career, I started realizing that its the people factors that end up determining success, though technology remains my first love :-).
IMHO, the solution is all in the “ethics” of the way you run projects. The first status report you fudge is the slippery slope down to failure and the first artefact (issues log, anyone?), you decide not to prepare because , is the beginning of the end. Running and managing projects is an engineering discipline, the more this starts to veer towards “marketing the project and the team”, the more you know you are in trouble.
Having worked both in IT and run my own IT company I can safely say that it is next to impossible to be unbiased toward either the IT or the management side of the project.
If you lean toward IT, you will feel the same pain the rest of your team is feeling, namely that “The Man” is stifling your output, trying to make you work faster, having you omit stuff they don’t consider “important” (“the users will want SSH connectivity. I know I do!) and generally not having a clue what they are doing. You feel not being taken seriously and you start to lose motivation rather quickly. Sooner or later, you just procrastinate and push the blame up the ladder.
On the management side, it’s pretty much the same thing. You have all these great ideas and strategies and plans and the programmers don’t even greet you when you enter the room. You know they’re making jokes, sometimes nerdy enough they could tell them to your face (“I’m waiting for the delivery of the flux capacitor. Can’t do anything before that arrives, sorry”). Nobody takes you seriously, and you don’t know how to stop them from procrastinating. Projects run over their deadline and budget frequently and you push the blame down the ladder.
Both sides don’t look like a lot of fun. One possibility would be to hire dumb programmers. They could just do their job without question and won’t give you a hard time. Except they can’t code worth a cr*p either and have no concept of how the final product is going to work.
Another way would be for one of the guys in the IT team to become the designated team leader. Unfortunately, it either works poorly because his loyalties lie with his crew instead of management or he becomes so good at managing he forgets how to code and sooner or later becomes an outcast, having been “corrupted by The Man”.
What all this boils down to, though, is respect. I know from experience that it is very hard to earn the respect of IT folks without being one of them, but nevertheless that is what a manager should aim for. A background in IT or considerable experience managing IT-related projects is one plus. Especially if its something that the IT staff have heard off.
If you know how to communicate their way (they do communicate, even if it doesen’t look like it from the outside) and you can have them work on something they are passionate about while still staying on schedule (Google does this with their 20% although many of their projects look pretty interesting) *and* you ask for and value their input (even on management related questions), you may have yourself a winner. (incidently, I just unconcously formulated the 3 ways of Gung Ho as laid down by Ken Blanchard in his book)
So, you need a creative, open-minded and respectful manager with experience in working with IT specialists. I don’t know of any one path of higher education that prepares one throroughly for a task like this. It’s pretty much all work experience.
I feel like over time companies like Google, Yahoo, Microsoft et al. will become very important sources of personnel that combines all these properties and has a great name (and hopefully a personality) to back them up. They may even offer short-term stints (I’m talking 2 years tops) for young managers to learn the ropes and become associated with the name, just so they are prepared for what lies ahead. I don’t know how they will earn money off of that, but then again, all of them have projects that make even less sense.
Management, too, will love these people. For one, they too know the big names behind them, and if there was ever somebody they’d thought of following in the web space, it’d probably be one of the big ones. Thus, even young people will gain a lot of advance cred and have a considerably easier time cutting outlandish demands down to size (“Well, at Google, we used to …”). They also have a proven track record with their company, so there is less incentive to micromanage somebody like that.
All in all, everybody wins.
@Krishna and @Berthold – Great comments. I find myself in violent agreement with both of you.
Great minds, right? :)
@Phil: Glad we think alike, although apart from this being a people issue, I feel Krishna and I draw quite different conclusions. I decidedly do not think that project management in IT projects should be delegated to the IT crew. A pure-bred project manager with the right kind of attitude (open and humble), communication skills and some cred and experience in the IT business is the better choice in my opinion. He has an easier time earning the trust of his management superiors and a lower tolerance for feature creep.
@Phil – “violently agree” I like the image that brings up :-).
@Berthold – hope I haven’t given the impression that I am advocating the “complete self-management” mantra going around the agile circles. That just doesn’t scale and even agile practitioners see the need for full time management for any long running project over more than 2-3 sprints. The discussion around stakeholders, core team and project management is very much with a more traditional structure in place with stakeholders including business, the funding source and senior management on both IT and business ends. The project manager alas, remains the single throat to choke :-).
On the other hand, I see no reason why the technical teams and management cannot be aligned towards the success of the project. The successful projects I have seen bridge the gap with respect and an understanding that everyone is shooting for the same goal. Very tough, but possible…
Ok, we are on the same page after all. I don’t think any project can be considered a success unless the entire team – management, marketing, IT and service – see it as a team effort. Sure, the pyramids got built that way, but from an economical standpoint, those things are terrible failures. Just ask the tourism board of Egypt. :D
Aligning the team requires respect, first and foremost. It’s also important not to confuse respect and authority. The former has to be earned (through personality and competence), the latter is more likely a matter of hierarchy than competence, and IT staff are notorious for ignoring hierarchical structures. Just because you can get people to call you boss doesen’t mean you have their respect.