Pages

Monday, October 29, 2007

Dysfunctional requirements and IT ideology


From my flickr
There are 3 types of requirements : functional, non-functional and dysfunctional (the latter are usually not documented).

The dysfunctional ones represent the distance between the proposed system and the organizational structure.

IM/IT systems must align against the business organization (people and processes) to work well. Management must decide who must budge before a new system is put in place in either camp. When deciding whether to build or buy a system, this effect must be considered. Neglecting to factor-in the cost of the organizational change can cancel the perceived economic benefit of using commercial off-the-shelf software (COTS).

Organizations usually have to change to fit the COTS assumptions, especially for large enterprises with large systems such as enterprise resource planning (ERP). This can also apply to core business systems, since they differentiate the company from others. Think of mobile phone billing for example.

If COTS is chosen (for ERP for example), it is likely that the finance and personnel departments will have to change their processes significantly. There are many case studies that show that CEO support is required for such migrations to succeed, since it becomes a governance issue. It has become the common wisdom now.

Replacing custom-built software with COTS is a form of devolution. Size matters in this kind of decision because the “cost of re-organizing versus the benefit of saving on development costs” equation must be juggled. The bigger the org the more the cost of adopting COTS since the org usually has to change. The converse holds as well.

So where does the argument that COTS is preferable come from? Why does upper management often fear custom work? Why do they see it a a dependence on their technical staff, and why should this be preferable to a dependence on vendors? One possibility is that the pattern emerged from the computer hardware world? COTS for hardware usually makes sense for non-manufacturing sectors since the costs of custom hardware are prohibitive and it is generally ridiculous to suggest that one should build computers from scratch. Again, there are notable exceptions, Google apparently configures boxes quite heavily to suit their needs.

The COTS for software approach is not so obvious with the advent of open source and the devolution from mainframes towards lower cost systems, the proliferation of high-level development tools and the commoditization of the software stack. For example networking and OS are now mostly standardized, and much of the higher level layers are becoming standard - Mail/Web, database and application servers. Even development environment possibilities. The database-to-presentation layers are now the target of service-oriented architecture (SOA) driven commoditization, but the vendors are all looking for their own lock-in (i.e. avoiding commoditization) while claiming interoperability to get the sale.

For example vendor A’s ESB makes integration with vendor B’s apps much more difficult than with vendor A’s. Interfaces are easy to import but hard to export. Such pitfalls can be avoided by a vendor diversification strategy across the application layer that is coupled with good cost control and use of custom solutions where appropriate. Thankfully ESB’s are not a prerequisite for SOA, but this is not generally well understood.

So IT planning strategies need to revisit the seemingly popular notion that custom is always a liability. I think that if cost control is the business objective, then it should be decoupled from the “customization vs. COTS” argument. There is no direct mapping. The organizational size attribute has to be factored in, amongst other things.

Another ideology-prone minefield is standardization. Standards should really be orthogonal to this discussion, but are related in strange ways. Standards are sometimes perceived as a factoring exercise to reduce the proliferation of types of solutions and products in an IT department. This can work very well with hardware. Where standards are problematic is when they prescribe major system software COTS without taking into account organization structure (again). Why prescribe COTS through standards? Well COTS is good, right? And less COTS variety should be even better. Hummm.

Vendors claim and often try to provide the flexibility in their products to allow (some) business process mapping and legacy integration but usually at the cost of configuration complexity or alternately through costly professional service customization.

The benefits of these approaches over custom development need to be weighed carefully; there are plenty of case studies that show success with either approach and the cost-benefit is not at all straightforward, especially when you consider upgrade cycles and other dependencies.

If product/portfolio standards are adopted by an organization, then it must also adopt compatible organizational and operational standards, since software is really an extension of organizational processes and COTS will force an organization into a process relationship with the vendor. It is a form of outsourcing.

This can ultimately lead to having the hired-guns, the management change consultants and the vendor IT/IM professional services too tightly coupled, resulting in them having a profound influence on organization structure and operations. Are organizations prepared for this kind of loss of control in exchange for perceived but generally illusory cost benefits?

I think that one of the sub texts of the discussion is at root ideological - outsource development versus do it in-house. Yes, IT departments do have much influence and are complex things to manage, and require expertise, but control cannot always be gained through outsourcing or by buying COTS, unless you are very small or have very simple needs. Horror stories abound. Control and accountability concerns need to be well understood and documented and the business objective must be stated explicitly and decoupled from the implementation. There are no silver bullets, only smart management.

No comments: