Software development has come a long way over the last few decades. We’ve gone from extremely laborious, protracted exercises to create even basic functionality (punch cards anyone?), to the drag and drop, WYSIWYG environment of today. We’ve also gone from a very small number of enthusiast programmers to literally millions of individuals writing software worldwide (there were over 1.3 million software engineers in the US alone in 2008). And we’re all looking for ways to make building software even easier.
As the software evolution has continued, common patterns have been identified and dedicated tools have emerged to abstract these patterns and enable developers to build applications more efficiently. Over time, enterprise software platforms have gained popularity, abstracting common patterns with the promise of readymade frameworks that promote rapid development, greater agility and lower support costs.
But do enterprise software platforms always achieve this objective? Are the gains of rapid development offset by other costs – hidden costs – which don’t expose themselves at the outset? Are these platforms really the panacea they’re cracked up to be?
What’s an enterprise software platform?
Enterprise software platforms attempt to solve abstracted requirements by providing a generic platform which can be applied to solve specific business problems. The examples I’ll refer to most frequently are Microsoft SharePoint and K2 workflow but the principals discussed are equally relevant to products such as Oracle Portal or Microsoft CRM.
Let’s start by understanding what K2 is based on their description:
The K2® software platform increases business efficiency and makes work simple. Its visual, familiar tools allow people with varying backgrounds and technical skill to create applications that automate business processes and streamline operations. And when something in the business changes, modifying the applications to keep up is easy.
And a little about SharePoint from the Microsoft perspective:
Microsoft Office SharePoint Server 2007 is an integrated suite of server capabilities that can help improve organizational effectiveness by providing comprehensive content management and enterprise search, accelerating shared business processes, and facilitating information-sharing across boundaries for better business insight. Additionally, this collaboration and content management server provides IT professionals and developers with the platform and tools they need for server administration, application extensibility, and interoperability.
Both of these are very vague and fraught with business buzzwords that don’t tell you an awful lot about what the product does. I ran Bullfighter over the SharePoint description just for interest sake:
Bullfighter is a bit tongue in cheek but it has a point about obfuscation and preponderance. Let me try and distil the nature of these products into a more consumer friendly fashion:
As application platforms, these products aim to streamline development by abstracting common solution behaviours and packaging them into products which may be consumed by business applications. They both provide APIs and integrate with IDEs to be utilised at design time as well as providing server based installations which are required at runtime.
One quick clarification about SharePoint as it’s a bit of a multifaceted product; I’m referring to using it as a platform on which to write custom application code as opposed to using the native teamsite and portal collaboration features. These features are fantastic straight out of the box and are in a very different category to using SharePoint as a foundation for writing code.
What’s wrong with this?
Where the problems first arise is in customer expectation, and is it any wonder after reading the quotes from the manufacturers above? Because the positioning of the products is so generic it’s easy for customers to form opinions and set expectations with little understanding of the actual effort involved. The bottom line is that many businesses experience unexpected surprises during the development and support cycles of applications built on these platforms.
To be fair, both these products and enterprise software platforms in general can be fantastic when used in the right way. The intention of this post is to highlight areas which can be problematic in order to contribute to a more pragmatic analysis of the platforms before commitments are made.
The people problem
A serious problem with creating a dependency on these platforms is that you create a dependency on people with niche skills. This severely restricts the potential candidates you can hire either directly or indirectly through an ISV. Remember also that we’re talking about server platforms so you not only need good developers, you need good infrastructure people who understand the product.
Let me demonstrate; Stack Overflow Careers allows developers to promote their CVs online and there are currently a couple of thousand people putting themselves out there. Let’s imagine we’re recruiting for a .NET developer:
Let’s compare that to a SharePoint search:
And now K2:
So for every potential candidate you could hire to develop SharePoint applications there are eight .NET developers lining up. K2 doesn’t even feature. But is this a realistic representation of people availability? Let’s reverse the process and see how many jobs are available on seek.com.au in Sydney for the same technologies:
A little less damning than the Stack Overflow results but you’re still looking at five and a half .NET jobs available for every SharePoint one. The K2 results are a bit misleading because five of them refer to the K2 search engine. The remaining result lists K2 as “beneficial”.
The relatively small representation of SharePoint and particularly K2 poses a number of problems. Firstly, the reduced number of candidates means recruitment is more protracted as good people are harder to find. This poses a direct risk to agility in terms of being able to rapidly scale resources up and down and in turn impacts negatively on those charged with the recruitment process.
Secondly, market forces tend to financially favour those with skills which are not commonly found but are in demand. In turn, these people tend to command higher rates. The last couple of times I recruited people for a SharePoint developer role they were consistently priced around 20% higher than their .NET counterparts. Great for the candidate, not great for the employer.
Finally, it doesn’t matter whether you’re engaging individuals directly or using the services of an ISV, the same risks and costs exist and ultimately they’re passed on to the customer. The same market forces drive the cost and availability of ISVs; there are far fewer competent vendors out there developing SharePoint or K2 than there are .NET and this is reflected in their pricing.
From a cost perspective, all this begins to erode the gains made in rapid development. The fact an application may be built 20% faster on one of these platforms is quickly offset (opportunity cost of earlier delivery aside), by 20% higher resource costs.
The development environment problem
Unsurprisingly, the process of building software on these platforms is a little different to your classic development model. The surprise comes when you understand just how different the development process can be.
Let’s look at the SharePoint approach compared to traditional ASP.NET development. In the .NET approach, the developer has Visual Studio on their everyday work machine and pretty much goes from there. In the SharePoint model, the developer needs to work on an actual SharePoint server. You simply cannot code against the object model any other way.
What the SharePoint development model usually entails is running a virtualisation product such as Microsoft Virtual PC and building a sever environment complete with OS, SQL Server, SharePoint server and finally Visual Studio.
The next thing you need to consider is that running a virtual machine, particularly a server running SQL Server, is very resource intensive. Usually the virtual disk of the machine is placed on an external drive to spare the host of additional HDD activity so there is a small investment required there.
The development machine is also going to need a good allocation of RAM (at least a GB), so this will often mean increasing the physical memory of the host if it’s expected to continuing performing other tasks normally expected of a desktop computer. A machine with 3GB is pretty much an entry point but it will incur a productivity tax. Of course anything beyond much more than 3GB means moving to a 64 bit operating system which adds yet another dependency to the equation.
These are not insurmountable problems but they may come as a surprise to the unprepared. There is a time and expertise dependency in building a virtual machine, a hardware dependency on having a machine capable of running it in an efficient fashion and this potentially even creates an OS dependency because of the 64 bit issue.
A final consideration, relevant to some businesses, is that the development model has a dependency on leveraging an “unmanaged” server installation which may breach corporate desktop rules.
Sidenote: these issues and many more are articulated very well in the Stack Overflow post about What are your biggest complaints about SharePoint?
The shared farm problem
These are expensive, specialised products requiring specialised expertise so it’s only natural that organisations aim to consolidate instances into shared farms. The theory is that by decreasing the infrastructure footprint of these products the business can minimise the costs of licensing, hardware and support. The same rationale is perfectly valid for products such as IIS or SQL Server and at least on the surface, it makes sense to do the same with the enterprise software platforms.
The problem with any shared farm is cohabitation. Any application on the farm is simply a tenant and it has to play nice with all the other tenants so the farm owner creates the server equivalent of strata by-laws. The by-laws define governance designed to ensure the harmony and sustainability of the environment in the best interests of all the tenants. The extent of the governance and the freedom provided within them is largely up to the model implemented by the underlying technology.
Let’s take the example of a website hosting provider offering a typical SQL Server backend and IIS front end on a shared farm. These technologies provide a robust sandboxed model which offers the customer a large degree of both security and autonomy in that in most cases, once the resources are provisioned there is a great deal of freedom in terms of what the customer can deploy to these resources without risking the integrity of other tenants in the same environment. Customers can regularly self-deploy web applications and databases in a secure, on-demand fashion which means in terms of application agility it’s a big thumbs up for the model.
The nature of these products makes the shared farm model a little trickier. It obviously varies between products but let’s take SharePoint as a fairly typical example. One of the shortcomings of this product is that it makes code releases difficult at the best of times (I’m talking about SharePoint 2007 here), let alone when you’re also trying to secure the integrity of other applications in the same environment. Deployment of SharePoint applications packaged as a Windows Solution Package (.wsp) requires permission levels across the environment and beyond the scope of just a simple application meaning the potential ramifications of granting these rights are large. Have a glance through the DevX article about Creating and Deploying SharePoint Solution Files and you’ll get a pretty good idea of how low level a SharePoint deployment is.
The problem this leaves us with is that SharePoint apps can’t just simply be deployed by the developers into a shared farm but rather must be managed by a group that has the best interests of the broader farm in mind. Some people would argue that this model is preferable in any environment, not just SharePoint, but the reality is that many organisations simply don’t implement this degree of governance. The bottom line is that this model of deployment has a level of governance you don’t necessarily find in other technology stacks, such as your classic ASP.NET and SQL Server, and that it can have an adverse affect on agility and cost.
Sidenote: SharePoint 2010 implements Sandbox Solutions which come with the promise of greater flexibility and autonomy to the developer than what we’ve seen in previous versions. However, there are still limitations which will mean the concept of managed deployments will need to persist.
The new version problem
Products like SharePoint and K2 are undoubtedly feature rich and inevitably complex beasts under the covers. They continue to evolve and incrementally improve with each new release and with a product like SharePoint, the release cycle is only a few years. Chances are that any application you build on one of these platforms is going to need to go through an upgrade process at some time.
The problem with both these platforms is that they’re simply not backward compatible, at least not without going through conversion and testing processes requiring various degrees of manual labour. K2, for example, went from heavily promoting the use of Smart Forms as a UI layer yet now recommends standard ASP.NET webforms or InfoPath. There is a K2 BlackPearl Migration Utility available but the feedback on the effectiveness of this and the amount of manual labour required indicates it’s by no means a panacea for upgrading.
In the K2 case you then have idiosyncratic rules such as “all workflows must completely execute in the same version of the workflow in which they were started” which is going to mean some very carefully crafted migrations.
Inevitably businesses take the decision to upgrade these platforms. Nobody wants to go on supporting a single technology version for perpetuity and as new releases become available, support for previous editions wane and the argument for building new applications on the latest product becomes more compelling. Ultimately, the upgrade path needs to be pursued at some point in time and the challenges described above will be faced.
Jumping to conclusions
A risk these products all pose is that they make it very easy for decision makers to jump to conclusions. Building an app with workflow? You’ll be needing K2. Need alerts? That’ll be SharePoint then. They’re reactions based on their understanding of the product as represented by the manufacturer and often communicated at a very “manager speak” level.
These reactions are understandable, to a degree, when you remember that K2 “increases business efficiency and makes work simple” while SharePoint “provides IT professionals and developers with the platform and tools they need”. Of course by now we know it’s not that simple and there are a number of other problems to contend with.
Let me draw on a workflow example to put the problem into context. Let’s say we have a requirement to seek an expense approval based on an ASL. The input to the workflow process is a price and the business logic involves assessing this against a list of potential approvers, each with their own signatory level. There will likely be the ability to view and approve outstanding tasks and email notifications will probably be used to either call people to action or advise them of a status change. This is your ubiquitous Hello World app for workflow.
Should this be a K2 workflow? Quite frankly, no, it shouldn’t. This is programming 101 and most competent developers could put the whole thing together from scratch within a day and without experiencing any of the problems described above. Sure, it won’t provide all the functionality that K2 can but then things like reporting and escalations weren’t part of this particular requirement and are often way down the feature priority list.
The point is that the way these products are positioned makes it easy to prematurely jump to conclusions. It’s easy for this to occur when the true nature of the product is not understood. It’s also easy for this to occur once a business has invested in the project and loss aversion sets in. Neither of these are very good reasons to use an enterprise software platform.
The alternative
It’s simple; custom coding.
The current state of .NET and SQL Server is extremely feature rich, as are many comparable non-Microsoft products. Just within the current generation of the technology we’ve had numerous new features which streamline the development process in a number of ways. Just off the top of my head; LINQ, implicit typing, WCF, WF, FILESTREAM and geospatial data types and the MERGE TSQL function. I’m not saying these entirely take the place of a workflow or collaboration platform but I am saying they streamline the development process and make the implementation of most requirements significantly easier than it was even just a few years ago.
In many cases, as with the one mentioned above for an ASL based approval, custom coding is a more than adequate means of fulfilling the requirement without suffering from the problems already described as systemic of the enterprise framework approach. There are downsides though; some features may take longer to build (indeed they could become quite complex) as you don’t get the same degree of native functionality. But when you can build the entire requirement with custom code in less time than it takes just to configure the development environment for K2, the alternative approach begins to look pretty good.
Edit: Judging by some of the comments I’ve received, this section has possibly been taken out of context and I want to be clear that by no means do I advocate replacing either K2 or SharePoint with custom code in a broad-brush fashion. This example is intended to illustrate that depending on the scenario, there are times where the “roll your own” approach may be more appropriate. Not every workflow requirement needs K2 nor does every collaboration requirement need SharePoint. I’ve been directly involved in projects which have used these tools to great effect and I wouldn’t do them any differently, but I’ve also seen instances here they have been detrimental for many of the reasons mentioned above. Pragmatism and an open minded, objective approach is key; there are no silver bullets.
Proceeding with an enterprise software platform
None of the above is intended to say that these platforms are evil. There is absolutely a time and a place for them but the use case is just not as broad as many people would have you believe. In an environment with sufficient scale to justify the investment, the right expertise to support through the lifecycle and possessing business problems closely aligned to the offerings of these platforms, there may be a valid case to be made.
Let’s take a pragmatic approach to choosing the technology and remain conscious of the following:
- Understand the resource needs for building on the platform. They may be harder to find and cost more than you expected.
- Plan carefully for the development environment. Ensure the correct hardware exists and that no corporate barriers prohibit working in the recommended fashion. Also ensure greater lead time is made available to ready the environment.
- Expect a higher degree of governance if deploying to a managed environment. This may have an adverse impact on agility and come with a financial overhead.
- Acknowledge that upgrading the platform as newer versions are released may impose a more significant overhead than you’re used to.
If someone goes through the analysis process and comes out the other side with a K2 or SharePoint based solution, fantastic. They’ve done the due diligence, they understand the risks and they’ve ultimately chosen the best technology for the job.
However, if conclusions are drawn prematurely and an enterprise software platform is committed to without fully understanding the ramifications, you’re going to need a lot of patience and very deep pockets. Caveat emptor!