This paper was first published in the Journal of the Samara State Aerospace University in June 2006, printed to support a conference held in celebration of its 30th anniversary. Andrew’s paper was published by the university in support of the conference.
Introduction to Basic SOFTWARE CHANGE and CONFIGURATION MANAGEMENT
Change is a constant factor in modern business and essential for it to adapt, grow, compete and thrive in the modern world. If a business does not change but instead stagnates then, unless its sector is similarly stagnant and devoid of competition, it is likely that the business will be overtaken by its competitors and perhaps ultimately even taken over by them as well. But change can be costly and, if not managed correctly, can potentially take the business in the wrong direction; in so doing it may be left in a worse position than if it had not attempted to change at all.
This paper, written by an experienced independent UK-based IT Change & Configuration Management (CM) Consultant, provides a very brief overview of the Management of Software Configuration and Change in the business environment. It looks at possible stimuli for change; how a potential change can be assessed for acceptability, impact and cost; how more detailed changes can also be managed; possible process quality gates and check-points, what metrics can be gathered as the change progresses though its life-cycle; and it touches upon both methodologies and the commercial toolsets available to assist in both the change and configuration management processes. As the paper progresses, it outlines a generic two-tier change management process, typical of that used in many organisations today. It also discusses exceptions to that process, both in the interests of speed and cost-saving, but without loss of quality.
While this paper will focus substantially on the management of change and configuration within the IT industry, the principles and practices that it recommends are equally applicable to many other business sectors. Due to space constraints, the closely related subjects of Release Management & Deployment are not discussed in this paper.
Source of Change
A request for change can be generated by either internal or external stimuli. Examples of external stimuli include competitors’ products, enhancement requests from customers and legislative pressure. Possible internal stimuli include enhancement requests resulting from market research, audit recommendations, cost-cutting and the correction of defects in an existing product line. In practice of course, the potential stimuli are limitless. When a Request for Change (hereinafter abbreviated to RFC) is received it should be recorded in a manner that is consistent with that used for other changes, and should include sufficient information to enable at least a broad decision to be made about the suitability of the change to the business. Key information to record is likely to include:
- Title / brief description of proposed change (50-100 characters)
- Detailed description of proposed change (100+ characters or attached document(s))
- Perceived severity and priority of the change
- Identity and contact information of Originator / Requester
- Business area(s) affected by change
- Implementation window during which change required
- Estimated cost of making change, and possible effect of not making change
- Any other information pertinent to either the business or the change
Based on an initial assessment of the business areas affected by the change, the RFC should now be forwarded to nominated representatives of those areas for consideration. It should include a realistic deadline for response, together with a clear statement of what will happen if none is received. The Impact Assessor should consider all implications of the change likely to be relevant to his/her business area, and record those details on the RFC. There are several solution vendors able to offer tools specifically targeted at Impact Assessment, perhaps through the use of a Configuration Management Database (CMDB) if required.
When either the response deadline is reached, or all responses have been received, the initial impact assessment is complete. The results should be collated and summarised, typically by a business analyst, and the RFC can be considered for approval.
Change Control/Approval Board
It is the responsibility of the Change Control Board (CCB) / Change Approval Board (CAB) – the terms are interchangeable –to consider every RFC put before it (although see also the later section on Process Exceptions). The CCB should look at the request itself; the results of all impact assessments carried out; the accuracy of the cost estimates; the probable effect of not making the change – in simple terms it should consider all of the information at its disposal – and on the basis of that information make a decision as to whether the RFC should be approved or rejected. For this reason, the CCB will frequently include a member with budgetary authority.
The CCB must make a decision on the basis of the information in front of it, although it can request more information if necessary. Its request for additional information can take any form including inviting custodians of the information to attend the CCB to answer questions verbally, or simply returning the RFC for further impact analysis. Importantly, it is not within the remit of the CCB to modify the RFC in order to approve it, merely to consider that which is presented to it.
The CCB’s goal should be to place every RFC it considers into one of several possible categories:
- Rejected (The requested change will not happen, either not at all or at least not without modification. The reasons for the rejection should be recorded with the RFC to give the originator a chance to modify and resubmit it.)
- More Information Required (and the nature of that information)
- Approved (and any caveats surrounding that approval)
The CCB will meet periodically to consider RFCs, either physically or virtually (meeting only in ‘cyberspace’) depending upon the geographic distribution of the organisation and commitments of board members. In any event a Change Manager should be appointed to act as facilitator and secretary to the CCB. Although the Change Manager will not necessarily have any authority himself, it should be the Change Manager’s responsibility to schedule CCBs.
Making The Change
Once the RFC has been approved, the work can be performed. Typically the decision as to precisely how that work will be performed will be left to local managers with sufficient autonomy to act as necessary to achieve the stated aims of the RFC, within any budgetary constraints imposed upon them by the CCB.
In the case of software changes in particular, it is unlikely that the full technical detail of the RFC will be understood, or even be appropriate for consideration, by the CCB. Having obtained high-level approval for the change, the local manager may thus wish to break the RFC down into a number of smaller and more manageable units, typically (though not necessarily) referred to as a Software Change Request (SCR). These SCRs will then be used to detail the change at a source-file level, and to provide and control the authority to access and modify those source-files.
Quality Gates & Audit Trail
By ‘relating’ (i.e. logically linking) all affected source-files to a SCR, and all SCRs to a RFC, a clear audit trail of both work performed and the authority given to perform it is generated and recorded as work progresses. If the relationship between executables and the source-files from which they are generated is also recorded, then the audit trail becomes both clearer and more valuable still.
Version & Baseline Management
By recording every revision of every artefact that is stored within the system and touched by the process, a clear record of the version history and evolution is established. This is basic version management.
By taking a baseline (a snapshot of all artefacts relevant to the development taken at a point in time, sometimes also referred to as labelling) the concept a fixed and reproducible set is introduced. The versions and inter-relationships of every member of that ‘set’ at the time the snapshot was taken is also clear, and permanently recorded. If the progression between these baselines is then linked to the software change requests (SCRs), the full relationship between the initial request for change (RFC) and the final product they produce becomes easily traceable in either direction.
Test & Release Management
The potential for a quantifiable improvement in quality is clearer still if working practices further dictate that releases into the production environment contain only executables that can be clearly demonstrated to have been generated from source held entirely within the CM repository, with all builds carried out in a controlled and repeatable manner (Build Management), and for which a clear set of test results is available.
The change management process can also interact with any release and deployment management processes to ensure that deployments occur strictly in accordance with any release plan or calendar that may exist.
Post Implementation Review
Once the change (be that RFC or SCR) has been completed then a review should be performed to determine not only whether it was successful, but also whether it was completed in accordance with the original request as approved by the CCB. In particular, the results and benefits actually obtained should be compared with those that were stated as requirements when the change was first proposed to see whether they have been realised. The aim of the review is not to apportion blame for failure (a ‘witch hunt’) but rather to learn lessons from which future improvements can be derived.
In a Closed Loop CM process, all lower-level changes (SCRs) that are subordinate to a higher-level change (RFC) should be closed and subject to a Post Implementation Review before the higher-level change is closed. It then becomes a simple matter when reviewing the overall change to ensure that no detail, however small, has been omitted or overlooked. The originator should be notified when the change has been completed.
The CM process should evolve along with the business and support it as it grows and its needs change. Naturally, changes to the CM Process should themselves be the subject of an RFC and performed in accordance with that process!
As with any process, there will of necessity be exceptions.
The most obvious is the case of a genuine emergency, such as a system failure that requires immediate action, possibly within the production environment, to recover it. In such an event there should still be a process, but of necessity much tighter and faster and supported by some form of Emergency Change Request (ECR), although again the name will in practice vary between organisations. This is simply to ensure that:
- Unauthorised and unrecorded access to the production system is not available under any circumstances.
- What is probably a ‘quick fix’ at the time will later be backed up and supported by careful analysis of the cause of the problem, together with a RFC/SCR as appropriate to authorise, effect and record the permanent solution.
Another exception that warrants a shorter process is when the change required is very small, both in terms of impact on other areas and implementation cost. Whilst following the full change process is clearly desirable in many respects, it can potentially have a significant administration cost associated with doing so and it is sometimes appropriate to reduce this cost through the use of a shorter management process. It could be argued that the relationship between the lighter-weight SCR, compared with the full RFC, is an example of this. Different change processes may also be necessitated by different development methodologies, such as waterfall versus iterative.
Costs of Configuration Management
Certainly this quality and control costs money to achieve, but failing to achieve it is likely to result in a higher cost still, not only in terms of the direct cost of correcting the error but also (most notably in the case of a business dealing with the general public, i.e. B2C) the indirect cost and effect on customer-confidence of having made the error in the first place. Experience shows us that it is not uncommon for the indirect cost of an error to far exceed both the direct cost of correcting it and the investment cost of avoiding it in the first place.
It is notoriously difficult to gather metrics on Configuration Management because, while it is easy enough to record the cost (both direct and indirect) of any failure and of rectifying that failure, and of course the cost of any control systems that are put in place can be relatively easily established, it is almost impossible to determine what failures those systems prevented and thus how much money they have saved. This does not make effective CM any less important to a business though; it simply requires a more enlightened management board to commission its implementation.
Once in place, the CM system can be used to efficiently and cost-effectively gather metrics as any change progresses through its lifecycle. Those metrics can then be used to assist in future planning exercises and so further improve quality.
Standards and Methodologies
The principle standards and methodologies pertinent to CM are:
- CMMI: Capability Maturity Model Integration (http://www.sei.cmu.edu/cmmi)
- ITIL: IT Infrastructure Library (http://www.ogc.gov.uk/itil)
- COBIT: Control Objectives for Information and related Technology (http://www.isaca.org/cobit)
The Capability Maturity Model (CMM) was developed by the Software Engineering Institute (SEI) at Carnegie Mellon University in Pittsburgh. It ranks software development organisations in a hierarchy of five levels of maturity, each with a progressively greater capability of producing quality software. Each level comprises different numbers of ‘rules’ to follow. If an organisation is on level 1, it only follows a few of the CMM rules; if on level 5, it follows everything from CMM. In practice it is relatively simple for a small organisation to achieve level 5, and substantial work for a multi-national to achieve level 3, so it is a rating that needs to be understood and used with caution.
CMMI is essentially an ‘updated’ version of the original CMM. It is designed to have applicability across a wider range of integrated disciplines than just software engineering, and additionally to support a more modern iterative development process rather than the more conventional waterfall approach.
The Information Technology Infrastructure Library (ITIL) is a customisable framework of best practices that promote quality computing services in the IT sector. ITIL addresses the organisational structure and skill requirements for an IT organisation by presenting a comprehensive set of management procedures, and a process model, with which an organisation can manage its IT operations. These procedures apply to all aspects of IT infrastructure and service management. ITIL was developed by the UK Office of Government Commerce (OCG), and is embodied in the ISO/IEC 20000 standard.
The Control Objectives for Information and related Technology (COBIT) framework is a set of best practices for IT management created by the Information Systems Audit and Control Association (ISACA), and the IT Governance Institute (ITGI). COBIT provides managers, auditors, and IT users with a set of generally accepted measures, indicators, processes and best practices to assist them in maximising the benefits derived through the use of IT, and in developing appropriate IT governance and control in a company.
Commercially Available Tooling
There are a number of solution providers who specialise in supplying tooling to assist in achieving any or all of the above, and all will be happy to supply not only their tools but also consultancy services to assist any business in achieving a successful implementation. The major vendors (in alphabetical order) are:
- Borland (http://www.borland.com)
- IBM Rational (http://www.rational.com)
- Serena (http://www.serena.com)
- Telelogic (http://www.telelogic.com)
It is important to realise however that the key requirement is for a business to identify and define an effective CM process that meets its requirements, and that the toolset must then be selected on its ability to support those requirements and processes. The process should not be determined by the toolset. Some vendors offer more than one tool, and some tools are more configurable to a customer-specific process than others. The trade-off is obvious enough in that the more configurable tool is better able to support the specific process, but it will also perhaps cost more to implement, deploy and maintain. It is to be expected that the salesman from any given vendor will identify a process to support the customer’s needs that can be precisely supported by their own company’s tool. The moral here is to shop around, and if you are still uncertain get some truly independent advice before making a commitment to a plan, process and tool that will be central to your business for the next decade or more.
Increasingly, interconnectivity with products from other vendors is becoming a deciding factor for many customers, and the sensible vendors are being steered by this evolving market trend. In particular a new open framework called Alf is evolving fast, and with the support of many key players in the software industry. You can read more about it at http://www.eclipse.org/alf.
Regardless of what is chosen, the tool-set alone does not provide good CM, any more than a car provides a good driver or a word-processor makes a good author! Good CM requires not just a process carefully tailored to the organisation’s exact method of working, but often also requires both a cultural change and significant changes to working practices throughout the organisation. Such a major change should be implemented carefully and only after careful consideration of all the factors involved.
While it is certainly true that it is faster to change the culture of a young and small company, than it is of a large and established multi-national corporation, it is the author’s experience that any such change should not be rushed. To use a military phrase, it is vital to “win the ‘hearts and minds’” of the planned recipients (and hopefully beneficiaries) of the system. This is best achieved through consultation, PR and (prior to deployment) training that is tailored to the process that is being deployed. The natural resentment of many people, and programmers in particular, to any perceived restriction of their freedom or creativity should not be underestimated.
While any of the above mentioned solution providers and many specialist training companies will be happy to supply generic training on either their products or CM in general; the benefit of providing training that is tailored to the individuals who will receive it, using their own processes and their own data, should never be underestimated. Indeed, it is the training and initial support (‘hand holding’) that can often make the difference between user acceptance, and a successful implementation that is ultimately worth far more than it cost … and an ill-received failure that is resented by all and eventually withdrawn.
However, the CM solution should be implemented with the clear and overriding intention of providing benefit to the business, and if carried out correctly there will ultimately also be benefit to staff, customers and shareholders alike.
About The Author
Andrew is an independent Change and Configuration Management consultant with more than 20 years experience in the IT industry. If you have any questions regarding the above, or think that he may be able to assist you in any way, you are welcome to email him via the Contact Us page of this website.