Risk management for laboratory automation projects:Part 2
| A Tutorial from the Journal of the Association for Laboratory Automation|
Originally appearing in JALA 2004 9 72-86 (view)
Risk factors during project definition and startup
The risk factors that may be encountered during the start-up phase of a project can be divided into four main areas:
- Project definition
- Sponsorship of the system and user commitment
- Impact of the system on the organization
- Management of the project
Each is discussed in more detail below. This section is the longest in this tutorial since, if this is not defined and agreed at the start, the whole project is worthless and has a low probability of success.
Risk management during project definition
Outlined in Table 1 are some of the key issues that should be considered for risk assessment and management during the proposal definition and writing of any automation project. The areas of low and high risk (relating to possible and probable risk, respectively) are highlighted in the two right-hand columns for each factor. Every factor is considered below with suggested ways of managing the risk posed. The main effort in this phase of a project is commonsense management. There are no excuses.
Project definition and deliverables
Before starting a project, it is common sense to define the overall scope and tasks the new system will replace or carry out. It is important for all concerned that this is achieved in a project proposal or definition document. The content should explain, in non-technical language, what the system is to achieve when it is delivered. As this is the baseline for all future work on the project, it is an essential deliverable.
Moreover, the users and management must accept and underwrite the content of this document. The alternative is a poorly defined project with no focus. Thus, it is easy to introduce trivial or non-essential functions at the whim on an individual, which can waste time and effort, or worse, functions with little practical use. Moreover, a poorly defined project can select the wrong equipment or application to meet business needs.
The deliverables expected throughout the SDLC should be outlined at the start of the project to avoid not meeting user, management, and, where appropriate, regulatory expectations. Therefore, time should be spent discussing with the users, management, and especially the project team members the importance of any deliverables. Within a regulatory environment, these deliverables will form the basis of the quality development and validation of any automated system.
Defined business benefits
To avoid premature cancellation of the project due to budget cuts or management change, define and, if possible, quantify the business benefits that the system is anticipated to deliver. Senior management will need this information for project approval if over a preset spending limit, and continued management support is essential for longer-term projects. The advantages or direct benefit to the organization and the user should be outlined. To obtain the information to write this authoritatively, involve experienced staff from the user environment as well as laboratory customers.
The business benefits are useful for defining, in another way, the target of the system to be developed. This definition can be of positive use in helping to make decisions concerning which functions are to be evaluated during selection and development.
System requirements and documentation of current procedures
At the start of a project, the system requirements are relatively vague and can hide a number of complex technical, procedural, and even organizational issues within them. However, even at this early stage, the requirements of a project and the operations carried out within the target laboratory should give an understanding of the degree of complexity involved. If the system to be implemented appears to be complex, a number of approaches to reduce the risk can be suggested:
- A complex system could benefit from a detailed systems analysis to understand the information and data inputs, internal operations, and outputs. This should give a better understanding of the requirements and may help the new system support decision-making.
- As users often find it difficult to explain exactly what functions they do or are uncertain what they want the system to do, this may suggest a prototyping development with engineering or software projects. This approach should help the users debate and develop what they require and reduce the risk of the overall project, as the target can be scoped and the functions defined based on the experience of the prototype.
- A review of the working practices of the laboratory should also reveal if the processes undertaken should be changed prior to the introduction of a new system. One area where this is important if electronic signatures are to be used: an electronic process has to be defined, as the existing paper-based process will be inefficient.
If current procedures are documented, these will help define the current practices and system. In contrast, poor, outdated, or no documentation can cause assumptions, perhaps wrong ones, to be drawn and requirements defined from incorrect information. Great care must also be taken not to assume that even if practices are defined, say in standard operating procedures (SOPs), that the current working practices in the laboratory match them. No assumptions should be made in this respect, as these documents may be major works of artistic fiction. Enlist the help of expert users to help define the current system.
It is essential to define current working practices, modify them where necessary, and map them onto a proposed system to help selection.
Knowledge of the project team members and users
The skills of all of the project team members should be assessed before the start of the project. Clearly, where the IT and laboratory automation members have been involved successfully with similar projects in the past, especially within the same area, there will a high degree of confidence and technical skill. In contrast, a team that is new and has little experience will require team building and technical training, ideally, before the project starts.
Equally, the experience of the user representatives working with the project team members may not be adequate to get a high-performance team working immediately. Therefore, some on-the-job training in project team membership may be appropriate.
An issue of major concern is the degree and depth of understanding and knowledge each member has in the other team members' disciplines. When there is none, a level of common understanding has to be developed. Equally important is the degree of knowledge and understanding the computer staff has in the laboratory. In the author's experience, this understanding takes much effort to acquire but is a worthwhile investment. A corollary is that carefully trained IT staff must be retained by the project; otherwise, momentum will be lost. To reduce risk before the project is fully underway, management has the responsibility to ensure that the team is formed and has the required level of knowledge and understanding to do the job.
Interfaces to other systems
This aspect is not always identified in project proposals, but for integration to form an efficient organization, IT and automation projects should not exist in a vacuum but interface with each other to provide an integrated information environment. If the system is a standalone, no interfaces with other systems are deemed necessary, and development can proceed unhindered. In contrast, if the system has to interface with one or more systems, this adds complexity and risk. The interfaces, especially the data inputs and outputs, must be carefully defined and documented along with the responsibilities of who does what. This is acceptable if the systems already exist and are functional, as the interfaces are tangible.
Problems can arise if the interfaces are with proposed or partially developed systems, since interfacing with these applications increases the risk assessment. Now, additional time is required to identify where the projects overlap and how they should interface; liaison between projects is essential. Liaison may include sharing project team members, planning inter-project dependencies, or identifying the other project's deliverables.
Resources for project development
Smaller projects carry less risk. Therefore, to minimize the risk for larger projects, there are a number of measures that can be used to reduce the risk to acceptable proportions:
- Ensure formal project planning and monitoring with clearly identified deliverables and milestones, although this can be time consuming, and project team members could lose enthusiasm for the project. It is essential to focus members on the original aim of the project.
- Large projects can be broken down into smaller ones with discrete endpoints. These smaller projects, when complete and aggregated together, constitute the overall system. Alternatives are to reduce the original project scope and produce a minimum working system that can prove its effectiveness before additional functions are added or by using a phased development approach.
- Large projects developed over long time periods can cause problems in maintaining enthusiasm and user commitment. Moreover, any organizational changes could result in changes in sponsorship and less commitment or resources for the project.
- Large projects could justify the use of application development tools, such as computer-aided software engineering (CASE), to enhance productivity and decrease the time taken for various phases of the project. If the team has used this approach successfully in the past, this would be beneficial to the project. However, if this approach means introducing new technology, it may increase risk instead of reducing it.
In today's organizations, a timescale of 2 to 3 years is a long time; in some companies, this can be the expectancy of an organizational structure or time between mergers. Therefore, if the timescale exceeds this, the project is unlikely to complete before the next change and is very unlikely to bring business benefit. Therefore, projects that last longer than 18 to 24 months are high risk. As described above, reduce the scope or break the project into a number of clearly defined phases. However, in doing this, it is essential that each phase provides defined and quantifiable business benefits of itself, or it is not work doing.
Risk factors associated with sponsorship and commitment
Presented in Table 2 and described below, are the risk factors associated with sponsorship of the proposed system and the commitment of the users to accept it. Although presented here as factors associated with the start-up of a project, they must be reassessed during the project in the light of any changes in senior personnel, organizational rearrangements, and influences on the users. This is especially so in a project that has a long timescale or has been delayed.
Sponsorship of the project
The best way of identifying a project sponsor is to ask the question, Who provides the money? Active sponsorship of large projects is important to persuade people to use the new system. A sponsor who is just a figurehead is a green light to wasting a large amount of money, as there will be few questions asked if the project fails to deliver. Senior management's understanding of automation and computer projects is usually based on two premises:
- The project is expensive
- It won't work
These perceptions must change.
If the project sponsor is not strong, political battles within the organization units under this individual can result in project delays due to a lack of decision or management commitment, especially in large projects. Therefore, to avoid these problems, procedures for resolving disputes should be instigated by the user management.
Attitude and commitment of user management
The attitude and commitment of the user management is essential for the success of any project. Apparent lack of commitment may indicate that they are unaware of the potential benefits that the system may bring or of plans to change the laboratory's direction. The manager of the user area should be briefed regarding the benefits that the system should deliver and its ability to enhance the business objectives of the specified area.
Winning the hearts and minds of user management is one option. If the project sponsor links a performance bonus to a successful project implementation, this adds the dimension of the wallet or purse to the equation.
Attitude of the users
Even with total commitment of the project sponsor and user management, users can cause serious problems throughout the whole project by refusing to cooperate during all stages of development. This may be the result of fears of radical change that would result from the operation of the system. Mechanisms for effective communication to representatives of the user community need to be established by regular status reports or meetings that should be continued throughout the development cycle. Concerns of the users should be communicated to the project team and management. If there are organizational impacts of the system (see “Changes in Organizational Structure”), these should be identified and communicated to the users and discussed to obtain a consensus agreement.
The maturity of a user organization to support a LIMS effectively is a factor to be considered early in a project. If, in the judgement of the project team or user management, the organization is not capable of supporting an automated system, then an education programme should be undertaken for the whole user base.
Not often considered when assessing the risks of a laboratory automation or IT project is the capability of the organization to implement the new system. This will vary, but a simple way to find out is to review how successful previous projects of this type have been. Projects that have been an outstanding success, rare but they do occur, will be a pointer towards the successful design and exploitation of automation or IT. It may also indicate that risk taking is encouraged, although this needs to be established by direct evidence of a corporate policy or equivalent.
More often, if this is the third or fourth attempt at a project, it will indicate a poor track record and, therefore, a high-risk project that has to be handled more carefully and in a risk-averse manner.
Relation of the project to the strategic plan
If a project falls outside of the scope of a strategic automation or IT plan, then the risk increases as the obvious question arises, Why is this project important? Conflicts may result from an unplanned project being given priority and resources over existing ones. It is best to find out the reasons for a new system that is outside of an existing plan. If there has been a change in the business strategy, then the IT strategy should be revised accordingly. The place of the new project and any dependencies among other projects should be identified. A strong and committed project sponsor may be required.
Risk associated with the impact of the system
Described below and presented in Table 3 are some of the risk factors to be considered when examining the impact of the new system on an organization.
New or replacement system?
Regardless of whether a new or replacement is considered, both present a high risk; however, the nature of the risks are slightly different and are discussed here. Both approaches impact the user community by requiring it to change; it is the issue of change that presents the risk and, thus, it must be effectively managed.
A replacement system should not present too many problems with respect to the impact on the organization, if, and only if, the replacement simply automates what was automated in the previous one. However, many IT applications do not fully automate the process, and automating the status quo simply perpetuates the problem. Replacement of an existing system should be undertaken as enumerated here:
- Understand the strengths and advantages of the current system, as these have to be maintained in the new system.
- Understand the weaknesses and disadvantages of the current system and ensure as many of these are overcome in the new system—this is the enticement for the current user community to migrate to the new system.
- Evaluate the current scope and boundaries of the system. Do they reflect current business needs?
- Review how the system is used (e.g., fully electronic versus manual input from paper printouts) and see what improvements can be made.
Design the replacement system with the current business needs and processes in mind, not the old ones. However, a new system tends to impact many areas, including:
- Changes to existing ways of working
- Time to learn to use the new system effectively
- Computer or engineering skills to use the new system
- Organizational maturity to use the system
- Staff might be unsure of their duties and responsibilities when the system becomes operational or they could resist its introduction
To counter these impacts, management should assess the effects of the new system on the organization and the users. Communication of the benefits of the new system to the users should be undertaken, but remember to keep the statements realistic to manage user expectations. This discussion can be achieved in groups or individually.
Involvement of the users in all stages of the project is essential. Areas where this can occur are project planning, analysis, testing prototypes, and implementation. Request input on how to structure and phase the training to use the new system. A champion or champions for the system should be identified and involved throughout the project.
Impact of the system on the IT department
As systems become highly integrated environments and work in close cooperation over networks, a new system can have different levels of impact. If there is little change in the operation and management of the computer, there will be few problems apart from negotiating the facilities management contract. At the other end of the spectrum, there must be enough capacity or bandwidth on the network to accommodate the anticipated data flows from the laboratory to the server plus sufficient workstations and peripheral devices to access the system. In short, ensure there is sufficient capacity for the new system to operate effectively.
If the IT department has no experience with newly acquired hardware, operating systems, and/or application software, this will have an impact on the support staff and will increase risk accordingly. Organization members should have the skills and experience to run these new methodologies efficiently. If not, they will have to be acquired by training existing staff or recruiting new personnel. The input from operations staff to the project team during evaluation and selection can identify many of these areas. To reduce the risk and aid communication between various applications, the first intent should be to purchase or develop a system that is consistent with the current systems in place within the organization. This aspect will be considered in more detail in “New or Non-Standard System Components.” The author would advocate using corporate standards whenever possible, as this will be the easiest to implement.
Documentation of system procedures, coupled with effective training, to use any new hardware and operating systems must be in place before the system goes live.
Procedural changes required by the new system
Project risk can be greatly increased with the failure to recognize early in a project the need for any new or revised procedures; thus, plan and implement them rapidly. The current working practices should be reviewed, and the level of the user's commitment to change procedures should be established. If change is resisted, do not implement change via the computer system but change procedures, if possible, by altering the manual ones first. This is a relatively cost-effective route to take, as small modifications can be undertaken easily and rapidly until the new manual procedure is streamlined and effective. Then, overlay the new system on top of the modified manual system. In this way, problems can be resolved with the new procedure without the computer being used as a scapegoat by dissatisfied users.
Changes in organizational structure
Computer systems have the power to cross functional and organizational boundaries with ease. The failure to recognize and plan for any changes may result in staff not knowing new responsibilities or roles or in disruption occurring from reorganization of organizational units; hence, there will be an increase in risk to the project.
The impact of any organizational changes should be documented clearly during design and development, although it may be alluded to in the project proposal wherever possible. There should be change management of any such changes over a specified time period. Always, if possible, allow time for the changes to settle down before implementing the new application to avoid too much change in a short time period. Again, the communication of the realistic benefits of the new system to the users should be undertaken.
Policy changes to accommodate the new system
Changes to policies should be identified and controlled by the user management. Since these are not always identified until the detailed design stage or the development stage, any delay in implementing these could delay the operation of the project. The resolution of these policy issues must be made before development can take place.
Policy changes may be the result of the introduction of new technology, organizational changes, or modification of procedures caused by the new system. Therefore, it is important to identify and resolve any policy changes rapidly but not before considering the impact of the changes. If large numbers of policy changes have to be made, there should be a mechanism in place to document and inform all staff of them—a user appointed as a coordinator might be one approach to use here.
Risk factors associated with management of the project
Presented in Table 4 are the common problems that can give rise to risk when assessing the approach to project management and the membership of the project team.
Experience of the project manager
An inexperienced project manager may have difficulty developing an efficient project plan and modifying that plan as the project progresses. Moreover, not all tasks may be identified, or the project plan may not be broken down to a sufficient level to enable accurate scheduling. Taken together, this often results in delays to the project and missed deadlines, with tasks rescheduled or additional ones included, often at short notice. The impact can be damaging to the project, as budgets may be increased, and there may be loss of confidence by the users or cancellation of the project, as benefits have not been obtained in a timely manner.
To counter this, the project manager should be trained in project management techniques. When drawing up the plan, allocate more time for the completion of the tasks to allow for slippage, or allow slack time. Regular reviews of project progress should be set up. To gain from the experience of others, read the status reports and reviews of similar projects completed within the organization.
Full-time project manager
Depending on the size of the project and the resource available, it is preferable to have a full-time project manager. This avoids conflicts of interest with line management responsibilities and allows the ability to focus on key issues that might not occur if it were a part-time position. In some respects, this is a management decision about the amount of time and resources allocated to a project. However, there is also the onus on the project manager to inform management if he feels overworked when given dual responsibilities.
Full-time project team
Whilst it is common for the project manager to be allocated full time to the project, the team members are usually allocated on a part-time basis. Here, the line/matrix conflicts outlined in the last section will become apparent as the project competes with the line for the resources of these skilled individuals. In this situation, errors can be made or delays occur that could impact on the project, ordinary work, or both.
To manage this situation effectively, it is important to define accurately the amount of time that a project team member should spend on his respective duties. This will reduce the amount of time available for line work, and, accordingly, the manager of the individuals should negotiate with clients regarding deadlines involving these staff. If specific tasks for the project such as in-house evaluation require an individual's time, this must be negotiated with the supervisor well in advance of the event.
Ideally, the project team member's position description should be changed to reflect the work done on the project so that both the line manager and the project manager can evaluate the individual's overall performance.
Project members operating as a team
When the project team is composed of members who have not worked together before, some delays may occur in the initial stages of a project. Team members need time to get to know other member's personalities, understand their skills, strengths, and weaknesses, and learn how to work together. Risk arises if the team lacks skills or understanding of the technology involved or the knowledge to complete the project successfully.
To overcome this and reduce the risk, attempt to use staff that has worked together as a team. Working to the strengths of an individual is always preferable to training another member. However, this approach carries the risk that IT and automation skills can often be in short supply, and one individual can often be carrying out several tasks, which can conflict with line duties. A way to transfer skills should be included when feasible and when time and resources allow.
Experience with the application or system
Often, the majority of project team members from the user areas have little or no experience with a new type of application. Without experience, the team will not have the insight to avoid mistakes or enter blind alleys. Additional time may be required for reviews and revisions of the plan and its execution.
If similar projects have been introduced in the organization, utilize the knowledge from some of the team members as an internal consultancy role. Ensure that more time is allocated to the project to allow for problems.
The concept and introduction of a corporate LIMS or an automation project may involve two or more sites. Whilst from the corporate viewpoint, this is an effective use of resources for development and maintenance, and the benefits will be significant over the development of different point solutions at every site, problems will be encountered. By its very nature, a project covering more than one site tends to be larger, more complex, and hence more expensive than one at a single site. Senior management should look carefully at these projects, as a significant proportion of their IT budgets will be involved.
Communication may be difficult, especially if there are time differences involved that are greater than one or two hours. This can be overcome by the use of electronic mail facilities or an Intranet site for common-interest items. Progress updates will need to be regular for all sites and held centrally at one location to ensure control of the overall project.
There may be lack of coordination at the sites where the project manager is not located. Travel, often extensive, will be involved for the manager and several key members. Budgeting of money for this and the associated subsistence is essential. Different sites may have different working practices, policies, and organizations. These typical issues, raised by a standard system, will have to be resolved at the senior management level before much progress can be made. In companies working on a global basis, there will also be cultural differences, methods of working, statutory holidays, and even the length of the lunch break to be taken into account. The timescales for the project will need to be increased to account for these factors. However, the overall benefits to the organization should outweigh these difficulties.
Goto Part 1; Part 3
- ↑ McDowall RD. Exploiting the benefits of 21 CFR 11. American Pharmaceutical Review. 2002;5(1):.
|Click [+] for other articles on||The Market Place for Lab Automation & Screening||The Market Place|