ITSM Tools Operation Continuity Plan – Part Two

We discussed the first several topics (scope, data considerations, and external dependencies) that should go into an ITSM tools operation continuity plan during a disaster recovery (DR) scenario. In this post, we will conclude the discussion by covering the people, procedures, checklist, and validation sections of the plan.

Recovery Team and Communication

The section spells out the key roles and the staff members who will be responsible for implementing the plan. It will include the typical information such as name, organization affiliation, location, and contact details. This section should also include agreed upon communication protocols for the duration of the recovery process. For example, teleconference bridge information could be something useful to include, if that is your organization’s standard practice. Also, if your organization has a major incident handling practice with standard communication and reporting requirements, you can cover the pertinent details within this section or at least make a reference to it.

Recovery Procedure and Configuration Details

This section outlines the recovery instructions and procedures with as many details as necessary for a successful recovery. This section also provides all configuration details that need to be in place in order for the recovery solution to work as designed. Just how much information do you really need to include in the document? I believe the level of detail will largely depend on whom your organization anticipate will execute the plan or heavily influenced by your organization’s disaster recovery plan.

By default, I would recommend having the level of detail based on the assumption that the people who will execute the plan are NOT the same folks who operate the production environment pre-disaster. This is the safer router in my opinion.

For example, your organization uses ITSM product XYZ, made by company ABC and operated by an off-shore team from vendor OPQ. When a disaster strikes your data center, the network connectivity between your data center and your production personnel from OPQ was cut off. At this time, your OPQ vendor has another team at a different location who can access the recovery site and knows your ITSM product. With a properly documented recovery plan with the right level of details, you can leverage your vendor’s second team quickly to start working on the recovery without any help from the production team.

Checklist of Key Actions or Milestones

To help everyone who needs to understand and execute the plan, I recommend having a section that outlines the key tasks and activities so no important activities will be missed. The checklist can also be used to show the various communication and escalation points.

Testing and Validation Steps

This section outlines the detailed testing strategy and validation activities required to implement the recovery plan. Similar to the recovery procedures section, it is highly recommended that these instructions are comprehensive, so the recovery solution can be fully tested by people who may not be the same team as your production crew. The section should also account for all potential conditions, events, and scenarios. The section should contain information that can be used during the testing such as:

  • What are the success criteria for the test objectives?
  • What are the assumptions defined for this test, if any?
  • When is the testing window?
  • How should the security setup be taken into consideration?
  • Which tool modules will be tested?
  • Who will be conducting the test?
  • What are the steps to validate the application and data integrity?

Return-To-Normal-Operation Procedure

This section describes the instructions/procedures necessary to return to the normal production environment once it has been restored. For this section, I would say probably the most key information to explain is how the data will be synchronized between the production and the DR systems. Again, the key objective is not to lose any data captured by the ITSM tools while it was operating in the DR setup. This section will also include the necessary testing and validation steps to ensure that the production system is back in full operation, so the DR system can be fully shut down and return to the stand-by mode.

An ITSM Tools Operation Continuity Plan Example

I hope this particular discussion on the ITSM tools continuity plan provides something of useful tidbits for your organization to think about and to plan for. Considering the importance of the data and transactions captured by the ITSM tools for many organizations, having a workable continuity plan is really not optional anymore. If you don’t have anything in place now, I recommend start small by drafting a plan and making the scope compact enough to cover just a few high probability and high impact scenarios. Work with someone within your IT organization who is responsible for the overall IT continuity plan, so your plan can integrate well with the overall IT and business objectives.

SACM and CMDB Tools – Strategy and Roadmap Example

Partly due to my track chair duties with the Fusion 12 conference, today’s post will be a bit light. I was going to leave the SACM and CMDB posts as they were, but I went ahead and put together a strategy and roadmap example deck. You can use the information presented in this deck as a starting point to formulate your own CMDB strategy and roadmap.

SACM and CMDB Tools – Strategy and Roadmap Example

As we discussed in the previous posts, the SACM process can enable and affect a number of other ITSM processes, so planning for it can seem overwhelming at times. My recommendation is to initially start with a well-controlled scope and focus on the value-added activities that can bring the quick wins. As usual, please feel free to comment or discuss anything directly with me if you like.

SACM and CMDB Tools – More Planning Considerations – Part 3

In part 2 of the series, we discussed some key considerations for a CMDB solution and expanded on those areas such as the Scope, the Roles and Responsibilities, and the Data Model. In this post, we will discuss additional areas such as Control and Verification, Key Performance Indicators, and Awareness Campaign, Communication & Training.

  • Control and Verification: Who can add, change, or delete information from the CMDB and under what circumstances? Will you use some type of a gate-keeper process, like Change Management, to regulate how information coming into and being updated in CMDB? Remember trust and authenticity of the CMDB data are what make the CMDB useful. How do the relationships between CIs get determined and mapped? Not all relationships can be mapped automatically by the tools, so some manual changes to the CIs and relationships are still necessary. All updates, automated or manual, to the CIs should be tracked by the CMDB tools with a complete history and audit trails. How often do you audit the accuracy of the CMDB and reconcile the differences? Can your CMDB tools help by automating the verification and reconciliation tasks as much as possible? When you have hundreds and thousands of CIs in the CMDB, manual audit and reconciliation get labor-intensive really quickly and degrade CMDB’s usefulness rapidly over time. This area will need some well-thought-out and clear work instructions, so people will know exactly how the CMDB will be maintained.
  • Key Performance Indicators: What metrics will you track in order to measure the effectiveness of your CMDB solution? I will outline some potential metrics, and you need to decide what ultimately is important to your organization. When I was planning for a CMDB solution not too long ago, I used Randy Steinberg’s “Measuring ITIL” book for gathering metric ideas and inspiration. I would recommend checking out Randy’s book, not just for configuration management but for other ITSM processes as well. The lists below outline some metrics suggested in Randy’s book and some of my own.
    • Some example operational metrics:
      • A: Total Number of CIs
      • B: Number of CIs Audited
      • C: Number of CI Errors Discovered
      • D: Number of CI Changes
      • E: Number of CI Changes Without Corresponding RFC
      • F: Number of  Incidents Related to Inaccurate CI Information
      • G: Number of  Change Failures Related to Inaccurate CI Information
      • H: Number of CIs Without Assigned Ownership
    • Some potential indicators:
      • CMDB Audit Ratio: B / A (Target 100%)
      • CMDB Accuracy Ratio: 1 – (C / A) (Target 100%)
      • CMDB Compliance Ratio: 1 – (E / D) (Target 100%)
      • CMDB Support to Incident Index: F (Target 0)
      • CMDB Support to Change Index: G (Target 0)
      • CMDB Ownership Rate: 1 – (H / A) (Target 100%)
  • Awareness Campaign, Communication & Training: What mechanisms will you use to get the words out about the CMDB solution? I can guarantee you that, as soon as people find out a CMDB is in the work with information that can impact or benefit their lives, they will want to have their say. This is where the governance decisions come in. Everyone is welcome to provide input, and they should especially during the data model design phase. How will the input get taken into account and how will the decisions be made? During the implementation, how will the progress be communicated? Expectations need to be reasonable and in-check because my experience tells me that people often have an inflated idea about CMDB and what it can do. Sometimes we don’t do ourselves a favor by hyping up the CMDB solution too much at the beginning. After the CMDB is ready, who will operate the tool and need to get trained? A CMDB solution is a good thing to get excited about, and people intuitively understand the benefits of CMDB when it is done well. However, we often get into the trouble of promising too much and delivering too little when it comes to CMDB. Awareness and communication are two things that will need to be sustained during design, during implementation, and well into the maintenance / care-and-feeding phases.

There are a lot to think about when planning for a CMDB solution and, for many organizations, those consideration areas I have mentioned barely scratch the surface of the real issues they face. Like many ITSM initiatives, SACM and CMDB solutions are very much people endeavors, with the process and tools, while still important, playing more of a secondary role. In the next post, I will walk through an example CMDB solution implementation scenario. It is by no mean and end-all-be-all approach, but it can be a starting point for some organizations that can use a starting position.

Links to other posts in the series

SACM and CMDB Tools – Some Planning Considerations – Part 2

In the previous post of the series, we discussed what initial considerations should be taken into account when deciding whether your organization needs a full CMDB solution. Let’s say you went through the analysis and decided that a more functional CMDB solution is needed. You need the solution to have a better ability to assess the impact of a change, incident or problem on a service because your current analysis capability is not meeting the business needs. What other considerations need to come into the planning of a CMDB solution? I would suggest the following…

  • Scope: What data need to be captured, stored, and processed? What processes and activities will make use of that information? At this point, you should have some pretty good ideas how to answer those two questions, at least with some degree of precision. If you need a better handle on managing the changes within the data center, what sort of hardware and configuration information will you need to capture from the servers and the networking gears in the data center? Do your needs for CMDB/asset management involve needing configuration information from the client devices? If so, how do you plan to reliably capture that very fluid information? What processes do you plan to target the CMDB information for in the near term, just change and incident or more? What services or processes will the CMDB enhance in the long run? I would recommend start with what you absolutely need right now to show improvements in service and gradually build it up over time.
  • Data Model: Talking about what data you need conceptually need is one consideration. Translating those concepts into actionable data model design is another. I am a believer that a CMDB is an information cornerstone to an organization’s IT service management activities. I often compare how an ITSM system relates to IT as how an ERP system relates to an organization. The CMDB is the foundational information store for that “ERP” system for IT. With that said, it pays to implement the data model well upfront because frequent structural changes to CMDB afterward will easily kill the productivity you gain from using CMDB.

There are some potential benefits and understanding that can be derived from the data model:

  • How all CIs within the scope of the process relate to IT services provided to the business
  • How various total cost of ownership components are related to an IT service
  • How individual availability or capacity figures can relate to groupings of CIs and overall availability or capacity targets
  • Which CIs facilitate or enable which IT services
  • Prioritization of CIs in relation to disaster recovery or continuity management

If your organization has one, now is the great time to get in touch with your Enterprise Architecture (EA) folks and work on the data model. They should be your organization’s information and data architecture expert, and designing (or assisting in design) an ERP data model for IT should be the very part of their charter. Design a data model with a long-term end in mind. Try to put into as much forethoughts into the design as possible. The data model should remain structurally stable for the most part, with an occasional addition or removal of fields or attributes. If you find yourself needing a brand new entity or object in the schema due to legitimate business needs, that is OK, too. In any case, my experience tells me data model design is something to be taken very serious of, because it frequently plays a big part in making or breaking a CMDB effort. Get some competent, professional help and start collaborating across the organization boundaries.

  • Roles and Responsibilities: Who is your configuration management process owner that has the overall accountability for governance matters with access to senior IT management team for escalating policy discussions, if necessary? A CMDB manager role should be named and responsible for managing the operational activities of the process and ensuring integration with the other service management processes. The process owner or its delegates need to work on developing the CMDB data model (with EA’s help), the core policies, maintenance processes and procedures, key performance indicators and producing ongoing management reports.

Also, another important topic to work out is the data ownership issue. Another word, who will own which portions of the data in CMDB? Naturally, I think it is most productive for the organization when the CMDB data repository structure is centralized. Does that also mean the CMDB care-taker team will also be able to own all data within CMDB? By owning I am referring to the acquisition, maintaining, validation, and reconciliation of the data. Will your CMDB team or the server admin folks manage the server device data in the CMDB by themselves, or will there be some kind of collaborative arrangement in place between the CMDB team and the server admins? How about the ownership for the networking device data in CMDB? How about the ownership for the application specific information in CMDB? The data modeling exercise may give us more insights into what data we need. I would strongly recommend the data ownership issue get thoroughly reviewed and agreed upon during the planning phase.

In the part 3 of the series, I will discuss more planning considerations such as Control and Verification, Key Performance Indicators, and Awareness Campaign, Communication & Training. Planning for a CMDB solution can be a complex but also a fun and educational exercise. You are doing the organization a great service by fulfilling its on-going need for great information and knowledge to carry out its mission. Do this value-add project well by planning ahead and think things through.

Links to other posts in the series

SACM and CMDB Tools – Initial Considerations – Part 1

Those who have implemented configuration management (CM) practice would agree that implementing a robust CM discipline with the accompanied tools is no trivial task. The assets we acquired and deployed in IT can be massive in number and sophisticated in complexity. Keeping track of things coming in and out plus all the on-going changes can be tedious and error-prone when doing it manually. That is one major reason why we look to a SACM/CMDB tool to reduce the complexity of managing those IT assets. At the same time, a lot of people I talked with have indicated that CM is also one discipline that is usually the least mature in many organizations, when compared to other ITSM processes. Intuitively most of us would agree CM is something we should do well, and properly implemented tools can only help. How should we approach this complex issue?

For the sake of simplicity, I am going to use the term CMDB as the general, overall reference to the SACM and CMDB terms and concepts in this post. In ITIL V3, SACM is much more comprehensive, and they are not the same as CMDB. That is fine. For many organizations, we are still struggling to get the basic CMDB stuff down, so that is what this series of blog posts will target.

I believe the first of two questions to ask is… “Do we know what a CMDB is?

This question may sound basic, but it is worth asking because there are widely varies opinions on what a CMDB is. I believe a data repository, which tracks and manages the inventory in the IT infrastructure with key attributes such as asset tag, description, location, owner, user, etc., is a CMDB in itself. The inventory list may be rudimentary, but it nevertheless contains the configuration items (CI) that a full-fledge CMDB tool would keep track off. With that assertion in mind, we can see that many organizations have a lot of those inventory lists in our environment. Not surprisingly, everyone seems to keep a set of those inventory lists for the stuff they own or operate. For example, the server admin folks have a set of servers, physical or virtualized, that they manage with a bunch of details about those servers. The DBA folks have a similar list for the databases under their management. The application folks have something similar for their application portfolios.

Many organizations also have asset management tools installed for their finance department, and those tools track the financial lifecycle of those assets for depreciation or accounting purposes. Many of those assets such as computers, equipment, and software are also used in IT with location and ownership information. Sure they are not well connected with each other so that you can extract the relationships between various items, but they do exist. The truth is… every organization has a number of inventory lists or assets databases that contain, in reality, CMDB information.

Now we have some ideas on what CMDB information we have. The second question to ask is… “What do we need a CMDB for?” In addition to tracking the inventory and financial information about those IT assets, do you have a genuine need of…?

  • Performing more informed impact assessments for incident and change management activities? The relationships documented between the items in CMDB can be a valuable tool for understanding what impacts a change or incident can have on a particular system or application. When one component changes or breaks in the IT environment, a good CM practice and system in place will highlight how the change in state affect the rest of IT environment. With the localized CMDB info, the relationships are sitting in each technology owner’s head, and that has always made assessing changes and incident a labor-intensive activity. The assessment can also be less than precise or accurate if you don’t have all the necessary people involved with the information you really need.
  • Facilitating systems upgrade or platform retirement? A comprehensive CMDB, with the full view to all applications and correctly documented relationships, can easily facilitate the analysis when considering introduction of the new applications, retirement of the old applications, and even changes to the overall enterprise architecture.
  • Optimizing financial and capital expenditure planning? By having the full financial information available for all CIs, it will be easier to plan or project the upcoming financial obligations in expected hardware leases, software license fees/renewal, and maintenance contracts.
  • Contributing to service continuity or contingency planning? By having a comprehensive list of CIs and a full picture of the environment, it makes easier to identify the essential, needed components or systems quickly during a disaster recovery or business continuity scenario.
  • Making changes more visible and accountable for the operations and service delivery functions? The change history attached to each CI can provide valuable insights or trigger the attention by IT Management into changes that might have data protection, regulatory compliance, or other operational implications.
  • Facilitating compliance adherence or audit obligations? Your organization or environment may have special legal or reporting requirements beyond what the typical spreadsheets and inventory lists can provide.

Once you have identified the needs and understand what you have to work with, you can make a more informed decision on how to bridge the gap. After all, the CMDB information is useful and productive only if:

  • The CMDB is the central data repository for all configuration items we want to track in the organization, plus
  • The CMDB has the data we can reasonably store and manage well, and
  • The CMDB is the TRUSTED source and is used by all IT disciplines

Buying and implementing a full-fledge CMDB tool is not the only way to plug the gap. The gap can also be filled by marrying various CMDB sources you have on-hand with some well-designed middleware mechanism that provides the integration between the CMDB sources. Of course, the gap can also be filled by strengthening what you have with more robust processes and solid discipline. It all depends on what your business really needs. In the next post, we will explore some planning considerations that can go into what does it take to build a CMDB if you find yourself really needing one.

Links to other posts in the series