Chaîne d’approvisionnement en vaccins, chaîne du froid et logistique

Authors: Mr. Peter Carrasco, Sabin Vaccine Institute; Mr. Richard Nolan, Red Lodge, Montana; and Mr. Henry Tuell, Red Lodge, Montana

Purpose and Overview of the Module

 

The purpose of this IAIM Management Toolkit module is to provide guidance to immunization managers for managing their vaccine supply chain and cold chain operations. The module does this by providing immunization managers with:

  • An orientation to their management responsibilities with respect to vaccine supply chain and cold chain operations;
  • A framework for understanding the key pieces of information needed for decision-making with respect to vaccine supply chain and cold chain operations; and
  • Checklists of management tasks to be completed for optimal planning, budgeting, and management of vaccine supply chain and cold chain operations.

 

There are six key topics discussed in this module, each of which has its own management objectives.

  • Topic 1: Communication and information: The management of the information system and the communication infrastructure that supports the flow of information regarding cold chain and supply chain operations. Providing real-time, quality information in easy-to-use formats for management decision-making and planning is the primary objective.
  • Topic 2: Cold chain equipment: The management and maintenance of the network of cold chain equipment which constitutes the cold chain infrastructure. For a manager to properly manage and plan the cold chain infrastructure, s/he will need information on the current performance of their equipment, the types and quantities of each piece of equipment, equipment storage capacity, and information from top management on any plans to introduce new vaccines.
  • Topic 3: Vaccine stock and distribution: The management of the quantities and distribution of vaccines and ancillary items (for example, diluents and syringes). Also, the transportation from the national level to sub-national level warehouses to final destinations where health providers administer them, maintained at the required temperatures. [Coming soon]
  • Topic 4: Transportation fleet and equipment: The management of the quantity and availability of the transportation fleet and other equipment to support vaccine distribution. This includes transportation of vaccines to health clinics, as well as logistics for supporting any field operations. The management of transportation fleet and equipment also includes the maintenance of said vehicles and equipment. [Coming soon]
  • Topic 5: Human resources: The management and training of personnel who receive, store, repackage, issue, reorder, transport, and track the vaccine and other items from the point of receipt at the national level warehouse to their final destinations. In many countries, equipment repair, maintenance, and vaccine distribution has been outsourced; therefore, managing and qualifying these service vendors is included in this topic. The management and performance of the supervisors, including their recruitment and training, is also an objective under this topic. [Coming soon]
  • Topic 6: Planning and budgeting: The preparation of the Annual Plan of Action (POA) that outlines the activities to be implemented for the coming year, accompanied by a line item budget. The POA should (among other things) provide the roadmap for managing cold chain and supply chain operations in support of immunization activities in health facilities, outreach activities, and immunization campaigns. [Coming soon]

 

Readers should note that the IAIM Management Toolkit is geared toward providing management guidance, NOT technical guidance, with respect to the seven major components of an immunization manager’s job. Within each toolkit module, we will provide references to technical and normative documents and tools that are commonly available from authoring stakeholders.

 

This module includes a set of Illustrative Checklists” after each topic to assist managers in assessing and improving their operations. Members can and should contribute to improving these Illustrative Checklists” by sending their contributions to info@iaimanagers.org

 

Introduction to Vaccine Supply Chain, Cold Chain & Logistics

In the area of vaccine supply chain and cold chain operations, the overall goal of the immunization manager is to manage the distribution of vaccines and ancillary items to all service providers so that the provision of immunization services to the population remains uninterrupted. This involves the following major tasks:

  • Distributing the right quantities of vaccines, on time, to all health facilities and field destinations, so that stock-outs do not occur;
  • Ensuring that vaccines are maintained at the right temperature throughout their transport, up until the point of service;
  • Organizing and facilitating the maintenance of cold chain equipment;
  • Ensuring that cold chain infrastructure has the storage capacity to accommodate the vaccines being distributed;
  • Managing vaccine transportation fleet and equipment;
  • Managing external vendors or private contractors, if some or all of the operations discussed in this section are outsourced; and
  • Planning the coming year’s operations and costs, with the following considerations in mind:

o   Reviewing budget performance for the previous year, comparing planned budget versus actual expenditures

o   Budgeting for staff training

o   Budgeting for supervision

o   Budgeting for any needed upgrades to their management information system

o   Budgeting for introduction of new vaccines and ancillary equipment

o   Expanding and sustaining supply chain and cold chain operations to the underserved populations in districts where vaccine coverage is below 80%.

o   Defining if the current human resource structure is adequate to meet the needs of the program

 

Some of the greatest challenges facing cold chain and supply chain operations today are:

  • Introducing new vaccines that occupy more space compared to older vaccine presentations;
  • Managing supply and cold chain operations that produce effective results; and
  • Reaching the 10-20% of children who still do not have access to the lifesaving vaccines.

 

While each country may have its own unique challenges that can be added to the list, these represent the common challenges that most (if not all) immunization programs face. To confront these challenges, adjust and expand operations, and (most importantly) advance program performance, managers need real-time, accurate information for decision-making.

 

Communication and Information

The management objectives of communication and information with respect to vaccine supply chain and cold chain operations are to ensure that:

  • Key information on cold chain and supply chain operations and or processes is visible (in real time, and if possible, online) to all managers or staff in order to detect and report on disruptions and solve problems;
  • An effective communication infrastructure is in place to permit the two way flow of information up and down the system; 
  • The management information system (MIS) can provide both historical information and current data on these operations for managers to evaluate performance and plan for the future.

 

The communication and information systems infrastructure can be thought of as the highway between the collectors of information and the service providers that need the information to execute their tasks.  While each country has its own management norms and infrastructure, each immunization manager and his staff need to work with those who are in command of the Ministry of Health’s MIS to assure that they can be provided the required information for decision making and planning. This ability is critical when confronting an emergency where a vaccine is one of the main interventions for stopping transmission of a disease and/or reducing mortality.

 

The operational objective in Topic 1 is for national immunization managers to make evidence-based decisions to optimize vaccine supply chain and cold chain operations, and to enable immunization staff at all levels to make evidence-based decisions and feed critical information up the chain of command. To achieve this, we outline two major task areas that national immunization managers must attend to.

 

Task I: Evaluate health facilities’ communication infrastructure (including hardware, software, and internet connectivity) and capability to receive and send information up and down the system.

 

In addition, for health facilities and outreach posts in remote geographic areas, managers should evaluate the frequency of supervisory visits made to each facility. Another key consideration for facilities in remote areas is the percentage of supervisors who have and review how many supervisors have cell phones that can be used to capture and transmit data electronically. [1]

 

Essentially, to complete this task, national immunization managers will need to holistically assess the current communication infrastructure in their country. The following illustrative checklist can be used to accomplish this task.

Illustrative Checklist to Evaluate Communication and Information Infrastructure:

What percent of health facilities that distribute vaccines and ancillary supplies are connected to the Internet?

What percent of all health providers have working computers?

What percent of facilities with internet connections can download and send reports on the status of inventories, status of shipments, and status of cold chain equipment?

What percent of all health facilities have a budget line item to support the communication infrastructure?

What percent of transportation fleet operators have devices to communicate with their supervisors or dispatcher?

What percent health facilities located in remote areas have access to either the telephone or digital cell grid?

What percent of immunization workers with management responsibilities at all levels have a cell phone that be used to send and receive information?

Is there a report (within last two years) that provides information on the above questions?

Are there any plans inside the Ministry of Health to up-grade the communication infrastructure? If so, what are the plans?

  • Is there a strategic plan that outlines the vision of the MOH regarding their MIS?
    • ​If so, does the plan spell out the role of new communication technologies, including a policy on the use of personal cell phones (BYOD)[2]?
  • ​Has the national immunization program carried out a costing study regarding the flow a paper based immunization information system?
    • If yes, what are the costs and how does it compare to the cost of using digital technologies?

For countries where immunization services are mainly provided by private health facilities, and/or where vaccine storage and delivery are outsourced: the offices that oversee the contracts with these enterprises must certify that these enterprises have the necessary equipment to communicate with their fleet and storage facilities, and certify that these vendors can provide the records that all vaccines have been stored and transported within the appropriate temperature range.

  • A good management practice is to develop a contact list with information from private health care services providers. This will permit the office with oversight responsibility in managing the private sector service providers to communicate with them when required.

Once the evaluation of the communication and information infrastructure is performed (according to the above checklist), national immunization managers and other staff can map out the system’s infrastructure and capabilities, and determine what needs to be done to enhance the information and communication system. Some national immunization managers will not have the budget or authority to undertake such a task. In such cases, the manager should estimate the cost of needed improvements as a basis for discussion with the appropriate authorities.

 

Potential improvements to communication and information systems could involve the following (as an example):

  • Providing supervisors and health care workers with laptops or cell phones with applications for collecting data which can be uploaded when the supervisor or health care worker arrives at a place where the data can be transmitted using cellular telephones or uploaded into a computer
  • In areas without access to the telephone, the person with responsibility for communications could make arrangements with civil and religious groups that have radios to transmit important information to the relevant office(s).
  • In remote areas, the MoH could request that other government entities that have radio access or satellite phones (forestry or agricultural offices, military outposts) to transmit urgent or required information to next level up.

 

Providing cell phones or similar mobile devices, including supporting the use of health workers’ personal cell phones, will require management to establish the following policies and procedures:

  • Secure transmission of data
  • Standard format(s) for collecting data
  • Integration of data transmitted by mobile device with current operating system and programs being run by the office with responsibility for managing this information
  • Establishment of protocols for the use of both employer-provided cell phones and personal cell phones

 

The management issues regarding the use of cell phones, especially when integrating the use of personal cell phones in the communication structure, are too extensive to cover in this toolkit. Managers and users can access the following links to view presentations and gather more information on the subject matter: [docs.media.bitpipe.com/item801031 or item 621283 or item 1044449]

 

Whatever arrangements are made to improve information and communication capabilities must be documented within the current MIS. One must not forget to include those offices or health providers that are not integrated into the formal telephone grids.

 

It is very important that top management at the national political level, from the president of the country to the cabinet of ministers that advise the president, understand that in a public health emergency, the ability to receive critical health information rapidly, to make an informed decision to protect the health of the public and save lives, will depend heavily on the public health communication infrastructure and capabilities. The communication capabilities of the public health system also will affect local level civil authorities and health care staff who need to receive critical information for taking action to save lives and reduce the impact of a public health emergency or a disaster – especially when a vaccine can stop disease transmission or when drug can reduce the impact resulting from the spread of a pathogen and save lives. Authorities need to know the specific details on the shipments being sent to each level so that they can make the appropriate operational preparations to rapidly and successfully distribute and administer vaccines and other medicines to target population(s).

 

Task II: Evaluate the capabilities of the MIS to provide the required data and reports for management decision-making.

 

Immunization managers at each level must determine the critical decisions to be made on a routine basis, and then determine the precise information that all immunization personnel need to map, track and report on to inform these decisions.

 

The evaluation should include assessing the types and the quality of data, given the decisions that need to be made for managing the day-to-day cold chain or supply chain operations or for use in planning. In today’s environment, information must be readily available (online if possible), and in easy-to-use formats to facilitate rapid decision making on the part of immunization managers at all levels. Sharing standardized information in simple formats facilitates joint planning and management of supply and cold chain operations; reduction in risk of stock-outs, as well as loss/spoilage of the products; and ultimately improved immunization program management.

 

This toolkit module purports that immunization managers at all levels need data to support six classes of assessment related to vaccine supply chain and cold chain[3]:

  • Assessing if the cold chain management and infrastructure for routine immunization activities assure product quality and security, and meet planned delivery timelines;
  • Assessing the performance of cold chain equipment to determine if equipment is operating at the required temperature standards;
  • Assessing if cold chain MIS system provides information needed to determine when targeted supervision is needed;
  • Assessing whether the information collected on cold chain and supply chain operations the preparation of an annual Plan of Action; and
  • Assessing the required surge capacity for effectively responding to public health emergencies.

 

Each manager and his/her staff need to map out what are the routine decisions to be made, given the aforementioned classes of assessment, so that the information system can provide the required reports or data. Managers may have more classes of assessments, or may wish to breakdown the above classes of information into more discreet categories to ensure that all information required to support effective decision making is accounted for.

 

As immunization programs respond to demographic changes and incorporate new vaccines and technologies, the MIS and inventory management system (IMS) may need to be updated or even re-engineered. When such a milestone is reached, immunization managers at all levels need to convey to the responsible officer(s) the requirements for any new MIS or IMS to provide the reports, analytics and other data required for management purposes.

 

Given the need for readily available information for decision-making, as well as the expensiveness of many software applications for information systems, health ministries may consider migrating to cloud-based applications[4]. Cloud-based applications are among the cheapest MIS solutions, and rely on only an internet connection, which means they can be accessed from any device that is connected to the internet, without the use of any expensive software. Readers who are interested in cloud-based applications can access additional information using the link here. It is beyond the scope of this toolkit to discuss this topic in detail, but managers can read more online and potentially discuss this with their country’s authorities.

 

To assess if the current MIS/IMS can produce the required data, immunization managers should review the scope and quality of information in existing reports, and check whether it matches the information needed for decision-making. Critical questions to consider through this process include the following:

  • Do the stock levels and usage data from each service provider for each vaccine/syringe match up with the planned targeted populations to be vaccinated[5] for the time-period under study, given the planned vaccination tactics?
  • Do the stock levels for each vaccine and ancillary item available at each subnational warehouse/store match the usage data (demand) for time-period under study, and given the planned vaccination tactics?
  • Is data on the inventory of cold chain equipment and the functional status for each kind of equipment up-dated or collected within the last 3 months for each service provider at all levels?
  • Do the supervisory reports corroborate the data asked by questions 1 and 2 above?
  • Is there any critical information not collected through current reporting?
  • Are there any reports that do NOT provide information needed for decision-making, and could therefore be removed from the required list?

 

Next, data quality should be reviewed. After this, immunization managers should assess whether the required data and reports can be produced in a timely manner for management use.  Finally, immunization managers should ensure that staff at all levels have access to the same data and reports. This will streamline joint planning and decision-making within the system.

 

Management reports provide various measures on how the vaccine cold chain and supply chain "engine" is performing. These reports provide what business analysts and managers call key performance indicators, or measurable numbers that are used to determine business success and failure and can be presented in ‘dashboards’[6].  For further discussion on dashboards and their use, the reader can access the following articles using this link www.logianalytics.com/dashboard-best-practices-part-1.

 

The following illustrative checklist can be used to accomplish the above-mentioned tasks.

Illustrative Checklist for Assessing Data Quality:
What percent of the data on cold chain equipment is collected on standardized formats?
Is the data on cold chain equipment stored electronically?
Can all managers and staff with responsibility for cold chain and supply chain operations access and update this information as needed?
What percent of the collected data and information do supervisors for their respective geographic areas validate?

Do the supervisory field reports confirm the information provided by the MIS or from cold chain inventory information?

  • If this is not case, then further investigation is warranted to determine the quality of the data.

Has the data for cold chain equipment inventory in the MIS, been collected or updated within the last 3 months?

  • If not you should question the validity of the data. 
  • If this data is older than 12 months, one should determine the reasons for this situation.

If the answers to these questions do not present a positive outlook, the national immunization program manager should review with sub-national managers and key staff and define the problems and develop a requirement analysis[7] with alternatives, including an estimated cost to change/adapt the MIS to include the required information.

 

 

 

Cold Chain Equipment Management

In many countries, the cold chain infrastructure is made up of many types and models of equipment that were procured and or donated to the immunization program many years ago. In the private sector, the refrigerators found in the physician’s office for storage of vaccine are also of many makes and models. In addition, new refrigeration technology is being made available and many supply chain contractors are using them because they assure product security and quality. In short, a country’s cold chain can have a kaleidoscopic mixture of equipment.

 

Cold chain infrastructure is a critical determinant of the agility of a country’s vaccine supply chain operations. The storage capacity and location of each type of cold room, refrigerator, or freezer, as well as the reliability of fuel supplies, dictates how many doses of vaccine can be supplied to a given health facility or other destination in support of vaccination activities. The dry storage space available for safely receiving and storing ancillary supplies may also affect supply chain operations. Also, if power sources provide only intermittent power, managing the amount of vaccines that can flow through supply chain routes can become a very complicated task.

 

The authors of this section of the toolkit assume that most health clinics do not have the physical space to accommodate additional refrigeration equipment within their facility. If this is the case, managers are left with three options for expanding cold chain storage space.

  • Construct additional sub-national cold rooms to expand capacity close to each major supply route and/or service providers;
  • Outsource the additional cold chain storage requirements to the private sector; or
  • Procure additional equipment for storing vaccines in other nearby facilities that possess additional space to permit the placement of additional equipment.

 

The officer responsible for overseeing the provision of immunization services by private sector health providers should obtain information regarding the condition of their refrigeration equipment and vaccine storage conditions. This will require additional efforts, including carrying out phone surveys and in-person inspections of these providers’ facilities.

 

The above discussion provides a framework for conceptualizing how to manage cold chain equipment. The following are some of the major management considerations:

  • Ensure that all cold chain equipment, including passive containers that are selected for supporting the required field operations, meet the temperature and storage requirements of vaccines being distributed. This includes the ability to store required vaccines at each of their specified temperatures at health facilities as well as on distribution routes between the national level and their destinations (at health facilities or warehouses).
  • Develop a servicing plan to ensure that all cold chain equipment is serviced, repaired or replaced on time. This will ensure that that the provision of immunization services are not suspended because vaccines cannot be stored at the concerned service points or warehouses.
  • Prepare an annual Plan of Action (POA) with a line item budget for servicing, repairing, or replacing cold chain equipment, and for covering supply chain operations.
  • Integrate the flow of information on vaccine supply and cold chain operations with planned vaccine distribution tactics[8].
  • Ensure the security of vaccines and vaccine-related products during their storage and distribution.[9]
  • In advance of introducing newer and more expensive vaccines, evaluate current vaccine storage capacity available at the national and sub-national stores. Then determine whether any adjustments to cold chain infrastructure are necessary to accommodate the new vaccines.

 

The following Illustrative Checklist is suggested for use by the national immunization manager (or cold chain manager- whoever has responsibility for vaccine supply chain operations) to assist in planning for the introduction of new vaccines.

Illustrative Checklist: Planning Introduction of New Vaccines

Assess cost implications (in terms of cost per dose of vaccine) for different equipment accommodation options. Illustrative options may include:

  • Purchasing and installing more refrigeration equipment at key depots or stores
  • Executing smaller but more frequent shipments to stores, depots, or health facilities

Assess the implications on cold chain capacity by:

  • Cost of increased shipments to health service providers by geographic zone
     
  • Cost of fuel for increasing number of trips by type of route
     
  • Cost of adding new vehicles to handle the demand for additional shipments to meet the calendar of distribution for each state/departmental/regional supply chain operation
     
  • Cost of building new structures to house new cold rooms and refrigeration equipment
     
  • Cost of additional thermal containers and ice packs
     
  • Cost of additional human resources that may be required at key stores/depots

 

Immunization managers can use the WHO Logistics Forecasting Tool for carrying out scenario analysis to determine the implication on the supply chain and cold chain operations. Managers should ensure that information regarding vaccine characteristics, presentation, and temperature storage requirements are made available to all who are involved in managing vaccine supply chain and cold chain operations.

 

An essential part of managing vaccine supply chain and cold chain operations is establishing ‘cold chain and supply chain visibility’. This concept refers to having rapid access to reports and data on each piece or class of cold chain equipment, location, current operational status, the net volume of vaccine that can be stored in each piece of refrigeration equipment, and fuel source for operating the equipment. These reports maybe stored in a computer and printed on command, including using a dashboard to display the most important data or information for daily management use. Today, both public and private immunization services are storing new vaccines that are relatively expensive compared to traditional vaccines from 30 years ago. Many of these new vaccines are packaged in single dose presentations (some in prefilled syringes) that occupy much more refrigerator/freezer space. Therefore, to effectively manage the supply chain operations, assure product safety, and provide timely servicing to equipment that has stopped functioning correctly, rapidly available information is required for sound decision-making.

Illustrative Checklist: Evaluating the capability of the MIS to provide vaccine supply chain and cold chain visibility:

Is the data/information on each piece of cold chain equipment and its operational status linked to its location/health facility when entered into the MIS?

  • Does the report display the fuel source and the operational status for each piece of refrigeration by each health service provider (both private and public)?

Does each piece of equipment have a documented history of repairs made to it?

Was last year’s budget allocation sufficient to attend to repairs and maintenance for refrigeration equipment and auxiliary equipment[10]?

Does the MIS provide the following information?

  • Location of each health facility that will receive shipments of vaccine and ancillary supplies
     
  • Quantities for each type of vaccine and ancillary supplies that can be safely stored at each health facility for a time period of 30 days, given the target population[11] 
     
  • Identification of which health facilities do not have the storage capacity to store the volume of vaccine for 30 days, given their target population
     
  • The types of passive containers or cold boxes by: quantity (number of cold boxes and/or other temperature controlled containers), weight, and cubic size available at each health service provider facility/service point
     
  • Quantities for each type of vaccine and ancillary supplies that can be safely stored at each warehouse/depot for a time period of 60-90 days, given the number of health service providers assigned to each depot and the numbers of shipments for a specific period of time/calendar and the quantities of vaccine and ancillary requirements required for each service provider.
     
  • The types of passive containers or cold boxes by: quantity (number of cold boxes and/or other temperature controlled containers), weight, and cubic size available at each depot
     
  • Ice making capability: Kilograms/per 24hours period that can be frozen. This could also be stated as the number of ice –packs by volume: .05/1.0 liters

 

Maintaining the security of vaccines and vaccine-related products is a growing concern because of the increasing value of the vaccines. Security procedures and protocols need to be in place and reviewed for each warehouse or depot at all levels. Listings of authorized personnel should be given to those persons with responsibility for ensuring that only the individuals listed are provided access to the storage facilities. In areas where there is record of known break-ins to the medical/vaccine stores, managers need to assure that security guards are well selected and trained. Managers should regularly review security procedures with the office in charge to assure all is being done to protect the vaccine and ancillary supplies. The details (contracts, contacts and telephone numbers for the enterprise or government office that is responsible for providing security) should be entered into the MIS.

 

We have stated that the capacity, quality, and location of the components of the cold chain infrastructure is a major determinant of how well a vaccine supply chain functions. Changes in population demographics in each country will also have an impact on supply chain operations. The United Nations recently estimated that approximately 50% of world’s populations now live in urban areas.[12] Supply chain operations will need to be tailored according to changes in health facilities’ catchment areas, taking into account facilities’ storage capacity.

 

The strategic placement of sub-national warehouse or depots to accommodate the changing demands for vaccines and other supplies will add flexibility in the planning of operations. Figures 1 and 2 below illustrate this point. The cost, calculated in terms of cost per dose delivered, should be calculated for each potential scenario to determine the most cost-effective way of adjusting to changing target populations.

Figure 1.  Centralized distribution of vaccines from national level

[13]

 

Figure 1 demonstrates that all health service providers are dependent on efficient supply chain operations from the central level. This diagram also implies that the managers will need to plan for additional national level cold rooms as time goes by. This configuration of warehouses, depots, and storage facilities allows for less flexibility in adjusting for incoming shipment from vendors, compared to Figure 2.

 

Lessons learned: During the 2009 influenza pandemic a country could not receive its donation of the 2009 influenza pandemic vaccine because its national stores had just received a very large shipment of measles containing vaccine for its measles campaign that had been planned more than 12 months previous to the 2009 influenza pandemic.

Figure 2. Sub-national depots are in place, and permit larger store volumes to be located closer to the health service providers.

[14]

 

Figure 2 shows that all health service providers have closer access to their sub-national supplier or private sector warehouse. Therefore, with the building of sub-national stores or warehouses, supply lines are shortened, and less time is required for a facility to obtain a shipment. This configuration also demonstrates that the national level can increase total national storage capacity by having sub-national stores to store the required vaccines and thus spread the risk in relation to either mechanical breakdowns or energy failures occurring in just one place. For immunization managers, this demonstrates the value of building sub-national vaccine storage spaces.  This configuration of vaccine warehouses and storage units allows for greater management flexibility in managing and planning supply chain operations, compared with Figure 1’s configuration.

 

 

 

References


 

[1] The practice of giving employees personal digital assistants (PDAs) or letting them use their own personal device- better known as BYOD (bring your own device)- is permitting management and employees to make rapid decisions based on the exchange of data and information. Source: Eschelbeck, G. and Schwartzberg, D (2012). BYOD risks and rewards: how to keep employee smartphones, laptops and tablets secure. Retrieved from http://www.pspinfo.us/FileLibrary/Sophos_IM_BYOD_Risks-Rewards.pdf. 

[2] Companies that practice BYOD (Bring Your Own Device) allow employees to use their personal cell phones to conduct business on behalf of the company, and the company has an arrangement to reimburse calls and data usage relate their work.

[3] Each country will need to determine their own classes of decisions to be made, and assure that quality data is made available in the required timeframes.

[4] In common usage, the term "the cloud" is essentially a metaphor for the Internet. Cloud-based services are therefore platforms which rely only on an internet connection (i.e. do not rely on downloading hardware or software to a device). Typically, the seller has actual energy-consuming servers which host products and services from a remote location, so end-users don't have to; they can simply log on to the network without installing anything. The major models of cloud computing service are known as software as a service, platform as a service, and infrastructure as a service. Voorsluys, W.; Broberg, J.; Buyya, R. (February 2011). "Introduction to Cloud Computing" in Buyya, R.; Broberg, J.; Goscinski, A. Cloud Computing: Principles and Paradigms. New York, USA: Wiley Press. pp. 1-44.

[5] “Stock levels,” “usage data” and “targeted populations to be vaccinated” are basic metrics required to align a program's operations and assess the performance of cold chain and supply chain operations.

[6] Lewis, J. How dashboards are adapted for business (article from The Houston Chronicle). Retrieved from http://smallbusiness.chron.com/dashboards-used-business-32876.html

[7] Requirement analysis is the process of describing how users’ needs, in relation to the data collected, are to be analyzed, structured and specified for use in design and implementation of management actions to solve a problem or confirm that the program is being implemented according to standards or meeting the expected results. Miller, R. (2008) The difference between ga analysis and requirements analysis. Retrieved from SearchSoftwareQuailty.Com. 

[8]  The Toolkit defines vaccination tactics to include: provision of daily immunization services by health service providers (whether public or private); planned outreach activities, supplementary immunization activities, nationwide vaccination campaigns or days, and vaccination operations to control a disease outbreak.

[9] While thefts of vaccine and syringes are rare, they have occurred in some countries. With the introduction of new and more expensive vaccines, managers need to direct their attention to this issue.

[10] This could include: emergency electrical generators; forklifts; and/or spare evaporators for cold rooms.

[11] Each manager will need to decide the minimum number of  stock working days for each  vaccine that should be stored at each service point given the risks for power/energy outage, the terrain to be traversed and seasonal conditions of roads/rivers

[12] United Nations Population Division. (2014). Urban and Rural Areas 2014. Retrieved from http://esa.un.org/unpd/wup/wallcharts/WUP_2014%20Urban-Rural%20Areas%20W...

[13] Carrasco, P. (2013). Managing the cold chain (PowerPoint presentation) Delivered at Sabin Vaccine Institute Vaccinology Course, Peru. 

[14] Carrasco, P. (2013). Managing the cold chain (PowerPoint presentation) Delivered at Sabin Vaccine Institute Vaccinology Course, Peru.