---
tags: Archive
---

# 20210308 Bag-Valve-Mask (BVM) Performance Evaluation Report

Project repository: https://github.com/tgh-apil/BVM-Evaluation

Report (Source File): https://hackmd.io/@bag-valve-test/BJU82dW2P

Data collection: Monday February 4, 2021 at Toronto General Hospital, 200 Elizabeth St, Toronto, ON M5G 2C4

Reports & Revisions:
* Original report: February 22, 2021.
* [February 28, 2021](https://hackmd.io/@bag-valve-test/SJQGltAMd).
* [March 3, 2021](https://hackmd.io/@bag-valve-test/HJOq0eVQu) Incorporating corrections from Carestream related to ISO standard
* [March 8, 2021]

Kate Kazlovic, Vahid Anwari, Azad Mashari MD,
Department of Anesthesia and Pain Management, Toronto General Hospital

[TOC]

## Abstract
**Background** Bag-valve-mask (BVM) devices, also known as manual resuscitators, are critical equipment in acute care, and are used routinely for short term manual ventilation as well as for pre-oxygenation prior to short procedures (such as ECT) or endotracheal intubation and initiation of mechanical ventilation. Unanticipated results during an unrelated experiment raise concerns regarding the performance of BVM models in use at our center.

**Methods** We evaluated inspiratory resistance and competence of the inspiratory/expiratory valve systems in 3 models of BVM available at our hospital (3 samples/model). Inspiratory flow and resistance were measured with expiratory ports open vs. blocked. If device valves are competent the state of the expiratory port should have no impact on inspiratory flow resistance. Testing was consistent with procedure outlined in CSA-Z10651-4-08 (R2018): Lung ventilators — Part 4: Particular requirements for operator-powered resuscitators, section A.4.8 (CSA revision of ISO 10651-4:2002), with the addition of testing under blocked expiratory port condition that is not included in the CSA/ISO standard.

**Results** All samples from two models (Laerdal LSR, Ambu Spur II) showed anticipated behaviour with no effect of expiratory port blockage on inspiratory flow and resistance, though several of the samples from these two models had inspiratory pressure drops that measured as much as 10% above the maximal limit of 5 cm H~2~O prescribed by CSA-Z10651-4-08(R2018).

All three samples of the third model, CARE-BVM showed lower values of inspiratory pressure drop [mean(SD)= 3.0(0.26) cm H~2~O], but showed consistent and significant increases in inspiratory resistance when the expiratory port was blocked, with pressure drops of 7.8(0.72) cm H~2~O, significantly exceeding the 5 cm H~2~0 limit. This suggests significant entrainment of outside air during negative pressure inspiration, via a leaking expiratory valve. To confirm this we directly measured entrained air flow from the expiratory port. At 50 L/min of inspiratory flow **43-74% of inspiratory gas consisted of room air from the expiratory valve** instead of flowing through the self-inflating reservoir of the BVM. This corresponds to a calculated **delivered oxygen fraction of 40-60%** assuming a perfect face seal and 100% oxygen flow from the BVM reservoir.

**Conclusion** Our results raise a significant safety concern regarding the performance of CARE-BVM devices in spontaneously breathing patients suggesting that these patients may often be receiving a significantly lower concentration of oxygen than assumed by care providers. However, the technical standard (ISO/CSA-Z 10651-4-08) required by Health Canada and FDA does not explicitly mandate competence of the valves and only requires delivery of 35% oxygen at an external flow 15 L/min of oxygen flow into the device, meaning that the CARE-BVM devices appears to meet the explicit requirements regarding inspiratory resistance and oxygen provision.
**There is significant inconsistency between clinician expectations, device performance and regulatory requirements for BVM devices**.

## Background
Bag-valve-mask (BVM) devices are critical pieces in the respiratory support armament of modern health care, providing high concentration oxygen and potentially ventilatory support across a range of acute and intensive care settings. The ingenious combination of valves and reservoir chambers in BVMs allows them to 1) function with or without a pressurized gas source and 2) efficiently provide concentrations of inspired oxygen approaching 100% using low to moderate unpressurized gas flows.

During the COVID-19 pandemic and especially during the acute ventilator shortages of the first wave, the availability and versatility of BVMs led them to become the cornerstone of numerous strategies to provide invasive and non-invasive respiratory support for patients unable to access regular ventilators. The majority of emergency use ventilator initiatives relied on the use of reservoir and valve assemblies from BVMs to greatly simplify the design of emergency ventilators.

We set-out to evaluate the function of the inspiratory-expiratory control valves in three common models of bag-valve-mask (BVM) devices: Ambu Bag Spur II, CAREstream CARE-BVM and Laerdal LSR. This study was motivated by the unexpected finding during testing of another device, that the inspiratory resistance of CAREstream BVMs appeared to change significantly with blockage of the expiratory port.

In addition to providing a means of assessing the valve function of BVMs, the inspiratory flow resistance is clinically relevant in situations were the BVM is used to provide high oxygen concentrations to spontaneously breathing patients. This typically occurs during procedures (for example electroconvulsive therapy) or during management of patients in respiratory distress, where the BVM is often used for pre-oxygenation prior to induction of general anesthesia and endotracheal intubation. Higher inspiratory resistance in such situations can potentially lead to significant reductions in tidal volume and inspiratory flows, especially in patients in respiratory distress with limited ability to generate the negative pressures required to obtain adequate flows. This can be partly mitigated by manually supporting the patient by squeezing the bag during the patients spontaneous inspiratory effort.

The inspiratory resistance of a BVM device is typically quantified as the negative pressure required at the inspiratory port to generate specific flow rates. If the valves in the device are competent these values will be independent of any blockage of the expiratory port (by a PEEP valve for example).

We report bench-top measurements of the inspiratory resistance of three BVM models from different manufacturers with, and without blockage of the expiratory ports, in order to test the competence of the valves. We also measured flow through the expiratory port during continuous inspiratory flows of 25 and 50 L/min for BVMs that showed changes in inspiratory resistance with blockage of the expiratory port.

## Procedure
### Devices Tested
Three samples of each of the following devices were tested.
1. [Ambu Spur II Adult BVM Disposable Resuscitator (Ambu A/S Copenhagen, Denmark)](https://www.ambu.com/emergency-care-and-training/resuscitators/product/ambu-spur-ii)
* Sample 1 Lot#: 1000269921
* Sample 2 Lot#: N/A
* Sample 3 Lot#: N/A
2. [CAREstream CARE-BVM CS-100-A100-F-Univ Disposable Resuscitator (CAREstream Medical, Oakville, ON - Eastern Canada, Surrey, BC - Western Canada)](http://carestreammedical.com/product/care-bvm/)
* Sample 1 Lot#: 220070
* Sample 2 Lot#: 230070
* Sample 3 Lot#: 140059
3. [Laerdal Silicone Resuscitator (LSR; Laerdal Medical, Toronto, CA)](https://laerdal.com/products/medical-devices/airway-management/laerdal-silicone-resuscitator/)
* Sample 1 Lot#: 4512
* Sample 2 Lot#: 3509
* Sample 3 Lot#: 0915

The Spur II and CARE-BVM devices are disposable models. The Laerdal LSR is a reusable device. For the disposable device three previously unused samples were selected from three different lots. The LSR devices were reprocessed units used clinically at our hospital. Flow and pressure values were measured using [FluxMed (R) GrH](https://mbmed.com/fluxmed-grh/) Respiratory mechanics monitors ([MBMed, Buenos Aires, Argentina](https://mbmed.com)) (S/N: 2802020022; 28002019032)

***Figure 1: Test set-up.** Suction was connected directly to the patient port of the BVM device (without a face mask) and titrated to achieved target flow rates of 25 and 50 L/min. The negative pressure required to generate each target flow value was recorded in two conditions: expiratory port open or sealed. The CAREstream devices underwent additional testing with the secondary Flow/Pressure meter at the Expiratory port to quantify entrained flow through the faulty expiratory valves.*

Inspiratory flow resistance was measured in accordance with CSA-Z10651-4-08 (R2018): Lung ventilators — Part 4: Particular requirements for operator-powered resuscitators, section A.4.8 (CSA revision of ISO 10651-4:2002). The test set-up is shown in Figure 1. The BVM was mounted on a retort stand attached to a table. The oxygen inflow of the BVM was connected to a standard wall oxygen flow meter connection at 15 L/min. The inspiratory port of the BVM device was connected to the in-line respiratory monitor which was then connected to wall suction.

For each device and test condition (expiratory port open vs. blocked) suction pressure was titrated to achieve flows of approximately 25 and 50 l/min. The pressure and flow values were recorded at 256 Hz sampling rate and averaged over 120 seconds. Inspiratory resistance was calculated by dividing the pressure in cm H~2~O by the flow in L/min. For the sealed measurements, the expiratory port was fully sealed with polyethylene plastic food wrapping. The respiratory monitor was calibrated prior to the testing of each device and condition.

## Results
All results are in format mean(standard deviation or standard error) unless otherwise indicated. Units as follows: flow (L/min), pressure (cm H~2~O), resistance (cm H~2~O.min/L).

Figure 2 illustrates the change in the pressure-flow relationship with blockage of the expiratory port. The expected behaviour is for the open and blocked conditions to be identical. The LSR and Spur II models display the expected behaviour while the CARE-BVM shows a significant difference between the two conditions. With leak flow through the expiratory port blocked the CARE-BVM also has the highest inspiratory resistance of the three models tested. With the expiratory port open however, the CARE-BVM had the lowest inspiratory resistance while all 3 samples of the Laerdal LSR and 1 of the 3 Ambu Spur II devices exceeded (by a maximum of 10%) the inspiratory pressure drop limit of 5 cm H~2~O mandated by the ISO standard.

Table 1 presents the measured inspiratory flows and corresponding inspiratory pressures with the expiratory ports open and blocked, averaged across all 3 samples of each BVM model. Results are separated by target inspiratory flow rate (25 or 50 L/min). The absolute and percentage change in resistance with blockage of the expiratory port are presented in the right-most columns and in figure 3. The expected values is 0, that is blockage of the expiratory port should have no effect on the inspiratory flow resistance. While the Spur II and LSR devices behaved as expected the CARE-BVM devices showed an average resistance increase of over 300% at 25 L/min and almost 200% at 50 L/min. This behaviour was consistent in all three samples of the CARE-BVM tested. Results for individual samples are presented in Supplemental Table 1.

### Figure 2. Pressure drop vs. inspiratory flow with expiratory ports open or blocked
***Figure 2.** Pressure drop vs. inspiratory flow with expiratory ports open (dotted lines) or blocked (solid lines) in 3 models of BVM devices. Each colour represents a different model. The expected behaviour is for the two lines of each colour to be identical as they are for the LSR and Spur II models (that is, the state of the expiratory port should have no influence on the inspiratory resistance). The CARE-BVM device shows a marked effect or expiratory port closure.*

### Table 1. Inspiratory flow resistance change with blockage of the BVM expiratory port.
***Table 1.** Inspiratory flow resistance chance with blockage of the BVM expiratory port. Results by model and target flow.*

### Figure 3. Percent change in inspiratory resistance with blockage expiratory port.
***Figure 3.** Percent change in inspiratory resistance with of blockage expiratory port in 3 models of BVM devices at 25 and 50 l/min of simulated inspiratory flow. The expected value is 0 (that is, the state of the expiratory port should have no influence on the inspiratory resistance). The CARE-BVM device shows a marked effect or expiratory port closure.*

The above results suggest a failure of the expiratory valve in the CARE-BVM. To verify this we repeated the experiment in the CARE-BVM devices with a second flow meter at the expiratory port in order to measure any leak flow through the expiratory port during continuous simulated inspiration. The results are shown in Table 2. Separated for 25 L/min and 50 L/min target inspiratory flows. Across the 3 samples, leak flow through the expiratory port accounted for 43-73% of the inspiratory flow.

***Table 2.** Entrainment of outside air during inspiration via leaking expiratory port in CARE-BVM devices. Results shown for each of the 3 samples at target inspiratory flows of 25 and 50 L/min.*

Complete test data available at the [project repository](https://github.com/tgh-apil/BVM-Evaluation).

## Discussion
Our testing revealed consistent and significant failure of CARE-BVM devices consisting primarily of large leaks via the expiratory port with the likely consequence of dramatically lowering the oxygen concentration provided to patients. In addition the CARE-BVM devices had markedly higher inspiratory resistance (when expiratory leak was blocked) than the other models at 50 L/min.

Our testing also revealed inspiratory pressure drops exceeding 5 cm H~2~O in all samples of LSR [Range 5.2-5.5 cm H~2~O] and 1 of 3 samples of Spur II [5.5 cm H~2~O] devices tested.

In addition to providing short time, manual positive-pressure ventilation, bag-valve-mask devices are frequently used to deliver high concentrations of oxygen in order pre-oxygenate patients prior to procedures that induce transient apnea, such as electroconvulsive therapy or endotracheal intubation outside of the operating room. In these situations, patients are often breathing spontaneously with no manual positive-pressure generated by the provider. The international technical standard for BVM performance reference by most regulatory agencies is *ISO 10651-4-08: Lung ventilators — Part 4: Particular requirements for operator-powered resuscitators.* In the US this standard supplanted the American Society for Testing and Materials standard *ASTM F920-93(1999): Specification for Minimum Performance and Safety Requirements for Resuscitators Intended for Use With Humans* in 2007.

The current version of the standard form 2008 (reaffirmed by the Canadian Standards Association in 2018) has three features that **render it inadequate for ensuring effectiveness of the device for pre-oxygenation of spontaneously ventilating patients**. First neither the stated requirements nor the test procedures outlined in the standard ensure the competence of the expiratory valve during negative pressure inspiration. Second, the device is only required to deliver 35% oxygen with a supplemental oxygen flow of 15 l/min or less, with the possibility of providing 85% oxygen with the use of "an attachment"[Clause 6.1]. While the standard indicates acceptable upper bounds for inspiratory and expiratory resistance, it does not explicitly state that these values should be maintained in the presence of the attachment required to deliver 85% oxygen. As a results the CARE-BVM device's pressure drop of 7.8(0.72) cm H~2~O in the presence of a PEEP valve (that would effectively block the inspiratory leak through the expiratory port) is not a clear breach of the standard.

Our testing is limited by the sample size. Also we did not explicitly test leak flow through the expiratory port of the LSR and Spur II since their stable inspiratory resistance despite blockage of the expiratory port did not raise any suspicion of a leak. In addition the geometry of these devices made it very challenging to directly measure expiratory valve leak flow with our set-up.

## Conclusion
The current international technical standard used by health regulators in Canada, US and Europe does not ensure the effectiveness of these devices for pre-oxygenation of spontaneously ventilating patients. Specifically it does not require adequate oxygen delivery in the absence of attachments or prevent excessive inspiratory resistance at oxygen concentrations above 35%.

While some marketed devices exceed ISO requirements and are clinically acceptable for pre-oxygenation, care providers and institutions should assess these on a case by case basis. Regulatory approval conditional on satisfaction of requirements as set out by ISO 10651-4-08 does not guarantee adequate clinical performance for this use case.

## Supplementary Data
### Table S1: Inspiratory flow resistance by individual BVM samples
***Supplementary Table 1a.** Inspiratory flow resistance change with blockage of the BVM expiratory port. Results for individual samples.*

### Set-up
***Figure S1.** Inspiratory Resistance Set-up*

***Figure S2.** Set-up for measuring expiratory leak flow in CARE-BVM devices*

[**CARE-BVM Performance Test**](https://drive.google.com/file/d/1_ppU5NFp3fZgbe4sZXgNGGjswEY7Ntem/view?usp=sharing)

***Video:** CARE-BVM reservoir bag not deflating despite nearly 50 L/min of suction flow at the inspiratory port and only 15 L/min of oxygen line flow*

## Authors
Azad Mashari MD FRCPC
Assistant Professor, Department of Anesthesia and Pain Medicine, University of Toronto; Staff Anesthesiologist and Director of Lynn and Arnold Irwin Advanced Perioperative Imaging Lab, Department of Anesthesiology and Pain Management, Toronto General Hospital, University Health Network, Toronto, ON

Vahid Anwari, MRT\(R\), MSc Student,
Rehabilitation Science Institute, University of Toronto; Research Assistant, Lynn and Arnold Irwin Advanced Perioperative Imaging Lab, Toronto General Hospital, University Health Network, Toronto, ON

Kate Kazlovich, PhD Candidate,
Institute of Biomaterials and Biomedical Engineering, University of Toronto; Research Assistant, Lynn and Arnold Irwin Advanced Perioperative Imaging Lab, Toronto General Hospital, University Health Network, Toronto, ON

## Competing Interests
None.