An Open Test Bed for Medical Device Integration and Coordination Andrew King, Sam Procter†Dan Andresen, John Hatcliff‡, Steve WarrenKansas State UniversityWilliam Spees, Raoul JetleyPaul Jones, Sandy WeiningerUS Food & Drug ndy.Weiningeremail@example.comAbstractMedical devices historically have been monolithic units – developed, validated, and approved by regulatory authorities asstand-alone entities. Modern medical devices increasingly incorporate connectivity mechanisms that offer the potential to streamdevice data into electronic health records, integrate informationfrom multiple devices into single customizable displays, and coordinate the actions of groups of cooperating devices to realize“closed loop” scenarios and automate clinical workflows. However, it is not clear what middleware and integration architecturesmay be best suited for these possibly numerous scenarios. Moretroubling, current verification and validation techniques used inthe device industry are not targeted to assuring groups of integrated devices, and regulatory regimes have not yet been developed that allow manufacturers to bring systems of cooperatingdevices (each approved individually beforehand) to market.In this paper, we propose a publish-subscribe architecture formedical device integration based on the Java Messaging Service,and we report on our experience with this architecture in multiple scenarios that we believe represent the types of deploymentsthat will benefit from rapid device integration. This implementation and the experiments presented in this paper are offered as anopen test bed for exploring a multitude of development, qualityassurance, and regulatory issues related to medical device coordination.1IntroductionMost early medical devices were manually operated andincorporated electrical and mechanical control functionality. As medical-device complexity increased, analog circuitry was replaced or supplemented with digital technology, driving an increased reliance on embedded and highlevel software for the implementation of critical devicefunctionality. For example, early pacemakers includedaround 10,000 lines of code. Today, a modern pacemakerwill host over 100,000 lines of code. Despite the increas SAnToS Tech Report 2008-02. Submitted for publication. This material is based upon work supported by the National Science Foundationunder Grants # 0454348 and 0734204 and by the Air Force Office of Scientific Research.† Author’s current affiliation: University of Nebraska, Lincoln‡ Corresponding Author.ing focus on medical device software, much of this codeis still developed by engineers whose primary training isgeared toward low-level hardware and firmware development – few are trained in state of the art software development technologies or formal quality assurance techniques.Conversely, challenges of medical device development havenot been sufficiently exposed in the software engineeringcommunity, leading to a situation where innovations in software engineering and validation techniques are slow to beincorporated in medical device development. Additionally, device regulation approaches have been rigid by nature to promote patient safety, so regulation is naturally outpaced by rapid advances in software technology that can bequickly adopted as functional improvements in generic consumer electronics, broadening the gap between what is expected from devices and what can be realistically validated.Historically, medical devices have been developed asmonolithic stand-alone units. Current Verification and Validation (V&V) techniques used in industry primarily target single monolithic systems. Moreover, FDAs regulatoryregimes are designed to approve single stand-alone devices.Currently, there are no guidelines in place for how the industry might bring to market a framework that provides newand extensible clinical functionality (in essence, creating“virtual devices”) by utilizing an open system of cooperating medical device components from different vendors.This is driven by uncertainty regarding how one might regulate device collections when the full suite of device-deviceinteractions is not fully known a priori [21, 20].This state of affairs stands in direct contrast to the pervasive integration of and cooperation among computing devices in our world today, and it is quite clear that numerousclinical motivations exist to deploy systems of integratedand cooperating medical devices. Many devices marketedtoday already include some form of connectivity (serialports, Ethernet, 802.11 or Bluetooth wireless) – typicallyused to unidirectionally log data/events from these devices.However, it is anticipated that medical systems will undergoa paradigm shift as device integration moves well beyondsimple connectivity to provide functionality such as device
data streaming directly into patient electronic health records(EHRs), integration of information from multiple devicesin a clinical context (e.g., hospital room) into a single tailorable composite display, and automation of clinical workflows via computer systems that control networks of devicesas they perform cooperative tasks. Indeed, companies including Cerner, CapsuleTech, Philips/Emergin, Sensitron,and iSirona are bringing to market infrastructure that facilitates streaming of device data into medical records. In addition, large-scale research projects such as the Medical Device “Plug and Play” Interoperability Program , fundedby the U.S. Army’s Telemedicine and Advanced Technology Research Center (TATRC), are developing standardsand prototypes for systems of cooperating devices.In summary, the technology exists to assemble manyof types of medical systems that can substantially improvehealth care quality while lowering costs of medical care. Itis, in fact, so easy to establish ad hoc networks and integratedevices that companies are rapidly pushing integration solutions into market, and increasing numbers of clinical technicians are “rolling their own” device networks. This stateof-affairs is dangerous because the V & V technology andregulatory processes to guarantee the safety and security ofthese systems is lacking.Progress must be made on a number of fronts to addressthe challenges described above. Which middleware and integration architectures arecandidates to support device integration across multiple interaction scenarios? Which programming models are suitable for rapid development, validation, and certification of systems ofinteracting medical devices? What V & V techniques are appropriate for compositional verification of envisioned medical systems, andhow can the effectiveness of the techniques be demonstrated so as to encourage adoption among commercialvendors? Can existing regulatory guidelines and device approvalprocesses that target single devices be (a) extended toaccommodate component-wise approval of integratedsystems and (b) established in a manner that encourages innovation and rapid transition of new technologies into the market while upholding a mandate of approving safe and effective technologies? What interoperability and security standards are necessary to encourage development of commodity marketsfor devices, displays, EHR databases, and infrastructure that can support low cost deployment of integratesystems and enable flexible technology refresh?To facilitate industry, academic, and government exploration of these issues, we are developing an open Medical Device Coordination Framework (MDCF) for designing, implementing, verifying, and certifying systems ofintegrated medical devices. This paper reports on ourexperience of using publish-subscribe architectures andcomponent-based model-driven development along withstandards-compliant open-source code-bases in the initialdevelopment of this infrastructure. The specific contributions of this paper are as follows: We identify three clinical contexts in which device integration has the potential to be especially effective,and we summarize performance, development, andregulatory requirements of each. We propose a flexible publish-subscribe middlewarearchitecture based on the Java Messaging Service(JMS) to support exploration of issues surroundingsystems of integrated medical devices. We propose a model-based programming environmentfor rapid development of systems of integrated devices that we believe has the potential to support a newparadigm of regulatory oversight that can accommodate approval of device integration scenarios. We describe experiments that evaluate our infrastructure against the data formats, performance requirements, system functionality that we believe are representative of the requirements of envisioned clinicalcontexts for systems of integrated devices.We are submitting this paper to the ICSE Experience Trackbecause we seek to (a) directly engage the software engineering community with initial experience and challengeproblems associated with this emerging paradigm of medical systems and (b) overcome community barriers that havepreviously inhibited interactions between the software engineering researchers/practitioners, industrial medical devicedevelopers, and government regulatory authorities. Thecontents of this paper should not be interpreted as an endorsement by the FDA of any particular technology, software infrastructure, or direction for regulatory policy. However, we expect experience with frameworks like the onepresented here to provide science-based input to ongoingregulatory policy and standards development efforts. Theinfrastructure is available for public download at .2Clinical ContextsDevice integration/coordination can be beneficial in anumber of contexts. However, requirements on computational resources, performance, data rates, information availability, safety, and programmability vary greatly acrossthese contexts. In this section, we note the characteristicsof three clinical device integration contexts (CDICs) thatwe believe are especially relevant.2.1Room-oriented Device Information PresentationConcept: A typical hospital room in an intensive care wardhosts a number of stand-alone devices (see Figure 1). Manymodern rooms are integrated with an EHR database to logclinical activities, lab data, treatment plans, and information
Figure 1. Integrated device displayfor patient billing. Connections to drug dosing databasesmay also be available to facilitate correct drug dispensing.In such contexts, a number of factors reduce efficiency,degrade the quality of the patient’s encounter, and increaseerror likelihood. Each device in the room has its own userinterface, and these interfaces are non-uniform – potentiallyleading to mental overload and confusion on the part of thecaregiver. The devices are physically separated from eachother (e.g., on different sides of the bed), and a caregivermust step from one device to another to read device displays or change settings. Different interfaces in differentlocations make it more difficult for clinicians encounteringan alarm or some other critical event to rapidly gain situational awareness. Clinicians with different roles (e.g., thepatient’s doctor versus a nurse tasked with dispensing prescriptions) are likely to need different “views” or presentations of device information, and such reconfigurations mustbe carried out manually. Finally, each device provides datanecessary to assess a patient’s condition, but data are loggedseparately for each device, are not easily searchable, and donot always become part of the patient’s EHR.Integration Solution: Figure 1 presents a vision of howdevices and medical information systems can be integratedto address these issues. Several vendors including Capsule Tech, Cerner, LiveData are now marketing integrationframeworks that cover one or more aspects of this vision.In these integrated solutions, traditional medical devices areviewed as data producers that publish periodic or streamingdata to several types of data consumers.An EHR database serving as a data consumer allows information from individual devices to be integrated directlyinto the EHR. This aggregates device data into one place(simplifying record keeping), facilitates data searches (tospeed diagnoses and track device performance), and facilitates discovery and tracking of statistical trends or anomalies that would otherwise go unnoticed by clinicians.Another type of data consumer would be a single “headsup” display that takes information from multiple devicesand an EHR database (in this case, acting as a data producer) and formats it on one or more large monitors nearthe patient bed. For example, Cerner’s recently-releasedCareAware infrastructure uses IBM’s Eclipse framework toaggregate data from multiple devices onto such displays. InCareAware, the Eclipse “view” mechanism is used to define one or more data views from a particular device andeven to combine and integrate data coming from multipledevices . The Eclipse “perspective” mechanism is usedto define different collections of views that are tailored tothe concerns of different caregivers.Implementation Issues and Requirements: The deviceintegration scenario above must support different dataamounts and rates. For example, a pulse oximeter fingerclip may yield only heart rate and blood oxygen saturationvalues that are updated as individual parameters every 10seconds, yet an electronic stethoscope may need to streamdata at 8 kilosamples per second over the network. Thearchitecture must allow easy insertion of “data transformation” components that aggregate, filter, and transform data.The infrastructure must be supported by a programming andvalidation environment that (a) allows easy definition andcomposition of data producers, consumers, and transformers and (b) provides facilities for thorough validation andeasy auditing so as to support regulatory oversight.Deployment Strategies: This integration scenario may beimplemented using a dedicated server for each room, or bya collection of one or more servers that support integrationactivities across an entire ward or suite of rooms. Thus,any integration framework that aims to support this CDICshould be evaluated to determine if can give acceptable performance as the number of rooms scales when deployed ina server-for-multiple-rooms configuration.V&V and Regulatory Issues: When considering the viability of different architectures and deployment strategies,one must first assess conformance to basic safety, performance, and security requirements. In this context, safetyand performance issues include the following: Can the infrastructure transfer data from producers to consumers atrequired rates while avoiding unacceptable latencies and jitter? Are data out-of-range and network babbling scenariosproperly anticipated and addressed? Security issues centeraround the ability of the infrastructure to ensure that private data are unobservable and unalterable by unauthorizedparties. In a shared server scenario, the most likely sourceof safety and security threats would be interference betweenactivities in different rooms supported by the server or othernetwork nodes. For example, heightened activity in oneor more rooms might degrade the performance in anotherroom. Data transmitted from one room may be leaked orobserved by a party in another room. A server-per-roomstrategy would seem to have safety and security advantages(e.g., no interference between activities in different rooms)
but would be subject to more difficult and costly maintenance (e.g., more machines to maintain).As stated in Section 1, almost any device-integrationconsideration raises a number of regulatory issues due tothe fact that devices currently are approved by the FDA asstand-alone units. No claims are made about their potentialintegration with other devices. Device approval addressesa number of issues including assessment of human factorsin the reading and understanding of device displays. Sinceone of the primary motivations for this integration scenariois to redisplay information produced by a device on an integrated display, any such redisplay should be faithful to theprecision and presentation of data on the original device.Any filtering, transformation, or integration of data runs therisk of masking important data or alarms from the originaldevice or leading to clinicians interpreting the data redisplay in a manner that differs from that of the original devicedisplay. In fact, for transformations that radically alter theoriginal data, it is possible that FDA may interpret such a redisplay as a new device (albeit, a virtual device) that wouldbe subject to FDA regulatory oversight.2.2Alarm Integration and ForwardingConcept: Many medical monitoring devices produce alarmsignals when device data fall outside of a prespecified range.Proper implementation of device alarms and clinical workflows to monitor/respond to alarms is a significant concern.The IEC 60601-1-8 standard  lists a variety of requirements for alarm categories, priorities, auditory and visualalarm signals, and vendor documentation of alarm logic andsettings. It also defines the notion of a distributed alarm system that forwards alarm information for multiple devices toa central nurse station, database, or paging system.1To combat problems of alarm false positives (the presence of many false positives can cause providers to ignorealarms or disable them), some researchers and vendors areproducing “smart alarms” that use some form of sophisticated reasoning (e.g., fuzzy logic) to improve accuracy.Substantially increasing connectivity between devices andhealth information databases as in Section 2.1 makes it possible to design alarms that consider not just readings from asingle device but also readings from multiple devices or accumulated health histories. For example, rather than adopting a “one size fits all” approach to alarm threshold parameter settings, which may lead to false positives/negatives,an automatic threshold setting algorithm could consider thecurrent patient body type, weight, etc. as well as large collections of patient histories to provide more accurate settings. Utilizing multiple device integration, MDPnP hasconsidered simple use cases where alarms are only raisedif two different devices (e.g., pulse oximeter and a respira1 Philips/Emergin is a good example of a comprehensive distributedalarm system.tory monitor) both show alarm conditions .Integration Solution, Implementation, and Deployment:To support open experimentation with these emerging technologies, an integrated distributed alarm and device systemmust be able to forwarding alarm events that include information about the priority and source of the alarm (room,patient, and device) as well as information signals that provide data from monitoring devices (e.g., an ECG waveformindicating ventricular fibrillation). Other requirements aresimilar to the CDIC in Section 2.1. Transformations on datastreams must be easily programmable to support data/alarmcorrelation from multiple devices and access to health information databases.2.3Critical Care Device CoordinationConcept: The above CDICs consider unidirectional flowsof information from devices (restricted to producers of data)to displays or medical databases. Relaxing this restrictionto allow devices to be data/control consumers substantiallyincreases risks but can provide tremendous benefits including increased precision, removal of the potential of humanerror in tedious and repetitive tasks, reduction of time intime-sensitive tasks, and cost reductions. In system concepts currently envisioned, e.g., by MDPnP, risks and safetyconcerns are mitigated by having an automated or semiautomated agent control and supervise communication between devices, thus avoiding unexpected interactions thatmay arise in direct inter-device communications.Example: An integrated X-ray/ventilator demonstrationfrom MDPnP is an example of useful coordination that addresses the common problem of acquiring chest X-rays ofpatients on ventilators. To keep the lungs’ movements fromblurring the image, doctors must manually turn off the ventilator for a few seconds while they acquire the X-ray image,but there are risks in inadvertently leaving the ventilator offfor too long. These risks can be minimized by automatically coordinating the actions of the X-ray imaging deviceand the ventilator: specifically, the ventilator can identifywhen the lungs are at full inhalation or exhalation (and thusexperiencing minimal motion) so that the X-ray image canbe automatically captured at the optimal point in time .Various levels of automated feedback mechanisms thatclose the monitor-assess-treat loop are present in single devices today. Examples include so-called “smart pumps”that use monitored parameters (e.g., blood pressure, heartrate, and blood glucose level), to control fluid infusion (e.g.,insulin) into the body. A framework that supports controlled device coordination can expand closed-loop capabilities from functionality hard-coded into single devices upto flexible and extensible functionality that spans multipledevices. Smart pumps have been helpful to assure precision infusions that reduce harmful over- or under-dosing offluids. However, the executable behavior of smart pumps
Figure 2. Critical care device coordinationis relatively complex compared to most modern medicaldevices. Recalls on such devices have occurred due tosoftware-related problems. As closed loop concepts arescaled to multiple devices, frameworks like ours that facilitate open experimentation and assessment are crucial.Integration Solution: Figure 2 presents a vision of thefunctional capabilities of a critical care device coordinationframework. A collection of network-capable devices areconnected to a coordination framework. A device databaserecords the unique identifiers (e.g., based on MAC address)and device drivers for devices that have been pre-approvedfor connection to the framework. A database of availablecoordination behaviors holds scripts written by experts thatimplement approved coordination protocols. A clinicianthat desires a particular coordination behavior chooses anappropriate script from the database. Each script contains alist of device types needed to carry out a coordination activity. During the script execution start-up phase, the script interpretation framework attempts to acquire devices that arecurrently on the network and allocated to the current clinicalcontext (e.g., present in the current room). The clinician assists with device acquisition (e.g., by selecting from a list ofavailable devices of the appropriate type). After a completeset of required devices has been selected and confirmed asavailable (a device may be restricted from participating inmore than one script at a time if it is being subjected to automated control, while “read only” devices may be shared),the script interpreter begins execution of the coordinationscript. Script execution may proceed without intervention,or may stop to receive input from the clinician.The device coordination framework functionality that wehave sketched above is similar to functionality being proposed in the ICEMAN draft standard from the MDPnPgroup. Although we have not designed our framework todirectly implement that draft standard, one of the goals ofthe experiments reported in this paper is to illustrate thatpublish/subscribe architecture is capable of supporting thefunctionality required by the standard.Implementation Issues and Requirements: The CDIC ofFigure 2 requires all of the functionality of the Device Information Integration CDIC of Figure 1. In addition, thecomponent categories (data producer, consumer, and transformer) must be enhanced to include coordination components that can be thought of as simple automata with I/O thatinteract with devices and clinicians. The notion of a devicecomponent must also be enhanced to include interface portsthat provide data input to or control of devices.Due to the increased levels of criticality associated withdevice coordination (as compared to the earlier scenariofocusing on device data presentation), the associated programming and validation environment must be able to support more rigorous notions of specification and validationfor greater regulatory oversight.Deployment Scenarios: In contrast to the CDICs that included possible deployment configurations in which sharedservers supported pub/sub activities across multiple hospitalrooms, we believe that, for this highly safety-critical context, it is important to reduce the risk of unanticipated andpotentially harmful interference by allowing a server to support only a single room at a time. Improvements in verification technology that can provide high confidence of noninterference between multiple coordination activities mayeventually lead to this restriction being lifted (in fact, onereason for our work is to provide a test bed that facilitiesresearch on and demonstration of such technology). However, for the present, the risks of potential harmful interference are best mitigated by using separate dedicated serversfor each coordination activity.3GoalsBelow we list the design goals for our MDCF.1. Provide distributed networking middleware infrastructure that enables devices/displays/databases from different vendors to be integrated with minimal effort.2. Provide payload capabilities that support common dataformats used in the medical device and medical informatics domains.3. Provide an architecture that enables tailoring, integrating, and transforming device information streams.4. Support the requirements of realistic device integrationcontexts as described in Section 2.5. Develop an architecture whose performance andapplication-level programmability scales gracefully asthe number of integrated devices and computational resources (e.g., server machines) increases.6. Provide basic functionality needed for reliability including options for guaranteed message delivery, logging/auditing, and persistent storage of messages.7. Support a script programming model that makes it easyto assemble new functionality from building blocks.8. Use infrastructure that is freely available and opensource (to enable academic research).
Figure 3. JMS primary objects9. Use standards-based frameworks that are supported byenterprise-level implementations that can provide suitable performance in a realistic enterprise setting.10. Because it will be difficult for academics to obtain realdevices, support both real and simulated devices.44.1ArchitectureMOM FoundationThe design of our core architecture is driven by practicalrealities of the clinical device integration contexts in Section 2, such as (a) flexible, dynamic information flow (frequently needing privacy), (b) heterogeneous systems, mechanisms, and needs, (c) many listeners, and many sources,and (d) time-critical, scalable performance. A messageoriented, publish-subscribe architecture with decentralizedhubs, dynamic queuing, reliable message passing, andenterprise-grade deployment fits these criteria nicely. Thus,we chose to build upon a messaging-oriented-middleware(MOM) foundation. To address the goals of Section 3, webelieve it is best to consider MOMs based on the Java Message Service (JMS) standard. JMS satisfies the criteria (a-d)above, while providing low-cost, open-source implementations for low barriers to entry and easy integration into research environments (Goal 8). In addition, there are multiple commercial enterprise-quality JMS implementationssuch as those found in IBM’s WebSphere and Oracle’s AQproducts (Goal 9). JMS provides point-to-point or publish/subscribe topologies (addressing Goals 1 and 3), reliable or unreliable message delivery (Goal 6), and high performance (Goal 9). It enables distributed communicationwhich is “loosely coupled, reliable, and asynchronous.” Inour application environment, its ability to pass simple datatypes as well as complex objects enables a clean integration with structured text standards such as HL7, as well ascomplex objects for seamless framework control (Goal 2).Figure 3 presents the primary objects involved in JMSpublish/subscribe communication. When a client wishesto originate a connection with a JMS provider, it uses theJava Naming and Directory Interface (JNDI) to locate aConnection Factory that encapsulates a set of connectionconfiguration parameters for the provider. The client thenFigure 4. JMS destinationsuses the Connection Factory to create an active Connectionto the provider (typically represented as an open TCP/IPsocket between the client and the providers service daemon). In our architecture, clients will do all of their messaging with a single Connection. A Connection supportsan Exception Listener that will be called when an connection fails (which we will use to handle situations in whicha device unexpectedly disconnects in the middle of an activity). Once a connection is established, a client uses theconnection to create a JMS Session.Figure 4 illustrates that a JMS destination is an abstractentity to/from which a client publishes or receives a message. Destinations are located/retrieved via JNDI calls. Asession serves as a factory for creating MessageProducersor MessageConsumers for a particular destination. To senda message, a client requests a session to create an emptymessage (of a particular type supported by JMS), the message contents are filled in, and a MessageProducer is calledto send the message. To receive messages asynchronously(which is the method we will use in our framework), theclient creates an object (a handler) that implements the MessageListener interface and sets that object as the listener fora particular MessageConsumer.A session is a single-threaded context designed for serial use by one thread at a time. It conceptually providesa thread for sending an
Historically, medical devices have been developed as monolithic stand-alone units. Current Veriﬁcation and Val-idation (V&V) techniques used in industry primarily tar-get single monolithic systems. Moreover, FDAs regulatory regimes are designed to approve single stand-alone devices. Currently, there are no guidelines in place for how the in-