Weimer, James
Email Address
ORCID
Disciplines
Search Results
Now showing 1 - 10 of 17
Publication Verifying the Safety of Autonomous Systems with Neural Network Controllers(2020-12-01) Ivanov, Radoslav; Carpenter, Taylor J.; Weimer, James; Alur, Rajeev; Pappas, George; Lee, Insup; Ivanov, Radoslav; Carpenter, Taylor J.; Weimer, James; Alur, Rajeev; Pappas, George; Lee, InsupThis paper addresses the problem of verifying the safety of autonomous systems with neural network (NN) controllers. We focus on NNs with sigmoid/tanh activations and use the fact that the sigmoid/tanh is the solution to a quadratic differential equation. This allows us to convert the NN into an equivalent hybrid system and cast the problem as a hybrid system verification problem, which can be solved by existing tools. Furthermore, we improve the scalability of the proposed method by approximating the sigmoid with a Taylor series with worst-case error bounds. Finally, we provide an evaluation over four benchmarks, including comparisons with alternative approaches based on mixed integer linear programming as well as on star sets.Publication Calibrated Prediction with Covariate Shift via Unsupervised Domain Adaptation(2020-03-01) Park, Sangdon; Bastani, Osbert; Weimer, James; Lee, Insup; Park, Sangdon; Bastani, Osbert; Weimer, James; Lee, InsupReliable uncertainty estimates are an important tool for helping autonomous agents or human decision makers understand and leverage predictive models. However, existing approaches to estimating uncertainty largely ignore the possibility of covariate shift—i.e., where the real-world data distribution may differ from the training distribution. As a consequence, existing algorithms can overestimate certainty, possibly yielding a false sense of confidence in the predictive model. We propose an algorithm for calibrating predictions that accounts for the possibility of covariate shift, given labeled examples from the training distribution and unlabeled examples from the real-world distribution. Our algorithm uses importance weighting to correct for the shift from the training to the real-world distribution. However, importance weighting relies on the training and real-world distributions to be sufficiently close. Building on ideas from domain adaptation, we additionally learn a feature map that tries to equalize these two distributions. In an empirical evaluation, we show that our proposed approach outperforms existing approaches to calibrated prediction when there is covariate shift.Publication Computer Aided Clinical Trials for Implantable Cardiac Devices(2018-07-12) Jang, Kuk Jin; Weimer, James; Abbas, Houssam; Jiang, Zhihao; Liang, Jackson; Dixit, Sanjay; Mangharam, Rahul; Jang, Kuk Jin; Weimer, James; Abbas, Houssam; Jiang, Zhihao; Liang, Jackson; Dixit, Sanjay; Mangharam, RahulIn this paper we aim to answer the question, ``How can modeling and simulation of physiological systems be used to evaluate life-critical implantable medical devices?'' Clinical trials for medical devices are becoming increasingly inefficient as they take several years to conduct, at very high cost and suffer from high rates of failure. For example, the Rhythm ID Goes Head-to-head Trial (RIGHT) sought to evaluate the performance of two arrhythmia discriminator algorithms for implantable cardioverter defibrillators, Vitality 2 vs. Medtronic, in terms of time-to-first inappropriate therapy, but concluded with results contrary to the initial hypothesis - after 5 years, 2,000+ patients and at considerable ethical and monetary cost. In this paper, we describe the design and performance of a computer-aided clinical trial (CACT) for Implantable Cardiac Devices where previous trial information, real patient data and closed-loop device models are effectively used to evaluate the trial with high confidence. We formulate the CACT in the context of RIGHT using a Bayesian statistical framework. We define a hierarchical model of the virtual cohort generated from a physiological model which captures the uncertainty in the parameters and allows for the systematic incorporation of information available at the design of the trial. With this formulation, the CACT estimates the inappropriate therapy rate of Vitality 2 compared to Medtronic as 33.22% vs 15.62% (pPublication Willingness to Use a Wearable Device Capable of Detecting and Reversing Overdose Among People Who Use Opioids in Philadelphia(2021-07-01) Kanter, Katie; Gallagher, Ryan; Eweje, Feyisope; Lee, Alexander; Gordon, David; Landy, Stephen; Gasior, Julia; Soto-Calderon, Haideliza; Weimer, James; Cocchiaro, Ben; Weimer, James; Brenner, Jacob; Lankenau, Stephen; Brenner, JacobBackground: The incidence of opioid-related overdose deaths has been rising for 30 years and has been further exacerbated amidst the COVID-19 pandemic. Naloxone can reverse opioid overdose, lower death rates, and enable a transition to medication for opioid use disorder. Though current formulations for community use of naloxone have been shown to be safe and effective public health interventions, they rely on bystander presence. We sought to understand the preferences and minimum necessary conditions for wearing a device capable of sensing and reversing opioid overdose among people who regularly use opioids. Methods: We conducted a combined cross-sectional survey and semi-structured interview at a respite center, shelter, and syringe exchange drop-in program in Philadelphia, Pennsylvania, USA during the COVID-19 pandemic in August and September 2020. The primary aim was to explore the proportion of participants who would use a wearable device to detect and reverse overdose. Preferences regarding designs and functionalities were collected via a questionnaire with items having Likert-based response options and a semi-structured interview intended to elicit feedback on prototype designs. Independent variables included demographics, opioid use habits, and previous experience with overdose. Results: A total of 97 adults with an opioid-use history of at least 3 months were interviewed. A majority of survey participants (76%) reported a willingness to use a device capable of detecting an overdose and automatically administering a reversal agent upon initial survey. When reflecting on the prototype, most respondents (75.5%) reported that they would wear the device always or most of the time. Respondents indicated discreetness and comfort as important factors that increased their chance of uptake. Respondents suggested that people experiencing homelessness and those with low tolerance for opioids would be in greatest need of the device. Conclusions: The majority of people sampled with a history of opioid use in an urban setting were interested in having access to a device capable of detecting and reversing an opioid overdose. Participants emphasized privacy and comfort as the most important factors influencing their willingness to use such a device. Trial Registration: NCT04530591Publication OpenICE-lite: Towards a Connectivity Platform for the Internet of Medical Things(2018-05-01) Ivanov, Radoslav; Nguyen, Hung; Weimer, James; Sokolsky, Oleg; Lee, Insup; Ivanov, Radoslav; Nguyen, Hung; Weimer, James; Sokolsky, Oleg; Lee, InsupThe Internet of Medical Things (IoMT) is poised to revolutionize medicine. However, medical device communication, coordination, and interoperability present challenges for IoMT applications due to safety, security, and privacy concerns. These challenges can be addressed by developing an open platform for IoMT that can provide guarantees on safety, security and privacy. As a first step, we introduce OpenICE-lite, a middleware for medical device interoperability that also provides security guarantees and allows other IoMT applications to view/analyze the data in real time. We describe two applications that currently utilize OpenICE-lite, namely (i) a critical pulmonary shunt predictor for infants during surgery; (ii) a remote pulmonary monitoring systems (RePulmo). Implementations of both systems are utilized by the Children’s Hospital of Philadelphia (CHOP) as quality improvements to patient care.Publication Resilient Linear Classification: An Approach to Deal with Attacks on Training Data(2017-04-01) Park, Sangdon; Weimer, James; Lee, Insup; Park, Sangdon; Weimer, James; Lee, InsupData-driven techniques are used in cyber-physical systems (CPS) for controlling autonomous vehicles, handling demand responses for energy management, and modeling human physiology for medical devices. These data-driven techniques extract models from training data, where their performance is often analyzed with respect to random errors in the training data. However, if the training data is maliciously altered by attackers, the effect of these attacks on the learning algorithms underpinning data-driven CPS have yet to be considered. In this paper, we analyze the resilience of classification algorithms to training data attacks. Specifically, a generic metric is proposed that is tailored to measure resilience of classification algorithms with respect to worst-case tampering of the training data. Using the metric, we show that traditional linear classification algorithms are resilient under restricted conditions. To overcome these limitations, we propose a linear classification algorithm with a majority constraint and prove that it is strictly more resilient than the traditional algorithms. Evaluations on both synthetic data and a real-world retrospective arrhythmia medical case-study show that the traditional algorithms are vulnerable to tampered training data, whereas the proposed algorithm is more resilient (as measured by worst-case tampering).Publication Cloud-Based Secure Logger for Medical Devices(2016-06-01) Nguyen, Hung; Ivanov, Radoslav; Haeberlen, Andreas; Phan, Linh T.X.; Sokolsky, Oleg; Weimer, James; Hanson III, C. William; Lee, Insup; Ivanov, Radoslav; Haeberlen, Andreas; Phan, Linh T.X.; Sokolsky, Oleg; Walker, Jesse; Weimer, James; Hanson III, C. William; Lee, InsupA logger in the cloud capable of keeping a secure, time-synchronized and tamper-evident log of medical device and patient information allows efficient forensic analysis in cases of adverse events or attacks on interoperable medical devices. A secure logger as such must meet requirements of confidentiality and integrity of message logs and provide tamper-detection and tamper-evidence. In this paper, we propose a design for such a cloud-based secure logger using the Intel Software Guard Extensions (SGX) and the Trusted Platform Module (TPM). The proposed logger receives medical device information from a dongle attached to a medical device. The logger relies on SGX, TPM and standard encryption to maintain a secure communication channel even on an untrusted network and operating system. We also show that the logger is resilient against different kinds of attacks such as Replay attacks, Injection attacks and Eavesdropping attacks.Publication Verisig: verifying safety properties of hybrid systems with neural network controllers(2019-04-01) Ivanov, Radoslav; Weimer, James; Alur, Rajeev; Pappas, George J.; Lee, Insup; Ivanov, Radoslav; Weimer, James; Alur, Rajeev; Pappas, George J.; Lee, InsupThis paper presents Verisig, a hybrid system approach to verifying safety properties of closed-loop systems using neural networks as controllers. We focus on sigmoid-based networks and exploit the fact that the sigmoid is the solution to a quadratic differential equation, which allows us to transform the neural network into an equivalent hybrid system. By composing the network’s hybrid system with the plant’s, we transform the problem into a hybrid system verification problem which can be solved using state-of-theart reachability tools. We show that reachability is decidable for networks with one hidden layer and decidable for general networks if Schanuel’s conjecture is true. We evaluate the applicability and scalability of Verisig in two case studies, one from reinforcement learning and one in which the neural network is used to approximate a model predictive controller.Publication Case Study: Verifying the Safety of an Autonomous Racing Car with a Neural Network Controller(2020-04-01) Ivanov, Radoslav; Carpenter, Taylor J.; Weimer, James; Alur, Rajeev; Pappas, George; Lee, Insup; Ivanov, Radoslav; Carpenter, Taylor J.; Weimer, James; Alur, Rajeev; Pappas, George; Lee, InsupThis paper describes a verification case study on an autonomous racing car with a neural network (NN) controller. Although several verification approaches have been recently proposed, they have only been evaluated on low-dimensional systems or systems with constrained environments. To explore the limits of existing approaches, we present a challenging benchmark in which the NN takes raw LiDAR measurements as input and outputs steering for the car. We train a dozen NNs using reinforcement learning (RL) and show that the state of the art in verification can handle systems with around 40 LiDAR rays. Furthermore, we perform real experiments to investigate the benefits and limitations of verification with respect to the sim2real gap, i.e., the difference between a system’s modeled and real performance. We identify cases, similar to the modeled environment, in which verification is strongly correlated with safe behavior. Finally, we illustrate LiDAR fault patterns that can be used to develop robust and safe RL algorithms.Publication Parameter-Invariant Monitor Design for Cyber Physical Systems(2018-01-01) Weimer, James; Ivanov, Radoslav; Chen, Sanjian; Roederer, Alexander; Sokolsky, Oleg; Lee, Insup; Weimer, James; Ivanov, Radoslav; Chen, Sanjian; Roederer, Alexander; Sokolsky, Oleg; Lee, InsupThe tight interaction between information technology and the physical world inherent in Cyber-Physical Systems (CPS) can challenge traditional approaches for monitoring safety and security. Data collected for robust CPS monitoring is often sparse and may lack rich training data describing critical events/attacks. Moreover, CPS often operate in diverse environments that can have significant inter/intra-system variability. Furthermore, CPS monitors that are not robust to data sparsity and inter/intra-system variability may result in inconsistent performance and may not be trusted for monitoring safety and security. Towards overcoming these challenges, this paper presents recent work on the design of parameter-invariant (PAIN) monitors for CPS. PAIN monitors are designed such that unknown events and system variability minimally affect the monitor performance. This work describes how PAIN designs can achieve a constant false alarm rate (CFAR) in the presence of data sparsity and intra/inter system variance in real-world CPS. To demonstrate the design of PAIN monitors for safety monitoring in CPS with different types of dynamics, we consider systems with networked dynamics, linear-time invariant dynamics, and hybrid dynamics that are discussed through case studies for building actuator fault detection, meal detection in type I diabetes, and detecting hypoxia caused by pulmonary shunts in infants. In all applications, the PAIN monitor is shown to have (significantly) less variance in monitoring performance and (often) outperforms other competing approaches in the literature. Finally, an initial application of PAIN monitoring for CPS security is presented along with challenges and research directions for future security monitoring deployments.