Weimer, James

Email Address
Research Projects
Organizational Units
Research Interests

Search Results

Now showing 1 - 10 of 23
  • Publication
    Towards Assurance Cases for Resilient Control Systems
    (2014-08-01) Weimer, James; Sokolsky, Oleg; Bezzo, Nicola; Lee, Insup
    The paper studies the problem of constructing assurance cases for embedded control systems developed using a model-based approach. Assurance cases aim to provide a convincing argument that the system delivers certain guarantees, based on the evidence obtained during the design and evaluation of the system. We suggest an argument strategy centered around properties of models used in the development and properties of tools that manipulate these models. The paper presents the case study of a resilient speed estimator for an autonomous ground vehicle and takes the reader through a detailed assurance case arguing that the estimator computes speed estimates with bounded error.
  • Publication
    Estimation of Blood Oxygen Content Using Context-Aware Filtering
    (2016-04-01) Ivanov, Radoslav; Atanasov, Nikolay; Weimer, James; Simpao, Allan F; Rehman, Mohamed A; Pappas, George; Lee, Insup; Pajic, Miroslav
    In this paper we address the problem of estimating the blood oxygen concentration in children during surgery.Currently, the oxygen content can only be measured through invasive means such as drawing blood from the patient. In this work, we attempt to perform estimation by only using other non-invasive measurements (e.g., fraction of oxygen in inspired air, volume of inspired air) collected during surgery. Although models mapping these measurements to blood oxygen content contain multiple parameters that vary widely across patients, the non-invasive measurements can be used to provide binary information about whether the oxygen concentration is rising or dropping. This information can then be incorporated in a context-aware filter that is used to combine regular continuous measurements with discrete detection events in order to improve estimation. We evaluate the filter using real-patient data collected over the last decade at the Children’s Hospital of Philadelphia and show that it is a promising approach for the estimation of unobservable physiological variables.
  • Publication
    Towards a Model-Based Meal Detector for Type I Diabetics
    (2015-04-13) Chen, Sanjian; Weimer, James; Rickels, Michael R.; Peleckis, Amy; Lee, Insup
    Blood glucose management systems are an important class of Medical Cyber-Physical Systems that provide vital everyday decision support service to diabetics. An artificial pancreas, which integrates a continuous glucose monitor, a wearable insulin pump, and control algorithms running on embedded computing devices, can significantly improve the quality of life for millions of Type 1 diabetics. A primary problem in the development of an artificial pancreas is the accurate detection and estimation of meal carbohydrates, which cause significant glucose system disturbances. Meal carbohydrate detection is challenging since post-meal glucose responses greatly depend on patient-specific physiology and meal composition. In this paper, we develop a novel meal-time detector that leverages a linearized physiological model to realize a (nearly) constant false alarm rate (CFAR) performance despite unknown model parameters and uncertain meal inputs. Insilico evaluations using 10, 000 virtual subjects on an FDA-accepted maximal physiological model illustrate that the proposed CFAR meal detector significantly outperforms a current state-of-the-art meal detector that utilizes a voting scheme based on rate-of-change (RoC) measures. The proposed detector achieves 99.6% correct detection rate while averaging one false alarm every 24 days (a 1.4% false alarm rate), which represents an 84% reduction in false alarms and a 95% reduction in missed alarms when compared to the RoC approach.
  • Publication
    Improving Classifier Confidence using Lossy Label-Invariant Transformations
    (2021-04-01) Jang, Sooyong; Lee, Insup; Weimer, James
    Providing reliable model uncertainty estimates is imperative to enabling robust decision making by autonomous agents and humans alike. While recently there have been significant advances in confidence calibration for trained models, examples with poor calibration persist in most calibrated models. Consequently, multiple techniques have been proposed that leverage label-invariant transformations of the input (i.e., an input manifold) to improve worst-case confidence calibration. However, manifold-based confidence calibration techniques generally do not scale and/or require expensive retraining when applied to models with large input spaces (e.g., ImageNet). In this paper, we present the recursive lossy label-invariant calibration (ReCal) technique that leverages label-invariant transformations of the input that induce a loss of discriminatory information to recursively group (and calibrate) inputs – without requiring model retraining. We show that ReCal outperforms other calibration methods on multiple datasets, especially, on large-scale datasets such as ImageNet.
  • Publication
    ModelGuard: Runtime Validation of Lipschitz-continuous Models
    (2021-07-01) Carpenter, Taylor J.; Ivanov, Radoslav; Lee, Insup; Weimer, James
    This paper presents ModelGuard, a sampling-based approach to runtime model validation for Lipschitz-continuous models. Although techniques exist for the validation of many classes of models, the majority of these methods cannot be applied to the whole of Lipschitz-continuous models, which includes neural network models. Additionally, existing techniques generally consider only white-box models. By taking a sampling-based approach, we can address black-box models, represented only by an input-output relationship and a Lipschitz constant. We show that by randomly sampling from a parameter space and evaluating the model, it is possible to guarantee the correctness of traces labeled consistent and provide a confidence on the correctness of traces labeled inconsistent. We evaluate the applicability and scalability of ModelGuard in three case studies, including a physical platform.
  • Publication
    An Intraoperative Glucose Control Benchmark for Formal Verification
    (2015-10-01) Chen, Sanjian; O'Kelly, Matthew; Weimer, James; Sokolsky, Oleg; Lee, Insup
    Diabetes associated complications are affecting an increasingly large population of hospitalized patients. Since glucose physiology is significantly impacted by patient-specific parameters, it is critical to verify that a clinical glucose control protocol is safe across a wide patient population. A safe protocol should not drive the glucose level into dangerous low (hypoglycemia) or high (hyperglycemia) ranges. Verification of glucose controllers is challenging due to the high-dimensional, non-linear glucose physiological models which contain both unobservable states and unmeasurable patient-specific parameters. This paper presents a hybrid system model of a closed-loop physiological system that includes an existing FDA-accepted high-fidelity physiological model tailored to intraoperative settings and a validated improvement to a clinical glucose control protocol for diabetic cardiac surgery patients. We propose the closed-loop model as a physiological system benchmark for verification and present our initial results on verifying the system using the SMT-based hybrid system verification tool dReach.
  • Publication
    Towards Synthesis of Platform-Aware Attack-Resilient Control Systems: Extended Abstract
    (2013-04-09) Pajic, Miroslav; Bezzo, Nicola; Weimer, James; Alur, Rajeev; Mangharam, Rahul; Michael, Nathan; Pappas, George J; Sokolsky, Oleg; Tabuada, Paulo; Weirich, Stephanie; Lee, Insup
  • Publication
    Data-driven Adaptive Safety Monitoring using Virtual Subjects in Medical Cyber-Physical Systems: A Glucose Control Case Study
    (2016-09-01) Chen, Sanjian; Sokolsky, Oleg; Weimer, James; Lee, Insup
    Medical cyber-physical systems (MCPS) integrate sensors, actuators, and software to improve patient safety and quality of healthcare. These systems introduce major challenges to safety analysis because the patient’s physiology is complex, nonlinear, unobservable, and uncertain. To cope with the challenge that unidentified physiological parameters may exhibit short-term variances in certain clinical scenarios, we propose a novel run-time predictive safety monitoring technique that leverages a maximal model coupled with online training of a computational virtual subject (CVS) set. The proposed monitor predicts safety-critical events at run-time using only clinically available measurements. We apply the technique to a surgical glucose control case study. Evaluation on retrospective real clinical data shows that the algorithm achieves 96% sensitivity with a low average false alarm rate of 0.5 false alarm per surgery.
  • Publication
    Parameter Invariant Monitoring for Signal Temporal Logic
    (2018-04-01) Roohi, Nima; Kaur, Ramneet; Weimer, James; Sokolsky, Oleg; Lee, Insup
    Signal Temporal Logic (STL) is a prominent specification formalism for real-time systems, and monitoring these specifications, specially when (for different reasons such as learning) behavior of systems can change over time, is quite important. There are three main challenges in this area: (1) full observation of system state is not possible due to noise or nuisance parameters, (2) the whole execution is not available during the monitoring, and (3) computational complexity of monitoring continuous time signals is very high. Although, each of these challenges has been addressed by different works, to the best of our knowledge, no one has addressed them all together. In this paper, we show how to extend any parameter invariant test procedure for single points in time to a parameter invariant test procedure for efficiently monitoring continuous time executions of a system against STL properties. We also show, how to extend probabilistic error guarantee of the input test procedure to a probabilistic error guarantee for the constructed test procedure.
  • Publication
    Verisig 2.0: Verification of Neural Network Controllers Using Taylor Model Preconditioning
    (2021-07-01) Ivanov, Radoslav; Carpenter, Taylor; Weimer, James; Alur, Rajeev; Pappas, George; Lee, Insup
    This paper presents Verisig 2.0, a verification tool for closed-loop systems with neural network (NN) controllers. We focus on NNs with tanh/sigmoid activations and develop a Taylor-model-based reachability algorithm through Taylor model preconditioning and shrink wrapping. Furthermore, we provide a parallelized implementation that allows Verisig 2.0 to efficiently handle larger NNs than existing tools can. We provide an extensive evaluation over 10 benchmarks and compare Verisig 2.0 against three state-of-the-art verification tools. We show that Verisig 2.0 is both more accurate and faster, achieving speed-ups of up to 21x and 268x against different tools, respectively.