lynx   »   [go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v253y2025ics0951832024005854.html
   My bibliography  Save this article

A benchmark on uncertainty quantification for deep learning prognostics

Author

Listed:
  • Basora, Luis
  • Viens, Arthur
  • Chao, Manuel Arias
  • Olive, Xavier
Abstract
Reliable uncertainty quantification on RUL prediction is crucial for informative decision-making in predictive maintenance. In this context, we assess some of the latest developments in the field of uncertainty quantification for deep learning prognostics. This includes the state-of-the-art variational inference algorithms for Bayesian neural networks (BNN) as well as popular alternatives such as Monte Carlo Dropout (MCD), deep ensembles (DE), and heteroscedastic neural networks (HNN). All the inference techniques share the same inception architecture as functional model. The performance of the methods is evaluated on a subset of the large NASA N-CMAPSS dataset for aircraft engines. The assessment includes RUL prediction accuracy, the quality of predictive uncertainty, and the possibility of breaking down the total predictive uncertainty into its aleatoric and epistemic parts. Although all methods are close in terms of accuracy, we find differences in the way they estimate uncertainty. Thus, DE and MCD generally provide more conservative predictive uncertainty than BNN. Surprisingly, HNN achieve strong results without the added complexity of BNN. None of these methods exhibited strong robustness to out-of-distribution cases, with BNN and HNN methods particularly susceptible to low accuracy and overconfidence. BNN techniques presented anomalous miscalibration issues at the later stages of the system lifetime.

Suggested Citation

  • Basora, Luis & Viens, Arthur & Chao, Manuel Arias & Olive, Xavier, 2025. "A benchmark on uncertainty quantification for deep learning prognostics," Reliability Engineering and System Safety, Elsevier, vol. 253(C).
    • Handle: RePEc:eee:reensy:v:253:y:2025:i:c:s0951832024005854
    DOI: 10.1016/j.ress.2024.110513
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0951832024005854
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ress.2024.110513?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Deng, Yingjun & Bucchianico, Alessandro Di & Pechenizkiy, Mykola, 2020. "Controlling the accuracy and uncertainty trade-off in RUL prediction with a surrogate Wiener propagation model," Reliability Engineering and System Safety, Elsevier, vol. 196(C).
    2. Arias Chao, Manuel & Kulkarni, Chetan & Goebel, Kai & Fink, Olga, 2022. "Fusing physics-based and deep learning models for prognostics," Reliability Engineering and System Safety, Elsevier, vol. 217(C).
    3. Xiang, Sheng & Li, Penghua & Huang, Yi & Luo, Jun & Qin, Yi, 2024. "Single gated RNN with differential weighted information storage mechanism and its application to machine RUL prediction," Reliability Engineering and System Safety, Elsevier, vol. 242(C).
    4. Lee, Juseong & Mitici, Mihaela, 2023. "Deep reinforcement learning for predictive aircraft maintenance using probabilistic Remaining-Useful-Life prognostics," Reliability Engineering and System Safety, Elsevier, vol. 230(C).
    5. Andrés Ruiz-Tagle Palazuelos & Enrique López Droguett & Rodrigo Pascual, 2020. "A novel deep capsule neural network for remaining useful life estimation," Journal of Risk and Reliability, , vol. 234(1), pages 151-167, February.
    6. Wang, Lei & Cao, Hongrui & Ye, Zhisheng & Xu, Hao, 2023. "Bayesian large-kernel attention network for bearing remaining useful life prediction and uncertainty quantification," Reliability Engineering and System Safety, Elsevier, vol. 238(C).
    7. Manuel Arias Chao & Chetan Kulkarni & Kai Goebel & Olga Fink, 2021. "Aircraft Engine Run-to-Failure Dataset under Real Flight Conditions for Prognostics and Diagnostics," Data, MDPI, vol. 6(1), pages 1-14, January.
    8. Kwon, Yongchan & Won, Joong-Ho & Kim, Beom Joon & Paik, Myunghee Cho, 2020. "Uncertainty quantification using Bayesian neural networks in classification: Application to biomedical image segmentation," Computational Statistics & Data Analysis, Elsevier, vol. 142(C).
    9. Xia, Jun & Feng, Yunwen & Teng, Da & Chen, Junyu & Song, Zhicen, 2022. "Distance self-attention network method for remaining useful life estimation of aeroengine with parallel computing," Reliability Engineering and System Safety, Elsevier, vol. 225(C).
    10. Chen, Jinglong & Jing, Hongjie & Chang, Yuanhong & Liu, Qian, 2019. "Gated recurrent unit based recurrent neural network for remaining useful life prediction of nonlinear deterioration process," Reliability Engineering and System Safety, Elsevier, vol. 185(C), pages 372-382.
    11. Li, Yuanfu & Chen, Yao & Hu, Zhenchao & Zhang, Huisheng, 2023. "Remaining useful life prediction of aero-engine enabled by fusing knowledge and deep learning models," Reliability Engineering and System Safety, Elsevier, vol. 229(C).
    12. Zhu, Rong & Chen, Yuan & Peng, Weiwen & Ye, Zhi-Sheng, 2022. "Bayesian deep-learning for RUL prediction: An active learning perspective," Reliability Engineering and System Safety, Elsevier, vol. 228(C).
    13. Li, Xiang & Ding, Qian & Sun, Jian-Qiao, 2018. "Remaining useful life estimation in prognostics using deep convolution neural networks," Reliability Engineering and System Safety, Elsevier, vol. 172(C), pages 1-11.
    14. Zhuang, Liangliang & Xu, Ancha & Wang, Xiao-Lin, 2023. "A prognostic driven predictive maintenance framework based on Bayesian deep learning," Reliability Engineering and System Safety, Elsevier, vol. 234(C).
    15. Minhee Kim & Kaibo Liu, 2020. "A Bayesian deep learning framework for interval estimation of remaining useful life in complex systems by incorporating general degradation characteristics," IISE Transactions, Taylor & Francis Journals, vol. 53(3), pages 326-340, December.
    16. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    17. da Costa, Paulo Roberto de Oliveira & Akçay, Alp & Zhang, Yingqian & Kaymak, Uzay, 2020. "Remaining useful lifetime prediction via deep domain adaptation," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xiong, Jiawei & Zhou, Jian & Ma, Yizhong & Zhang, Fengxia & Lin, Chenglong, 2023. "Adaptive deep learning-based remaining useful life prediction framework for systems with multiple failure patterns," Reliability Engineering and System Safety, Elsevier, vol. 235(C).
    2. Dehghan Shoorkand, Hassan & Nourelfath, Mustapha & Hajji, Adnène, 2024. "A hybrid CNN-LSTM model for joint optimization of production and imperfect predictive maintenance planning," Reliability Engineering and System Safety, Elsevier, vol. 241(C).
    3. Li, Zhanhang & Zhou, Jian & Nassif, Hani & Coit, David & Bae, Jinwoo, 2023. "Fusing physics-inferred information from stochastic model with machine learning approaches for degradation prediction," Reliability Engineering and System Safety, Elsevier, vol. 232(C).
    4. Nejjar, Ismail & Geissmann, Fabian & Zhao, Mengjie & Taal, Cees & Fink, Olga, 2024. "Domain adaptation via alignment of operation profile for Remaining Useful Lifetime prediction," Reliability Engineering and System Safety, Elsevier, vol. 242(C).
    5. Xiao, Lei & Tang, Junxuan & Zhang, Xinghui & Bechhoefer, Eric & Ding, Siyi, 2021. "Remaining useful life prediction based on intentional noise injection and feature reconstruction," Reliability Engineering and System Safety, Elsevier, vol. 215(C).
    6. Zhuang, Liangliang & Xu, Ancha & Wang, Xiao-Lin, 2023. "A prognostic driven predictive maintenance framework based on Bayesian deep learning," Reliability Engineering and System Safety, Elsevier, vol. 234(C).
    7. Wang, Wei & Song, Honghao & Si, Shubin & Lu, Wenhao & Cai, Zhiqiang, 2024. "Data augmentation based on diffusion probabilistic model for remaining useful life estimation of aero-engines," Reliability Engineering and System Safety, Elsevier, vol. 252(C).
    8. Kamariotis, Antonios & Tatsis, Konstantinos & Chatzi, Eleni & Goebel, Kai & Straub, Daniel, 2024. "A metric for assessing and optimizing data-driven prognostic algorithms for predictive maintenance," Reliability Engineering and System Safety, Elsevier, vol. 242(C).
    9. Keshun, You & Guangqi, Qiu & Yingkui, Gu, 2024. "Optimizing prior distribution parameters for probabilistic prediction of remaining useful life using deep learning," Reliability Engineering and System Safety, Elsevier, vol. 242(C).
    10. Xu, Zhiqiang & Zhang, Yujie & Miao, Qiang, 2024. "An attention-based multi-scale temporal convolutional network for remaining useful life prediction," Reliability Engineering and System Safety, Elsevier, vol. 250(C).
    11. Lin, Yan-Hui & Chang, Liang & Guan, Lu-Xin, 2024. "Enhanced stochastic recurrent hybrid model for RUL Predictions via Semi-supervised learning," Reliability Engineering and System Safety, Elsevier, vol. 248(C).
    12. Li, Yuanfu & Chen, Yifan & Shao, Haonan & Zhang, Huisheng, 2023. "A novel dual attention mechanism combined with knowledge for remaining useful life prediction based on gated recurrent units," Reliability Engineering and System Safety, Elsevier, vol. 239(C).
    13. Li, Yuanfu & Chen, Yao & Hu, Zhenchao & Zhang, Huisheng, 2023. "Remaining useful life prediction of aero-engine enabled by fusing knowledge and deep learning models," Reliability Engineering and System Safety, Elsevier, vol. 229(C).
    14. Fu, Song & Lin, Lin & Wang, Yue & Guo, Feng & Zhao, Minghang & Zhong, Baihong & Zhong, Shisheng, 2024. "MCA-DTCN: A novel dual-task temporal convolutional network with multi-channel attention for first prediction time detection and remaining useful life prediction," Reliability Engineering and System Safety, Elsevier, vol. 241(C).
    15. Zhu, Yongmeng & Wu, Jiechang & Wu, Jun & Liu, Shuyong, 2022. "Dimensionality reduce-based for remaining useful life prediction of machining tools with multisensor fusion," Reliability Engineering and System Safety, Elsevier, vol. 218(PB).
    16. Zhan, Yuling & Kong, Ziqian & Wang, Ziqi & Jin, Xiaohang & Xu, Zhengguo, 2024. "Remaining useful life prediction with uncertainty quantification based on multi-distribution fusion structure," Reliability Engineering and System Safety, Elsevier, vol. 251(C).
    17. Yan, Jianhai & He, Zhen & He, Shuguang, 2023. "Multitask learning of health state assessment and remaining useful life prediction for sensor-equipped machines," Reliability Engineering and System Safety, Elsevier, vol. 234(C).
    18. Chen, Jiaxian & Li, Dongpeng & Huang, Ruyi & Chen, Zhuyun & Li, Weihua, 2023. "Aero-engine remaining useful life prediction method with self-adaptive multimodal data fusion and cluster-ensemble transfer regression," Reliability Engineering and System Safety, Elsevier, vol. 234(C).
    19. Xiao, Dasheng & Lin, Zhifu & Yu, Aiyang & Tang, Ke & Xiao, Hong, 2024. "Data-driven method embedded physical knowledge for entire lifecycle degradation monitoring in aircraft engines," Reliability Engineering and System Safety, Elsevier, vol. 247(C).
    20. Pan, Junlin & Sun, Bo & Wu, Zeyu & Yi, Zechen & Feng, Qiang & Ren, Yi & Wang, Zili, 2024. "Probabilistic remaining useful life prediction without lifetime labels: A Bayesian deep learning and stochastic process fusion method," Reliability Engineering and System Safety, Elsevier, vol. 250(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:253:y:2025:i:c:s0951832024005854. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.
    Лучший частный хостинг