Reliability Test based on a Binomial Experiment for Probabilistic Worst-Case Execution Times
Ref: CISTER-TR-201103 Publication Date: 1 to 4, Dec, 2020
Reliability Test based on a Binomial Experiment for Probabilistic Worst-Case Execution Times
Ref: CISTER-TR-201103 Publication Date: 1 to 4, Dec, 2020Abstract:
Measurement-Based Probabilistic Timing Analysis
(MBPTA) produces Probabilistic Worst-Case Execution Times
(pWCETs), i.e., WCET estimates associated with known low
exceedance probabilities. Despite applicability and goodness-offit
tests being used within MBPTA, any method based on the
sampling of a population is subject to a degree of uncertainty.
The acceptance of MBPTA in industrial engineering processes depends
on obtaining enough evidence that the produced pWCETs
are indeed reliable. In this paper we propose a statistical
hypothesis test to check the reliability of pWCET estimates, done
at a specified significance level. We assume as null hypothesis that
the pWCET estimate is reliable, and as alternative hypothesis that
it is optimistic. Both Type I and Type II errors are considered.
The reliability test is based on a binomial experiment and it
is complementary to applicability and goodness-of-fit tests. We
evaluated the test using multiple synthetic and real-hardware
execution time samples, and applied it on 20 pWCET estimates
generated for each of them. The combined use of the proposed
reliability test with applicability and goodness-of-fit tests could
detect most of the knowingly unreliable estimates on synthetic
samples. Similar behaviour was observed for real-hardware
samples, evidencing the test’s usefulness for selecting pWCET
estimates with increased confidence.
Events:
41st IEEE Real-Time Systems Symposium (RTSS 2020).
Online.
Record Date: 16, Nov, 2020