Suppose you are trying to measure a value P and you have two unreliable sensors. Sensor A returns 0.5P + 0.5U, where U is uniform random noise over the same domain as P. Sensor B will return either P or U with 50% likelihood. In other words, sensor A is a noisy measurement of your variable, and B is sometimes the correct value and sometimes pure noise. If we have the readings of both sensor A and sensor B, what can we say about P?
Let’s make a graph to give us some intuition. Given 100 samples of P taken uniformly from [0, 1), this shows the relative error of sensor A vs. the relative error of sensor B.
As we would expect, errorA is 0 half the time. The rest of the time, it can range from -1 to 1; errorB is rarely exactly 0, but its range is half that of errorA.
Let’s look at some numerical summary stats for errorA, errorB, and the error of the average of A and B over 100k trials.
The mean error of the average of A and B seems a bit lower than the error of either A or B on its own. But wait! The choice to give equal weight to sensor A and sensor B seems a bit arbitrary. Let’s explore the different weights numerically, considering each weighting of A and B in increments of 0.1. Here the x-axis gives us the weight w we assign to sensor A, while sensor B is assigned weight 1 – w. The y-axis shows the average value of the absolute error of this combination over 100k simulations.
As you can see, the best weight is not 50-50, but around 0.58, where our mean error drops to around 0.1524. We could theoretically improve the precision of this estimate with a ternary search, but in practice numerical accuracy becomes a problem and I can’t nail it down much tighter than 0.586 or so. (I’m sure there’s some nice technique for this that’s eluding me at the moment.)
So that’s interesting. What’s the theoretical grounding for this mixture of variables, and can we do better than a linear mixture? Well, for one thing, it would be really great to know when sensorB is returning the correct value, and when it’s just lying to us. One intuition is, if A and B are pretty close together, then probably that’s because B is not random noise, and we should just return B instead of mixing it with A. But if they’re pretty far apart, then we should ignore B since it’s probably noise and just return A.
It turns out this strategy is pretty good, and much better than a linear mixing of the two variables!
Here the x-axis is the “cutoff” of the absolute difference between A and B. If abs(A – B) is less than X, we use sensor B, else we use sensor A. Here the location of the dip is enough to make anyone suspect good ol’ 1/e ≈ 0.367, but sadly numerical data points clearly to a minimum near 0.41, instead. We can get a mean absolute error around 0.1175, which is much better than the best we could do with a linear mixing of the signals, so maybe we don’t care that much about finding the perfect mixing, anyway.
Well, that’s a nice improvement, but can we do still better? It’s not really clear that we should switch straight from trusting sensor B to sensor A. Maybe there’s some grey zone in the middle where we should still try to mix the two sensor readings. A numerical grid search with a resolution of 0.01 and 1 million trials in each case says that we do best when
- If | A – B | < 0.367, yield sensor B’s reading;
- If | A – B | > 0.445, yield sensor A;
- Else yield a mixture of the two. (I just used the mean again here, I’m sure we could drive it down a bit by searching for the best proportion again.)
We can get down to a mean absolute error of 0.1163, markedly better than the single-cutoff case.
The discerning reader will note that the first cutoff looks an awful lot like 1/e. (I was sure that it would make an appearance at some point, somehow.) The second cutoff does not look familiar. One also starts to suspect that the ideal mixing formula might not be linear, even in the middle zone.
It’s been a long time since I was a stats student; anyone have an idea how to improve the estimate further?
Published