MAPS performance
Sensor performance criteria quoted industry-wide are often ill defined or plain misleading. MAPS performance is assessed in several ways, each with different utility. These specifications are explained here.
Definitions
Accuracy
Accuracy measures how close the result is to the actual value we were trying to achieve. The accuracy values shown below are based on our simulated tests. In our case, the accuracy is the maximum error belonging to a certain axis (x, y, z) or angle (pitch, yaw, roll) that we got during the 10000 random decodes.
Precision
Precision measures the consistency of our system and gives information about the relative accuracy in application. High precision is enough if achieving a high global accuracy is not needed but to measure the degree of displacement. In our case, the precision is the standard deviation of the errors belonging to the abovementioned 10000 decodes and the confidence intervals with a 95% confidence level. The precision could still be high even if the accuracy is low.
Resolution
Resolution shows the theoretical accuracy of our system, i.e. the absolute limit of the technology.
Noise floor
Noise floor measures the degree of change in a stationary state, which is originated from the real tests.
Repeatability
Repeatability shows if the results can reliably be repeated in various setups. In the case of the MAPS device, experiments have been conducted at NPL (National Physical Laboratory), and they have been meaningfully repeated in our own laboratory. Simulated tests have also been conducted.
Conditions
accuracy, resolution, repeatability, noise floor. TODO (?).
Results
The following table shows some of our test results and performance specifications. The first row (with megapixel values) refers to the size of the generated images used for position decoding in simulated tests. For further understanding, see the definitions above.
| 1 megapixel | 4 megapixel | 9 megapixel | |
|---|---|---|---|
| Planar position (x, y) | |||
| Accuracy | 1.12e-06 m | 3.37e-07 m | 4.46e-07 m |
| Precision* | 2.55e-08 (2.51e-08, 2.59e-08) m | 6.80e-08 (6.70e-08, 6.89e-08) m | 9.06e-08 (8.94e-08, 9.19e-08) m |
| Resolution | 8.95e-13 m | 8.95e-13 m | 8.95e-13 m |
| Noise floor | |||
| Repeatability | |||
| Measurement range | 0.021355 m to 0.02136 m | 0.021355 m to 0.02136 m | 0.021355 m to 0.02136 m |
| Depth (z) | |||
| Accuracy | 4.13e-06 m | 1.63e-07 m | 1.62e-07 m |
| Precision* | 7.15e-07 (7.05e-07, 7.25e-07) m | 1.82e-08 (1.79e-08, 1.84e-08) m | 1.85e-08 (1.82e-08, 1.87e-08) m |
| Resolution | 1.22e-13 m | 1.22e-13 m | 1.22e-13 m |
| Noise floor | |||
| Repeatability | |||
| Measurement range | 0.0037 m to 0.007 m | 0.0037 m to 0.007 m | 0.0037 m to 0.007 m |
| Off-angle (x-, y- axis rotation) | |||
| Accuracy | 3.50e-02 rad | 1.40e-04 rad | 1.33e-04 rad |
| Precision* | 1.00e-02 (9.92e-03, 1.02e-02) rad | 1.60e-05 (1.57e-05, 1.62e-05) rad | 1.59e-05 (1.57e-05, 1.61e-05) rad |
| Resolution | 1.59e-13 rad | 1.59e-13 rad | 1.59e-13 rad |
| Noise floor | |||
| Repeatability | |||
| Measurement range | 0 rad to 0.0349065850399 rad | 0 rad to 0.0349065850399 rad | 0 rad to 0.0349065850399 rad |
| Planar rotation (z axis) | |||
| Accuracy | 1.08e-03 rad | 2.67e-05 rad | 2.51e-05 rad |
| Precision* | 2.21e-05 (2.18e-05, 2.24e-05) rad | 2.73e-06 (2.69e-06, 2.77e-06) rad | 2.78e-06 (2.74e-06, 2.81e-06) rad |
| Resolution | 1.07e-14 rad | 1.07e-14 rad | 1.07e-14 rad |
| Noise floor | |||
| Repeatability | |||
| Measurement range | 0 rad to 0.7853981633974 rad | 0 rad to 0.7853981633974 rad | 0 rad to 0.7853981633974 rad |
* confidence interval for the standard deviation of 10000 random tests at a 95% confidence level
