MAPS performance: Difference between revisions
(The table of results is added.) |
(Modifying definitions.) |
||
| Line 5: | Line 5: | ||
=== Accuracy === | === Accuracy === | ||
Accuracy measures how close the result is to the actual value we were trying to achieve. The accuracy values shown below are based on our simulated tests. | Accuracy measures how close the result is to the actual value we were trying to achieve. The accuracy values shown below are based on our simulated tests. In our case, the accuracy is the maximum error belonging to a certain axis (x, y, z) or angle (pitch, yaw, roll) that we got during the 10000 random decodes. | ||
=== Precision === | === Precision === | ||
Precision measures the consistency of our system and gives information about the relative accuracy in application. High precision is enough if achieving a high global accuracy is not needed but to measure the degree of displacement. | Precision measures the consistency of our system and gives information about the relative accuracy in application. High precision is enough if achieving a high global accuracy is not needed but to measure the degree of displacement. In our case, the precision is the standard deviation of the errors belonging to the abovementioned 10000 decodes and the confidence intervals with a 95% confidence level. The precision could still be high even if our accuracy is low. | ||
=== Resolution === | === Resolution === | ||
| Line 14: | Line 14: | ||
=== Noise floor === | === Noise floor === | ||
Noise floor measures the degree of change in a stationary state. | Noise floor measures the degree of change in a stationary state, which is originated from the real tests. | ||
=== Repeatability === | === Repeatability === | ||
Revision as of 14:27, 5 December 2023
Sensor performance criteria quoted industry-wide are often ill defined or plain misleading. MAPS performance is assessed in several ways, each with different utility. These specifications are explained here.
Definitions
Accuracy
Accuracy measures how close the result is to the actual value we were trying to achieve. The accuracy values shown below are based on our simulated tests. In our case, the accuracy is the maximum error belonging to a certain axis (x, y, z) or angle (pitch, yaw, roll) that we got during the 10000 random decodes.
Precision
Precision measures the consistency of our system and gives information about the relative accuracy in application. High precision is enough if achieving a high global accuracy is not needed but to measure the degree of displacement. In our case, the precision is the standard deviation of the errors belonging to the abovementioned 10000 decodes and the confidence intervals with a 95% confidence level. The precision could still be high even if our accuracy is low.
Resolution
Resolution shows the theoretical accuracy of our system, i.e. the absolute limit of the technology.
Noise floor
Noise floor measures the degree of change in a stationary state, which is originated from the real tests.
Repeatability
Repeatability shows if the results can reliably be repeated in various setups. In case of the MAPS device experiments have been conducted at NPL (National Physical Laboratory), ant they have been meaningfully repeated in our own laboratory as well as simulated tests.
Conditions
accuracy, resolution, repeatability, noise floor.
Results
| 1 megapixel | 4 megapixel | 9 megapixel | |
|---|---|---|---|
| Planar position (x, y) | |||
| Accuracy | 1.12e-06 m | 3.37e-07 m | 4.46e-07 m |
| Precision* | 2.55e-08 (2.51e-08, 2.59e-08) m | 6.80e-08 (6.70e-08, 6.89e-08) m | 9.06e-08 (8.94e-08, 9.19e-08) m |
| Resolution | 8.95e-13 m | 8.95e-13 m | 8.95e-13 m |
| Noise floor | |||
| Repeatability | |||
| Measurement range | 0.021355 m to 0.02136 m | 0.021355 m to 0.02136 m | 0.021355 m to 0.02136 m |
| Depth (z) | |||
| Accuracy | 4.13e-06 m | 1.63e-07 m | 1.62e-07 m |
| Precision* | 7.15e-07 (7.05e-07, 7.25e-07) m | 1.82e-08 (1.79e-08, 1.84e-08) m | 1.85e-08 (1.82e-08, 1.87e-08) m |
| Resolution | 1.22e-13 m | 1.22e-13 m | 1.22e-13 m |
| Noise floor | |||
| Repeatability | |||
| Measurement range | 0.0037 m to 0.007 m | 0.0037 m to 0.007 m | 0.0037 m to 0.007 m |
| Off-angle (x-, y- axis rotation) | |||
| Accuracy | 3.50e-02 rad | 1.40e-04 rad | 1.33e-04 rad |
| Precision* | 1.00e-02 (9.92e-03, 1.02e-02) rad | 1.60e-05 (1.57e-05, 1.62e-05) rad | 1.59e-05 (1.57e-05, 1.61e-05) rad |
| Resolution | 1.59e-13 rad | 1.59e-13 rad | 1.59e-13 rad |
| Noise floor | |||
| Repeatability | |||
| Measurement range | 0 rad to 0.0349065850399 rad | 0 rad to 0.0349065850399 rad | 0 rad to 0.0349065850399 rad |
| Planar rotation (z axis) | |||
| Accuracy | 1.08e-03 rad | 2.67e-05 rad | 2.51e-05 rad |
| Precision* | 2.21e-05 (2.18e-05, 2.24e-05) rad | 2.73e-06 (2.69e-06, 2.77e-06) rad | 2.78e-06 (2.74e-06, 2.81e-06) rad |
| Resolution | 1.07e-14 rad | 1.07e-14 rad | 1.07e-14 rad |
| Noise floor | |||
| Repeatability | |||
| Measurement range | 0 rad to 0.7853981633974 rad | 0 rad to 0.7853981633974 rad | 0 rad to 0.7853981633974 rad |
* confidence interval for the standard deviation of 10000 random tests at a 95% confidence level
