Sensor-Fusion Tuning and Performance Assessment

4.2.19 Sensor-Fusion Tuning and Performance Assessment

Sensor-fusion design does not end when the estimator equations are implemented and the filter converges. A practical spacecraft navigation system must also be tuned, verified, and assessed quantitatively.

The estimator contains several quantities that strongly influence its behaviour:

\[ \mathbf Q \]

for process noise,

\[ \mathbf R \]

for measurement noise,

\[ \mathbf P_0 \]

for initial uncertainty,

together with gating thresholds, bias models, update rates, sensor-validity rules, and numerical implementation choices.

These parameters determine how the estimator balances

\[ \boxed{ \text{Dynamic Model} \quad\text{and}\quad \text{Sensor Measurements}. } \]

A poorly tuned filter may be numerically stable but inaccurate.

A different poorly tuned filter may appear accurate in one simulation but become inconsistent when sensor noise, initial error, geometry, or modelling uncertainty changes.

Therefore filter tuning should not be treated as

“adjust values until the plots look smooth.”

Instead, tuning should be based on:

\[ \boxed{ \text{Physical Sensor Knowledge} + \text{Dynamic-Model Uncertainty} + \text{Statistical Consistency} + \text{Monte Carlo Verification} } \]

The objective is to produce an estimator that is not only accurate, but also consistent, robust, explainable, and reliable across the expected operating envelope.

4.2.19.1 What Does Filter Tuning Mean?

Filter tuning means choosing the uncertainty models and estimator parameters that determine how information is weighted.

For a Kalman-type filter,

\[ \mathbf P_k^- = \mathbf F_k \mathbf P_{k-1}^+ \mathbf F_k^T + \mathbf Q_k \]

and

\[ \mathbf S_k = \mathbf H_k \mathbf P_k^- \mathbf H_k^T + \mathbf R_k. \]

The Kalman gain is

\[ \mathbf K_k = \mathbf P_k^- \mathbf H_k^T \mathbf S_k^{-1}. \]

Therefore both \(\mathbf Q\) and \(\mathbf R\) directly influence

\[ \mathbf K_k. \]

This means filter tuning controls how strongly the estimator trusts:

\[ \boxed{ \text{Prediction} } \]

relative to

\[ \boxed{ \text{Measurement}. } \]

4.2.19.2 Main Quantities That Require Tuning

A practical estimator may require tuning of:

• initial covariance \(\mathbf P_0\),
• process-noise covariance \(\mathbf Q\),
• measurement-noise covariance \(\mathbf R\),
• sensor bias process noise,
• scale-factor process noise,
• timing-state process noise,
• outlier thresholds,
• NIS thresholds,
• measurement weighting,
• covariance inflation,
• fault-detection persistence,
• sensor reintroduction thresholds,
• convergence criteria.

These quantities should not be chosen independently.

They interact.

For example,

\[ \mathbf Q \]

affects predicted covariance,

which affects

\[ \mathbf S, \]

which affects

\[ \mathrm{NIS}, \]

which affects measurement acceptance.

Thus:

\[ \boxed{ \text{Tuning Parameters Form an Interconnected System}. } \]

4.2.19.3 Role of Process Noise \(\mathbf Q\)

The process-noise covariance represents uncertainty in the dynamic model.

For

\[ \mathbf x_{k+1} = f(\mathbf x_k,\mathbf u_k) + \mathbf w_k, \]

assume

\[ \mathbf w_k \sim \mathcal N(\mathbf 0,\mathbf Q_k). \]

The matrix \(\mathbf Q_k\) represents uncertainty caused by effects such as:

• unmodelled accelerations,
• disturbance torques,
• drag uncertainty,
• solar-radiation-pressure uncertainty,
• thruster execution error,
• sensor-bias evolution,
• model truncation,
• environmental uncertainty.

Thus,

\[ \boxed{ \mathbf Q = \text{Uncertainty in State Evolution}. } \]

4.2.19.4 Physical Meaning of Small \(\mathbf Q\)

A small process-noise covariance tells the estimator:

\[ \boxed{ \text{The dynamic model is highly reliable}. } \]

Then prediction covariance grows slowly:

\[ \mathbf P^- \approx \mathbf F \mathbf P^+ \mathbf F^T. \]

The filter tends to trust the model more strongly.

This can be beneficial when the model is genuinely accurate.

But if the real system experiences unmodelled disturbances, the estimator may become overconfident.

4.2.19.5 Consequences of \(\mathbf Q\) Too Small

If

\[ \mathbf Q_{\text{assumed}} < \mathbf Q_{\text{actual}}, \]

the filter underestimates model uncertainty.

Possible symptoms include:

• covariance becoming too small,
• large innovations,
• excessive NIS,
• frequent measurement rejection,
• slow adaptation to disturbances,
• biased state estimates,
• poor recovery after manoeuvres.

A filter with too-small \(\mathbf Q\) often appears smooth but may be statistically inconsistent.

Therefore:

\[ \boxed{ \text{Smooth Estimate} \neq \text{Correctly Tuned Estimate}. } \]

4.2.19.6 Consequences of \(\mathbf Q\) Too Large

If

\[ \mathbf Q_{\text{assumed}} \gg \mathbf Q_{\text{actual}}, \]

prediction uncertainty grows rapidly.

The estimator then trusts measurements more strongly.

Consequences may include:

• noisy state estimates,
• large measurement-driven corrections,
• excessive sensitivity to sensor noise,
• reduced benefit from an accurate dynamic model,
• unnecessarily large covariance.

Thus,

\[ \boxed{ \mathbf Q \text{ too large} \rightarrow \text{measurement-dominated filter}. } \]

4.2.19.7 Role of Measurement Noise \(\mathbf R\)

The measurement model is

\[ \mathbf z_k = h(\mathbf x_k) + \mathbf v_k, \]

with

\[ \mathbf v_k \sim \mathcal N(\mathbf0,\mathbf R_k). \]

The matrix

\[ \mathbf R_k \]

represents measurement uncertainty.

For several measurement components,

\[ \mathbf R = E[ \mathbf v\mathbf v^T ]. \]

Thus,

\[ \boxed{ \mathbf R = \text{Sensor Measurement Uncertainty}. } \]

4.2.19.8 Sources of Measurement Noise

Measurement uncertainty may arise from:

• electronic noise,
• quantization,
• image-processing error,
• centroiding error,
• feature-matching uncertainty,
• LiDAR ranging noise,
• GNSS solution uncertainty,
• star-tracker centroiding and catalog error,
• calibration uncertainty,
• environmental interference,
• thermal variation,
• vibration.

The value of \(\mathbf R\) should reflect the uncertainty of the measurement actually supplied to the filter, not merely the sensor hardware specification.

4.2.19.9 \(\mathbf R\) Too Small

If the filter assumes

\[ \mathbf R_{\text{assumed}} < \mathbf R_{\text{actual}}, \]

it believes measurements are more accurate than they really are.

The Kalman gain becomes too aggressive.

Possible consequences:

• noisy state estimate,
• excessive state correction,
• underestimated covariance,
• overreaction to outliers,
• poor statistical consistency.

Thus:

\[ \boxed{ \mathbf R \text{ too small} \rightarrow \text{over-trust measurements}. } \]

4.2.19.10 \(\mathbf R\) Too Large

If

\[ \mathbf R_{\text{assumed}} \gg \mathbf R_{\text{actual}}, \]

the estimator underweights measurements.

Consequences may include:

• slow convergence,
• excessive dependence on the model,
• larger state errors after disturbances,
• delayed bias correction,
• unnecessarily large uncertainty.

Thus:

\[ \boxed{ \mathbf R \text{ too large} \rightarrow \text{under-use useful sensor information}. } \]

4.2.19.11 Relative Effect of \(\mathbf Q\) and \(\mathbf R\)

A useful conceptual ratio is

\[ \boxed{ \frac{\text{Model Uncertainty}} {\text{Measurement Uncertainty}}. } \]

If model uncertainty is large compared with sensor uncertainty, measurements dominate.

If sensor uncertainty is large compared with model uncertainty, prediction dominates.

This relationship determines much of the filter's qualitative behaviour.

4.2.19.12 Tuning Is Not Arbitrary Gain Adjustment

The Kalman gain is computed from uncertainty:

\[ \mathbf K = \mathbf P^-\mathbf H^T ( \mathbf H\mathbf P^-\mathbf H^T+\mathbf R )^{-1}. \]

Therefore \(\mathbf Q\) and \(\mathbf R\) should represent physical uncertainty.

One should not reduce \(R\) simply because:

“I want the filter to follow the sensor more.”

Nor should \(Q\) be increased merely because:

“I want the filter to react faster.”

Those behaviours should emerge from realistic uncertainty models.

4.2.19.13 Diagonal Measurement Covariance

A simple measurement covariance may be

\[ \mathbf R = \begin{bmatrix} \sigma_1^2 & 0 & 0\\ 0 & \sigma_2^2 & 0\\ 0 & 0 & \sigma_3^2 \end{bmatrix}. \]

This assumes measurement errors are uncorrelated.

That assumption may be reasonable for some sensors.

But it should be justified.

4.2.19.14 Correlated Measurement Errors

If measurement components share common error sources,

\[ R_{ij} \neq0. \]

For example, image-derived pose coordinates may be correlated because the same feature geometry contributes to multiple pose components.

Then

\[ \mathbf R = \begin{bmatrix} \sigma_x^2 & \rho_{xy}\sigma_x\sigma_y\\ \rho_{xy}\sigma_x\sigma_y & \sigma_y^2 \end{bmatrix}. \]

Ignoring strong correlations can produce overconfident fusion.

4.2.19.15 State-Dependent Measurement Noise

Measurement uncertainty may change with state.

For a camera, angular uncertainty may remain approximately constant while position uncertainty grows with range.

Thus:

\[ \mathbf R = \mathbf R(\mathbf x). \]

For example,

\[ \sigma_Z \]

may increase strongly with distance.

Therefore a fixed \(\mathbf R\) may not represent the sensor accurately across the full mission envelope.

4.2.19.16 Quality-Dependent Measurement Noise

Sensors may provide quality indicators:

\[ q_k. \]

Then:

\[ \boxed{ \mathbf R_k = f(q_k). } \]

Examples include:

• GNSS dilution of precision,
• star-tracker quality,
• feature count,
• reprojection residual,
• LiDAR return strength,
• point-cloud fit quality.

This enables the estimator to use more realistic dynamic weighting.

4.2.19.17 Adaptive Measurement Covariance

An adaptive estimator may adjust

\[ \mathbf R_k \]

online.

For example,

\[ \mathbf R_k = \alpha_k \mathbf R_0. \]

If residual variance increases,

\[ \alpha_k>1. \]

This reduces measurement influence.

Adaptive tuning can be useful, but it must be bounded and validated.

Otherwise the estimator may hide modelling errors by continuously changing \(\mathbf R\).

4.2.19.18 Adaptive Process Noise

Similarly,

\[ \mathbf Q_k \]

may depend on operating condition.

For example:

\[ \mathbf Q_{\text{coast}} \]

during quiet propagation,

and

\[ \mathbf Q_{\text{maneuver}} > \mathbf Q_{\text{coast}} \]

during thrust activity.

This reflects increased uncertainty during manoeuvres.

Thus:

\[ \boxed{ \text{Mode-Dependent Dynamics} \rightarrow \text{Mode-Dependent }Q. } \]

4.2.19.19 Process Noise from Continuous-Time Models

Suppose:

\[ \dot{\mathbf x} = \mathbf F_c\mathbf x + \mathbf G_c\mathbf w, \]

with continuous spectral density

\[ \mathbf Q_c. \]

The corresponding discrete process noise is

\[ \boxed{ \mathbf Q_d = \int_0^{\Delta t} e^{\mathbf F_c\tau} \mathbf G_c \mathbf Q_c \mathbf G_c^T e^{\mathbf F_c^T\tau} \,d\tau. } \]

This is preferable to choosing discrete covariance values arbitrarily.

4.2.19.20 Simple Random-Walk Process Noise

For a scalar random walk,

\[ \dot x=w \]

with spectral density

\[ q, \]

the discrete process variance is approximately

\[ \boxed{ Q=q\Delta t. } \]

This relationship is especially important for bias states.

4.2.19.21 Bias Random-Walk Tuning

For gyro bias,

\[ \dot b_g=w_{bg}. \]

Then

\[ Q_{bg} = q_{bg}\Delta t. \]

If \(Q_{bg}\) is too small, the estimator assumes the bias is nearly constant.

If it is too large, the bias estimate may wander excessively.

Therefore bias process noise should reflect actual bias instability.

4.2.19.22 Gauss-Markov Bias Model

A bias may instead follow

\[ \dot b = -\frac1\tau b+w. \]

Here

\[ \tau \]

is the correlation time.

This gives a first-order Gauss-Markov model.

It is often more physically realistic than a pure random walk for slowly varying sensor biases.

4.2.19.23 Using Allan Variance

IMU noise parameters can be estimated from Allan-deviation analysis.

Typical quantities include:

• angle random walk,
• velocity random walk,
• bias instability,
• rate random walk.

These can help derive physically meaningful stochastic models.

Thus:

\[ \boxed{ \text{Sensor Characterization} \rightarrow Q,\ R \text{ Selection}. } \]

4.2.19.24 Innovation as the Main Tuning Diagnostic

The innovation is

\[ \mathbf r_k = \mathbf z_k - h(\hat{\mathbf x}_k^-). \]

It contains the difference between what the sensor measures and what the estimator predicts.

A well-tuned filter should produce innovations that are broadly compatible with the predicted statistics.

Therefore innovation analysis is one of the most useful tuning tools.

4.2.19.25 Innovation Mean

For an unbiased, consistent estimator,

\[ E[\mathbf r_k] \approx \mathbf0. \]

Persistent nonzero innovation mean may indicate:

• sensor bias,
• model bias,
• frame error,
• timing error,
• calibration error,
• poor initialization,
• unmodelled dynamics.

Thus innovation mean tests bias consistency.

4.2.19.26 Innovation Covariance

Predicted innovation covariance is

\[ \boxed{ \mathbf S_k = \mathbf H_k \mathbf P_k^- \mathbf H_k^T + \mathbf R_k. } \]

Empirical covariance can be estimated over \(N\) samples:

\[ \hat{\mathbf S} = \frac1{N-1} \sum_{k=1}^{N} ( \mathbf r_k-\bar{\mathbf r} ) ( \mathbf r_k-\bar{\mathbf r} )^T. \]

Ideally:

\[ \hat{\mathbf S} \approx \mathbf S. \]

4.2.19.27 What If Empirical Innovation Variance Is Too Large?

If

\[ \hat{\mathbf S} > \mathbf S \]

consistently, possible causes include:

• \(Q\) too small,
• \(R\) too small,
• unmodelled dynamics,
• sensor degradation,
• wrong measurement model.

The filter is predicting less uncertainty than the data actually exhibit.

This indicates overconfidence.

4.2.19.28 What If Innovation Variance Is Too Small?

If

\[ \hat{\mathbf S} < \mathbf S, \]

possible causes include:

• \(Q\) too large,
• \(R\) too large,
• overly conservative covariance assumptions.

The filter may be more pessimistic than necessary.

This may reduce achievable performance.

4.2.19.29 Innovation Whiteness

For a correctly modelled estimator,

\[ E[ \mathbf r_k \mathbf r_{k-\tau}^T ] \approx 0 \qquad \tau\neq0. \]

If innovations are strongly correlated in time, the estimator may be missing systematic dynamics.

Thus whiteness provides information that innovation variance alone cannot reveal.

4.2.19.30 Innovation Autocorrelation

Define

\[ \mathbf C_r(\tau) = E[ \mathbf r_k \mathbf r_{k-\tau}^T ]. \]

For a well-tuned filter:

\[ \mathbf C_r(\tau) \approx 0 \]

for nonzero lag.

Significant repeated structure may indicate:

• underestimated process noise,
• missing bias state,
• timing error,
• periodic disturbance,
• incorrect dynamic model.

4.2.19.31 Normalized Innovation Squared

The NIS is

\[ \boxed{ \epsilon_k = \mathbf r_k^T \mathbf S_k^{-1} \mathbf r_k. } \]

For a measurement dimension \(m\), under ideal assumptions:

\[ \epsilon_k \sim \chi_m^2. \]

This provides a quantitative consistency test.

4.2.19.32 Expected NIS

The expected NIS is approximately

\[ E[\epsilon_k] = m. \]

Thus if

\[ m=3, \]

one expects the long-run NIS average to be near

\[ 3. \]

This does not mean every individual NIS should equal 3.

It is a statistical expectation.

4.2.19.33 Average NIS

Across \(N\) samples,

\[ \boxed{ \overline{\mathrm{NIS}} = \frac1N \sum_{k=1}^{N} \epsilon_k. } \]

If the value is persistently too high, the estimator may be overconfident.

If it is persistently too low, the filter may be too conservative.

4.2.19.34 NEES

When truth is available, use the Normalized Estimation Error Squared:

\[ \boxed{ \mathrm{NEES}_k = \tilde{\mathbf x}_k^T \mathbf P_k^{-1} \tilde{\mathbf x}_k } \]

where

\[ \tilde{\mathbf x}_k = \mathbf x_k^{truth} - \hat{\mathbf x}_k. \]

For an \(n\)-state estimator,

\[ E[\mathrm{NEES}] \approx n. \]

NEES is one of the most important filter-consistency metrics in simulation.

4.2.19.35 NIS vs NEES

NIS evaluates consistency in measurement space.

NEES evaluates consistency in state space.

Thus:

\[ \boxed{ \text{NIS} \rightarrow \text{Measurement-Prediction Consistency} } \]

while

\[ \boxed{ \text{NEES} \rightarrow \text{State-Error/Covariance Consistency}. } \]

Both should be used where possible.

4.2.19.36 Why NEES Cannot Normally Be Used in Flight

NEES requires

\[ \mathbf x^{truth}. \]

In actual flight, true spacecraft state is not known exactly.

Therefore NEES is primarily a simulation and test metric.

Flight consistency monitoring relies more heavily on:

• NIS,
• innovation statistics,
• redundant sensor comparisons,
• independent reference systems.

4.2.19.37 Chi-Square Confidence Bounds

For measurement dimension \(m\),

\[ \mathrm{NIS} \sim \chi_m^2. \]

A two-sided consistency interval can be defined:

\[ \chi^2_{\alpha/2,m} < \mathrm{NIS} < \chi^2_{1-\alpha/2,m}. \]

Frequent violation indicates statistical inconsistency.

4.2.19.38 Why One NIS Violation Is Not Enough

Even a perfectly tuned Gaussian estimator occasionally produces large residuals.

Therefore:

\[ \boxed{ \text{One Threshold Violation} \neq \text{Bad Tuning}. } \]

The important quantities are long-run frequencies, averages, trends, and distributions.

4.2.19.39 Confidence-Bound Coverage

In simulation, state error can be compared with covariance bounds.

For a scalar state:

\[ \tilde x_k = x_k^{truth} - \hat x_k. \]

Compare with:

\[ \pm\sigma_k = \pm\sqrt{P_{ii,k}}, \]

or

\[ \pm3\sigma_k. \]

If the filter is statistically consistent, the actual error should remain inside these bounds with approximately the expected frequency.

4.2.19.40 Three-Sigma Coverage

For a Gaussian scalar variable, approximately

\[ 99.7\% \]

of samples lie within

\[ \pm3\sigma. \]

Therefore large numbers of excursions outside \(3\sigma\) indicate that the covariance may be too optimistic.

However, state samples are often temporally correlated, so interpretation should still be statistical rather than purely pointwise.

4.2.19.41 Overconfident Filter

A filter is overconfident when:

\[ |\tilde x| \]

is larger than predicted by

\[ P. \]

Typical symptoms:

• NEES too high,
• NIS too high,
• frequent \(3\sigma\) violations,
• frequent gate rejection.

Possible tuning causes:

• \(Q\) too small,
• \(R\) too small,
• \(P_0\) too small.

4.2.19.42 Conservative Filter

A filter is conservative when:

\[ P \]

is larger than necessary relative to actual error.

Symptoms may include:

• NEES too low,
• NIS too low,
• covariance significantly larger than empirical error.

This is generally safer than overconfidence but can reduce mission performance.

4.2.19.43 Accuracy Versus Consistency

Accuracy and consistency are different.

A filter may have small RMS error but still report unrealistically small covariance.

That filter is accurate in the tested scenario but statistically inconsistent.

Conversely, a filter may be statistically consistent but insufficiently accurate for the mission requirement.

Therefore:

\[ \boxed{ \text{Accuracy} \neq \text{Consistency}. } \]

Both must be assessed.

4.2.19.44 Precision Versus Accuracy

Precision refers to repeatability or spread.

Accuracy refers to proximity to truth.

A tightly clustered estimate may still be biased.

Thus:

\[ \boxed{ \text{High Precision} \neq \text{High Accuracy}. } \]

Persistent innovation bias often reveals this situation.

4.2.19.45 Root Mean Square Error

For state error

\[ e_k = x_k-\hat x_k, \]

RMSE is

\[ \boxed{ \mathrm{RMSE} = \sqrt{ \frac1N \sum_{k=1}^{N} e_k^2 }. } \]

For vector quantities,

\[ \mathrm{RMSE}_{\mathbf r} = \sqrt{ \frac1N \sum_{k=1}^{N} \| \mathbf r_k-\hat{\mathbf r}_k \|^2 }. \]

RMSE gives a useful overall accuracy measure.

4.2.19.46 Mean Absolute Error

Another metric is

\[ \boxed{ \mathrm{MAE} = \frac1N \sum_{k=1}^{N} |e_k|. } \]

MAE is less sensitive to a small number of extreme errors than RMSE.

Both metrics can be useful.

4.2.19.47 Maximum Error

For safety-critical analysis, mean metrics alone are insufficient.

Define

\[ \boxed{ e_{\max} = \max_k |e_k|. } \]

A filter may have excellent average performance but still produce occasional large errors.

Maximum error is therefore important for operational margins.

4.2.19.48 Percentile Error

Monte Carlo analysis often uses percentiles:

\[ e_{50}, \qquad e_{95}, \qquad e_{99}. \]

For example:

\[ P( |e|

Percentiles provide more information than only mean or worst-case values.

4.2.19.49 Bias Error

Define mean estimation error:

\[ \boxed{ \bar e = \frac1N \sum_{k=1}^{N} e_k. } \]

If

\[ \bar e\neq0, \]

systematic bias may remain.

This should be evaluated separately from random spread.

4.2.19.50 Standard Deviation of Error

The empirical error spread is

\[ \boxed{ \sigma_e = \sqrt{ \frac1{N-1} \sum ( e_k-\bar e )^2 }. } \]

Comparing this with

\[ \sqrt{P_{ii}} \]

provides another covariance-consistency check.

4.2.19.51 Steady-State Performance

After convergence, evaluate performance over a steady-state window.

Exclude the startup transient if the objective is steady-state accuracy.

Metrics may include:

• RMSE,
• MAE,
• covariance level,
• innovation variance,
• NIS,
• bias error.

This separates initialization behaviour from normal operation.

4.2.19.52 Transient Performance

Startup and sensor transitions should be evaluated separately.

Important quantities include:

\[ T_{\text{conv}}, \]

peak error,

\[ e_{\text{peak}}, \]

and maximum covariance.

A filter with excellent steady-state performance may still be unacceptable if transient errors are too large.

4.2.19.53 Recovery Performance

After sensor dropout or disturbance, define recovery time:

\[ \boxed{ T_{\text{recover}} = t_{\text{performance restored}} - t_{\text{event}}. } \]

This is particularly important for autonomous spacecraft navigation.

4.2.19.54 Measurement-Rejection Rate

Define:

\[ \boxed{ f_{\text{reject}} = \frac{ N_{\text{rejected}} }{ N_{\text{total}} }. } \]

Unexpectedly high rejection rate may indicate:

• overly tight gating,
• covariance underestimation,
• sensor degradation,
• model mismatch.

4.2.19.55 Sensor Utilization Rate

Another useful quantity is:

\[ \boxed{ f_{\text{use}} = \frac{ N_{\text{fused}} }{ N_{\text{available}} }. } \]

A sensor may be available but rarely fused because it repeatedly fails validation.

That is operationally important.

4.2.19.56 State-Correction Magnitude

Monitor:

\[ \Delta\hat{\mathbf x}_k = \hat{\mathbf x}_k^+ - \hat{\mathbf x}_k^-. \]

Large repeated corrections may indicate:

• model mismatch,
• sensor bias,
• incorrect \(Q\),
• incorrect \(R\),
• poor initialization.

Small corrections alone do not prove good performance.

4.2.19.57 Kalman Gain Monitoring

The gain

\[ \mathbf K_k \]

shows how strongly each measurement affects each state.

Very small gain may indicate:

• large \(R\),
• small \(P^-\),
• weak measurement sensitivity.

Very large gain may indicate:

• small \(R\),
• large \(P^-\).

Gain monitoring can therefore help explain filter behaviour.

4.2.19.58 Covariance Trace

A scalar uncertainty metric is:

\[ \boxed{ \operatorname{tr}(\mathbf P). } \]

This summarizes total variance across states.

It is useful for trend monitoring but should not replace state-specific analysis.

A small trace can still hide one poorly estimated critical direction.

4.2.19.59 Covariance Determinant

Another metric is:

\[ \boxed{ \det(\mathbf P). } \]

For Gaussian uncertainty, this is related to the volume of the uncertainty ellipsoid.

A smaller determinant implies smaller total uncertainty volume.

However, numerical scaling can make determinant-based metrics difficult to interpret.

4.2.19.60 Covariance Eigenvalues

Eigenvalue decomposition gives:

\[ \mathbf P = \mathbf V \boldsymbol\Lambda \mathbf V^T. \]

The largest eigenvalue corresponds to the weakest uncertainty direction.

Therefore:

\[ \boxed{ \lambda_{\max}(\mathbf P) } \]

is a useful worst-direction uncertainty metric.

4.2.19.61 Why Diagonal Covariance Alone Can Mislead

Suppose:

\[ P_{11}, P_{22}, P_{33} \]

are moderate, but strong correlations exist.

The actual uncertainty ellipsoid may still have a large principal axis.

Thus covariance eigenanalysis can reveal information hidden from diagonal plots.

4.2.19.62 Sensitivity Analysis

Tuning should be tested by varying key parameters.

For example:

\[ Q \rightarrow \alpha_Q Q, \]

\[ R \rightarrow \alpha_R R. \]

Evaluate the resulting change in:

• RMSE,
• NEES,
• NIS,
• convergence time,
• rejection rate.

This reveals whether the estimator is robust to tuning uncertainty.

4.2.19.63 One-Parameter-at-a-Time Sweep

A simple tuning study varies one parameter while holding others fixed.

For example:

\[ \alpha_Q = 0.1,\, 0.3,\, 1,\, 3,\, 10. \]

Then evaluate performance.

This provides intuition about sensitivity.

4.2.19.64 Limitation of One-at-a-Time Sweeps

Filter parameters interact.

A value of \(Q\) that performs well for one \(R\) may not perform similarly for another.

Therefore multidimensional tuning studies are often necessary.

4.2.19.65 Grid Search

A grid can evaluate:

\[ (\alpha_Q,\alpha_R). \]

For each pair compute:

\[ J(\alpha_Q,\alpha_R). \]

A performance cost might combine:

\[ J = w_1 \mathrm{RMSE} + w_2 T_{\text{conv}} + w_3 J_{\text{consistency}}. \]

This gives a structured tuning map.

4.2.19.66 Tuning Cost Function

A possible cost function is:

\[ \boxed{ J = w_r \mathrm{RMSE}_r + w_v \mathrm{RMSE}_v + w_c J_{\text{consistency}} + w_t T_{\text{conv}}. } \]

Weights reflect mission priorities.

Care is needed because optimizing one scalar cost can hide unacceptable behaviour in individual metrics.

4.2.19.67 Do Not Tune Only for RMSE

Minimizing RMSE alone may produce an overconfident covariance.

The better objective is:

\[ \boxed{ \text{Accuracy} + \text{Consistency} + \text{Robustness}. } \]

A slightly larger RMSE may be acceptable if the filter is far more statistically reliable.

4.2.19.68 Monte Carlo Tuning

Monte Carlo analysis should vary uncertain quantities such as:

• initial-state errors,
• sensor noise,
• sensor bias,
• process disturbances,
• timing offsets,
• update rates,
• dropouts,
• environmental uncertainty.

For each configuration, compute filter metrics across all runs.

This tests whether tuning works statistically rather than only for one trajectory.

4.2.19.69 Monte Carlo Performance Distribution

Instead of reporting one RMSE, evaluate:

\[ P( \mathrm{RMSE} < \mathrm{RMSE}_{req} ). \]

Also record percentiles:

\[ \mathrm{RMSE}_{50}, \qquad \mathrm{RMSE}_{95}, \qquad \mathrm{RMSE}_{99}. \]

This provides a more meaningful characterization of estimator reliability.

4.2.19.70 Monte Carlo Consistency

For \(N_{MC}\) runs, compute NEES for each.

Then evaluate average NEES:

\[ \overline{\mathrm{NEES}} = \frac1{N_{MC}} \sum_i \mathrm{NEES}^{(i)}. \]

Compare with appropriate chi-square confidence bounds.

This is a standard way to validate filter covariance consistency.

4.2.19.71 Tuning Across Operating Conditions

A filter should not be tuned only at one state.

Test across:

• different ranges,
• different attitudes,
• different orbital locations,
• different lighting,
• different sensor geometries,
• different manoeuvre profiles,
• different sensor availability.

This is particularly important because observability and measurement quality can change substantially with operating condition.

4.2.19.72 Mission-Phase-Dependent Tuning

Different spacecraft phases may justify different tuning.

For example:

\[ \mathbf Q_1,\mathbf R_1 \]

for quiet orbital propagation,

and

\[ \mathbf Q_2,\mathbf R_2 \]

for close-proximity relative navigation.

However, phase-dependent tuning should correspond to real changes in system uncertainty, not arbitrary performance manipulation.

4.2.19.73 Sensor-Specific Tuning

Each sensor should have its own uncertainty model.

For example:

\[ \mathbf R_{\text{GNSS}}, \]

\[ \mathbf R_{\text{camera}}, \]

\[ \mathbf R_{\text{LiDAR}}, \]

\[ \mathbf R_{\text{star tracker}}. \]

Different sensors have different noise structure, update rate, bias behaviour, and failure modes.

4.2.19.74 Update-Rate Dependence

Changing measurement rate affects estimator information flow.

A sensor at

\[ 20~\mathrm{Hz} \]

provides more frequent corrections than the same sensor at

\[ 1~\mathrm{Hz}. \]

However, if noise is strongly correlated between samples, the information increase is less than proportional to update rate.

4.2.19.75 Sample Correlation

Kalman filters often assume measurement errors are temporally independent:

\[ E[ v_kv_{k-j}^T ] = 0. \]

Real sensors may violate this.

Examples include image-processing smoothing or GNSS filtering.

If correlated measurements are treated as independent, information may be double-counted.

4.2.19.76 Effective Measurement Covariance Under Correlation

If a sensor internally filters its output, its reported sample-to-sample variance may appear small.

But consecutive samples may be strongly correlated.

Then simply assigning small \(R\) can make the navigation filter overconfident.

Sensor-processing architecture should therefore be understood before tuning \(R\).

4.2.19.77 Cross-Sensor Correlation

Two sensors may share:

• common clock,
• common ephemeris,
• common attitude transformation,
• common calibration,
• common environmental error.

Then:

\[ E[ v_i v_j^T ] \neq 0. \]

Ignoring correlation can overstate information gain from fusion.

4.2.19.78 Double Counting Information

Suppose an external subsystem already fuses GNSS and IMU and provides a navigation solution.

If the main filter also separately fuses the same raw GNSS and IMU measurements, information may be double-counted.

Thus estimator architecture must identify the origin of every measurement product.

4.2.19.79 Measurement Covariance From Sensor Specifications

Datasheets may provide quantities such as:

• RMS error,
• \(1\sigma\),
• \(3\sigma\),
• bias stability,
• resolution,
• accuracy.

These are not interchangeable.

If a specification states

\[ 3\sigma = a, \]

then approximately

\[ \sigma=\frac a3 \]

under Gaussian assumptions.

Careful interpretation is required before constructing \(R\).

4.2.19.80 Resolution Is Not Measurement Noise

Sensor resolution describes the smallest quantized increment.

It is not automatically equal to measurement standard deviation.

Similarly:

\[ \boxed{ \text{Resolution} \neq \text{Accuracy} \neq \text{Noise}. } \]

Using resolution directly as \(R\) can severely underestimate uncertainty.

4.2.19.81 Bias Stability Is Not White Noise

Bias instability describes low-frequency stochastic behaviour.

White-noise density describes high-frequency random noise.

They affect different parts of the estimator model.

Bias instability often belongs in:

\[ Q_{\text{bias}} \]

or bias-state initialization,

while white measurement noise influences:

\[ R. \]

4.2.19.82 Sensor Noise Density

An IMU may specify noise density such as:

\[ \sigma_n \; \frac{\mathrm{unit}}{\sqrt{\mathrm{Hz}}}. \]

The discrete noise depends on sampling bandwidth.

Thus converting datasheet noise density into discrete covariance requires careful treatment of sample rate and filtering.

4.2.19.83 Empirical Measurement-Noise Estimation

If stationary or controlled data are available, measurement covariance can be estimated:

\[ \hat{\mathbf R} = \frac1{N-1} \sum_k ( \mathbf z_k-\bar{\mathbf z} ) ( \mathbf z_k-\bar{\mathbf z} )^T. \]

But this estimate includes only the variation present in that test condition.

It may not capture dynamic environmental effects.

4.2.19.84 Residual-Based Covariance Estimation

Innovation data can also help infer noise mismatch.

Because:

\[ \mathbf S = \mathbf H \mathbf P^- \mathbf H^T + \mathbf R, \]

one can compare empirical innovation covariance against predicted terms.

However, \(Q\) and \(R\) influence each other, so uniquely identifying both from residuals can be difficult.

4.2.19.85 Identifiability of \(Q\) and \(R\)

Different combinations of \(Q\) and \(R\) may produce similar innovation statistics.

Therefore tuning is not always uniquely identifiable from data alone.

Physical sensor knowledge and dynamic understanding remain essential.

4.2.19.86 Tuning Through Likelihood

For a Gaussian innovation:

\[ p(\mathbf r_k) = \frac{ 1 }{ \sqrt{ (2\pi)^m |\mathbf S_k| } } \exp \left[ -\frac12 \mathbf r_k^T \mathbf S_k^{-1} \mathbf r_k \right]. \]

Parameters can be tuned to maximize measurement likelihood.

This provides a statistically principled approach.

4.2.19.87 Negative Log-Likelihood

Ignoring constants:

\[ \boxed{ J = \sum_k \left[ \log|\mathbf S_k| + \mathbf r_k^T \mathbf S_k^{-1} \mathbf r_k \right]. } \]

Minimizing this cost can help estimate tuning parameters.

However, the resulting values should still be checked physically and through Monte Carlo validation.

4.2.19.88 Innovation-Based Adaptive Estimation

Adaptive methods may estimate \(R\) using residual statistics.

For example:

\[ \hat{\mathbf R}_k = \beta \hat{\mathbf R}_{k-1} + (1-\beta) \mathbf r_k\mathbf r_k^T. \]

More sophisticated formulations subtract predicted state contribution.

The principle is that recent innovation behaviour influences the assumed measurement covariance.

4.2.19.89 Risks of Adaptive Tuning

Adaptive tuning can mistake:

• real spacecraft manoeuvre,
• model error,
• sensor fault,
• temporary outlier,

for a permanent change in sensor noise.

Therefore adaptive methods should include:

• bounds,
• time constants,
• health logic,
• fault isolation.

4.2.19.90 Tuning Out Model Errors

A common mistake is increasing \(Q\) until residuals look statistically acceptable even when the true problem is a wrong model.

For example:

\[ \text{wrong frame} \]

or

\[ \text{incorrect sensor lever arm} \]

cannot truly be fixed by larger process noise.

Large \(Q\) may merely hide the problem.

4.2.19.91 Tuning Out Sensor Errors

Likewise, increasing \(R\) can hide:

• calibration error,
• time offset,
• incorrect measurement model,
• sensor bias.

The estimator may appear more stable but use less useful information.

Therefore tuning should not substitute for correcting deterministic errors.

4.2.19.92 Deterministic Error vs Stochastic Error

A useful distinction is:

\[ \boxed{ \text{Deterministic Error} \neq \text{Random Uncertainty}. } \]

Deterministic errors should ideally be modelled or calibrated.

Stochastic uncertainty belongs in covariance.

Examples:

• fixed boresight offset → calibration state,
• white image noise → \(R\),
• unmodelled random acceleration → \(Q\).

4.2.19.93 Bias State vs Inflated Measurement Noise

Suppose a sensor has persistent bias.

One option is to increase \(R\).

Another is to estimate bias:

\[ z = h(x)+b+v. \]

If the bias is observable, explicit bias estimation is often more physically meaningful.

Increasing \(R\) only says the measurement is uncertain; it does not explain its systematic offset.

4.2.19.94 Process-State Augmentation

Unknown disturbance can sometimes be represented as a state:

\[ \mathbf x_a = \begin{bmatrix} \mathbf x\\ \mathbf d \end{bmatrix}. \]

Then:

\[ \dot{\mathbf d} = \mathbf w_d. \]

This may be preferable to simply enlarging \(Q\) on unrelated states.

Again, observability must be checked.

4.2.19.95 Tuning and Observability

Poor observability cannot be fixed by tuning.

If a state direction is unobservable,

\[ \mathbf Q \]

and

\[ \mathbf R \]

cannot create missing information.

Tuning changes how available information is weighted.

It cannot generate new information.

4.2.19.96 Tuning Weakly Observable States

For weakly observable states, tuning becomes especially delicate.

Too-small covariance may make the filter overconfident.

Too-large process noise may cause uncertainty to grow continuously.

These states should be assessed using:

• observability metrics,
• covariance growth,
• state correlation,
• Monte Carlo truth error.

4.2.19.97 Tuning and Convergence

The convergence behaviour discussed in Section 4.2.18 depends strongly on \(Q\), \(R\), and \(P_0\).

For example:

\[ P_0\uparrow \]

may produce stronger initial measurement corrections.

Similarly:

\[ R\uparrow \]

usually slows measurement-driven convergence.

Therefore convergence time should be one of the tuning metrics.

4.2.19.98 Tuning and Innovation Gating

The innovation covariance is:

\[ S = HP^-H^T+R. \]

Thus gating thresholds depend indirectly on \(Q\), \(R\), and \(P\).

A badly tuned covariance model can cause good measurements to fail the gate.

Therefore gating should be validated after filter tuning, not independently.

4.2.19.99 Tuning and FDIR

Fault-detection logic often uses NIS and residual statistics.

If the estimator covariance is poorly tuned, FDIR thresholds may produce:

• excessive false alarms,
• missed detections,
• slow fault detection.

Thus:

\[ \boxed{ \text{Filter Tuning} \rightarrow \text{FDIR Performance}. } \]

4.2.19.100 Tuning and Robust Fusion

Robust weighting often uses normalized residuals.

Therefore residual normalization depends on:

\[ \mathbf S. \]

If \(S\) is wrong, robust weights may also be wrong.

A measurement could be downweighted because of covariance mis-tuning rather than genuine outlier behaviour.

4.2.19.101 Tuning and Sensor Handover

When the active sensor set changes, estimator uncertainty and gain structure change.

For example:

\[ \{\text{GNSS},\text{IMU}\} \rightarrow \{\text{Camera},\text{IMU}\}. \]

The filter may require mode-specific \(R\), \(Q\), or gating parameters.

Sensor handover therefore belongs in the tuning verification campaign.

4.2.19.102 Relative-Navigation Mention

In spacecraft relative navigation, measurement geometry and sensor quality can change strongly with distance.

For example:

• bearing sensitivity,
• range precision,
• feature visibility,
• GNSS availability.

Therefore a fixed tuning valid at one range may not perform equally well across the complete relative-navigation envelope.

The same principle applies generally to any navigation problem with changing geometry.

4.2.19.103 Performance Requirements

Estimator performance should be evaluated against explicit requirements.

Examples may include:

\[ \mathrm{RMSE}_r < r_{\mathrm{req}}, \]

\[ \mathrm{RMSE}_v < v_{\mathrm{req}}, \]

\[ \sigma_\theta < \theta_{\mathrm{req}}, \]

\[ T_{\mathrm{conv}} < T_{\mathrm{req}}. \]

The exact values depend on the mission.

4.2.19.104 Requirement Categories

Typical navigation requirements include:

• accuracy,
• precision,
• convergence time,
• availability,
• continuity,
• integrity,
• update latency,
• fault tolerance,
• computational load.

Filter tuning should be evaluated against all relevant categories.

4.2.19.105 Accuracy Requirement

An accuracy requirement may specify:

\[ P( |e_r| < r_{\max} ) \ge p_{\mathrm{req}}. \]

This is more informative than requiring only a mean RMSE.

It expresses both magnitude and probability.

4.2.19.106 Covariance Requirement

The estimator may also require:

\[ \sqrt{P_{ii}} < \sigma_{i,\mathrm{req}}. \]

This ensures the estimator not only produces a state estimate but also reaches acceptable confidence.

4.2.19.107 Consistency Requirement

One possible requirement is that average NEES remains inside statistical confidence bounds over a defined Monte Carlo campaign.

Similarly, NIS acceptance frequencies may be specified.

This turns covariance credibility into a formal verification requirement.

4.2.19.108 Availability Requirement

Navigation availability can be defined as:

\[ \boxed{ A = \frac{ T_{\text{usable}} }{ T_{\text{mission}} }. } \]

A sensor-fusion architecture may have excellent instantaneous accuracy but poor availability because of frequent dropouts or long reconvergence times.

4.2.19.109 Continuity Requirement

Continuity concerns uninterrupted valid navigation during an operation.

A possible measure is:

\[ P( \text{No unexpected navigation interruption} ) \]

over a specified time interval.

This is particularly important for autonomous manoeuvres.

4.2.19.110 Integrity Requirement

Integrity concerns the probability that a dangerous error remains undetected.

A generic requirement may relate:

\[ P( |e|>\text{Alert Limit} \land \text{No Alert} ). \]

Filter covariance and fault monitoring both contribute to integrity.

4.2.19.111 Latency Requirement

State age can matter as much as state accuracy.

Define:

\[ T_{\text{latency}} = t_{\text{available}} - t_{\text{measurement epoch}}. \]

A precise but delayed state may be unsuitable for high-rate control.

Therefore performance assessment should include timing.

4.2.19.112 Computational Performance

A real-time estimator must complete within its update period:

\[ T_{\text{compute}} < T_{\text{cycle}}. \]

Metrics include:

• worst-case execution time,
• CPU load,
• memory,
• measurement queue length,
• missed deadlines.

These are part of estimator performance, not merely software engineering.

4.2.19.113 Filter Performance Dashboard

A useful evaluation dashboard may include:

\[ \boxed{ \begin{array}{c} \text{RMSE}\\ \text{Bias}\\ \text{Covariance}\\ \text{NIS}\\ \text{NEES}\\ \text{Innovation Mean}\\ \text{Innovation Whiteness}\\ \text{Convergence Time}\\ \text{Rejection Rate}\\ \text{Availability} \end{array} } \]

No single metric should dominate the assessment.

4.2.19.114 Simulation Truth

In simulation, truth should be generated independently from the estimator whenever possible.

If truth and estimator use exactly the same model, performance may appear unrealistically good.

Useful truth-model differences include:

• higher-order dynamics,
• more complete disturbances,
• true sensor biases,
• independent noise realizations,
• actuator uncertainty.

4.2.19.115 Avoid the Inverse Crime

Using the same model to generate truth and estimate the state can make the problem artificially easy.

This is sometimes called an inverse crime in estimation and inverse problems.

A robust verification environment should include model mismatch representative of reality.

4.2.19.116 Monte Carlo Truth Perturbations

Monte Carlo truth parameters can vary:

\[ \mathbf x_0, \quad \mathbf b, \quad \mathbf R_{\text{actual}}, \quad \mathbf Q_{\text{actual}}, \quad \text{timing}, \quad \text{environment}. \]

The estimator may retain nominal assumptions.

This reveals robustness to uncertainty mismatch.

4.2.19.117 Nominal Case

The nominal case verifies expected behaviour under representative conditions.

It should demonstrate:

• stable estimation,
• acceptable accuracy,
• consistent covariance,
• correct sensor fusion,
• reasonable convergence.

But nominal performance alone is not sufficient.

4.2.19.118 Stress Cases

Stress testing may include:

• poor geometry,
• high sensor noise,
• large bias,
• initial-state error,
• sensor dropout,
• timing offset,
• model mismatch,
• manoeuvre uncertainty.

The objective is to determine the boundary of acceptable performance.

4.2.19.119 Corner Cases

Corner cases involve combinations of difficult conditions.

For example:

\[ \text{poor geometry} + \text{high noise} + \text{sensor dropout}. \]

These combinations may reveal failures that individual tests do not.

4.2.19.120 Worst-Case Testing

Monte Carlo is probabilistic.

Worst-case analysis instead asks:

\[ \boxed{ \text{What is the most damaging credible combination of uncertainties?} } \]

This is useful when rare but safety-relevant conditions matter.

4.2.19.121 Parameter Uncertainty Sweep

Suppose measurement noise uncertainty is

\[ R_{\text{actual}} = \alpha_RR_{\text{nominal}}. \]

Test:

\[ \alpha_R = 0.5,\, 1,\, 2,\, 5. \]

Similarly:

\[ Q_{\text{actual}} = \alpha_QQ_{\text{nominal}}. \]

This quantifies robustness to covariance-model mismatch.

4.2.19.122 Bias Sweep

For each sensor bias:

\[ b \in [-b_{\max},b_{\max}], \]

evaluate:

• state RMSE,
• convergence time,
• innovation bias,
• fault detection.

This identifies the maximum tolerable bias before estimator performance becomes unacceptable.

4.2.19.123 Timing Sweep

Introduce:

\[ \delta t \]

between measurement and estimator epochs.

Evaluate:

\[ \delta t = 0,\, 1~\mathrm{ms},\, 10~\mathrm{ms},\, 100~\mathrm{ms}, \]

as appropriate for the system.

This reveals sensitivity to synchronization error.

4.2.19.124 Dropout Sweep

For sensor dropout duration:

\[ T_d, \]

evaluate state uncertainty growth and recovery.

Metrics include:

\[ P_{\max}, \]

\[ e_{\max}, \]

\[ T_{\text{recover}}. \]

This links performance assessment with degraded navigation.

4.2.19.125 Update-Rate Sweep

Evaluate:

\[ f_s = 1,\, 5,\, 10,\, 20~\mathrm{Hz} \]

for applicable sensors.

This identifies the minimum update rate needed to meet navigation requirements.

4.2.19.126 Noise Sweep

Scale measurement noise:

\[ \sigma = \alpha\sigma_0. \]

Then evaluate performance sensitivity.

This can help translate mission requirements into sensor-noise requirements.

4.2.19.127 Geometry Sweep

For optical navigation, vary:

• target range,
• viewing angle,
• feature distribution,
• lighting geometry.

For GNSS, vary satellite geometry.

This determines whether performance depends excessively on favourable geometry.

4.2.19.128 Performance Maps

Instead of reporting one scalar result, build maps such as:

\[ \mathrm{RMSE} = f( \rho,\, \sigma_z ). \]

For example:

\[ \boxed{ \text{Range} \times \text{Measurement Noise} \rightarrow \text{Position Accuracy}. } \]

Performance maps can be highly useful for mission planning.

4.2.19.129 Convergence Maps

Similarly:

\[ T_{\text{conv}} = f( e_0,\, \sigma_z,\, \text{geometry} ). \]

This shows where the estimator converges quickly and where it becomes marginal.

4.2.19.130 Probability-of-Success Maps

A Monte Carlo map may show:

\[ P_{\text{success}} = f( \text{range}, \text{noise}, \text{initial error} ). \]

This provides a clear operational view of estimator robustness.

4.2.19.131 Performance Versus Sensor Configuration

Compare configurations such as:

\[ S_1, \]

\[ S_1+S_2, \]

\[ S_1+S_3. \]

Evaluate:

• accuracy,
• convergence,
• consistency,
• availability.

This reveals the actual value contributed by each sensor.

4.2.19.132 Information Contribution of a Sensor

A sensor contributes information approximately through:

\[ \mathbf H^T \mathbf R^{-1} \mathbf H. \]

Therefore the value of adding a sensor depends on both:

\[ \mathbf H \]

and

\[ \mathbf R. \]

A highly accurate sensor that measures a state direction already well constrained may add less value than a moderate sensor that constrains a previously weak direction.

4.2.19.133 Sensor Ablation Study

An ablation study removes one sensor at a time.

Compare:

\[ \text{Full sensor set} \]

with:

\[ \text{Full set} - S_i. \]

Measure resulting performance degradation.

This quantifies sensor importance and supports redundancy analysis.

4.2.19.134 Algorithm Comparison

Different estimators may be compared using identical datasets and uncertainty conditions.

For example:

\[ \text{EKF}, \quad \text{ESKF}, \quad \text{UKF}, \quad \text{PF}. \]

Metrics should include more than RMSE:

• computation,
• robustness,
• consistency,
• convergence,
• numerical stability.

4.2.19.135 Fair Estimator Comparison

A comparison is only meaningful if estimators use:

• the same sensor data,
• the same truth trajectory,
• equivalent process models,
• comparable noise assumptions,
• equivalent initialization.

Otherwise differences may reflect setup rather than algorithm quality.

4.2.19.136 EKF Performance Assessment

For the EKF, pay particular attention to:

• linearization error,
• Jacobian correctness,
• initial-error sensitivity,
• covariance consistency.

Large nonlinearities may produce optimistic covariance even when state estimates appear reasonable.

4.2.19.137 ESKF Performance Assessment

For the ESKF, examine:

• nominal-state accuracy,
• error-state magnitude,
• reset consistency,
• quaternion normalization,
• covariance reset transformation,
• bias convergence.

The error state should generally remain small enough for the chosen linearization assumptions.

4.2.19.138 UKF Performance Assessment

For the UKF, examine:

• sigma-point scaling,
• covariance positivity,
• quaternion/manifold handling,
• computational cost,
• nonlinear propagation accuracy.

UKF tuning includes not only \(Q\) and \(R\), but also sigma-point parameters.

4.2.19.139 Particle-Filter Performance Assessment

For particle filters, evaluate:

• number of particles,
• effective sample size,
• resampling frequency,
• weight degeneracy,
• computational cost,
• multimodal tracking.

Accuracy alone does not indicate whether the particle representation remains healthy.

4.2.19.140 Computational Complexity

Approximate complexity often grows with state dimension:

\[ n_x. \]

Kalman-type covariance operations may scale roughly with:

\[ O(n_x^3) \]

for matrix inversions or decompositions, depending on implementation.

Particle-filter cost scales approximately with particle count:

\[ O(N_p). \]

Thus algorithm choice influences real-time feasibility.

4.2.19.141 Numerical Stability

Monitor:

• covariance symmetry,
• positive semidefiniteness,
• matrix-conditioning,
• Cholesky failures,
• NaN/Inf values.

A filter that produces accurate estimates but occasionally loses numerical validity is not flight-ready.

4.2.19.142 Covariance Symmetry Check

Numerically:

\[ \mathbf P \]

should satisfy

\[ \mathbf P \approx \mathbf P^T. \]

A diagnostic is:

\[ \boxed{ e_{sym} = \| \mathbf P-\mathbf P^T \|. } \]

Large symmetry error indicates numerical implementation problems.

4.2.19.143 Positive-Semidefinite Check

Eigenvalues should satisfy:

\[ \lambda_i(\mathbf P)\ge0. \]

Substantial negative eigenvalues indicate invalid covariance.

This check is particularly important after nonlinear updates and error-state resets.

4.2.19.144 Condition Number

A matrix condition number such as:

\[ \kappa(\mathbf P) = \frac{ \lambda_{\max} }{ \lambda_{\min} } \]

can reveal numerical sensitivity.

Very large condition numbers may indicate poor scaling or weakly observable state directions.

4.2.19.145 State Scaling

States may have very different numerical magnitudes.

For example:

\[ r \sim10^6, \]

while:

\[ b_g \sim10^{-5}. \]

Poor scaling can degrade numerical conditioning.

State normalization or square-root filtering can help.

4.2.19.146 Square-Root Filter Assessment

A square-root filter propagates:

\[ \mathbf S \]

such that:

\[ \mathbf P = \mathbf S\mathbf S^T. \]

Performance assessment should verify:

• valid factorization,
• numerical robustness,
• equivalence with covariance-form estimates.

Square-root methods can improve stability for ill-conditioned problems.

4.2.19.147 Sensitivity to Initial Covariance

Repeat simulations with:

\[ \mathbf P_0 \]

scaled by several factors.

For example:

\[ 0.1\mathbf P_0, \quad \mathbf P_0, \quad 10\mathbf P_0. \]

Evaluate whether filter behaviour remains acceptable.

This tests robustness to imperfect initialization knowledge.

4.2.19.148 Sensitivity to Initial State

Similarly vary:

\[ \hat{\mathbf x}_0. \]

Determine the convergence basin.

Large sensitivity may indicate:

• poor nonlinear robustness,
• weak observability,
• inappropriate linearization,
• insufficient initialization logic.

4.2.19.149 Sensitivity to Bias Initialization

Vary:

\[ \hat{\mathbf b}_0. \]

Measure:

\[ T_{\text{bias conv}}, \]

and resulting state error.

This is especially important for inertial navigation.

4.2.19.150 Filter Tuning Across Sensor Faults

A robust tuning should remain meaningful after sensor loss.

When one sensor is removed:

\[ \mathbf H \]

and information content change.

The estimator should not remain tuned as though the missing sensor were still contributing.

Mode-specific covariance growth and degraded performance should be assessed.

4.2.19.151 Data-Driven Tuning

Recorded sensor data can be used to estimate:

• noise distributions,
• correlations,
• bias drift,
• quality dependence,
• dropout statistics.

This often produces more realistic tuning than using datasheets alone.

4.2.19.152 Hardware-in-the-Loop Tuning

HIL testing captures effects often missing from simulation:

• timestamp jitter,
• packet delay,
• digital quantization,
• sensor startup,
• processor timing,
• interface faults.

Final tuning should ideally be checked against hardware behaviour where possible.

4.2.19.153 Software-in-the-Loop Tuning

SIL is ideal for:

• large Monte Carlo campaigns,
• parameter sweeps,
• fault injection,
• reproducible comparison.

It enables systematic tuning before hardware becomes available.

4.2.19.154 Recorded-Data Replay

A useful test method is replaying identical sensor data through different tuning configurations.

Then estimator differences arise only from filter configuration.

This provides a fair tuning comparison.

4.2.19.155 Truth-Reference Testing

When a high-accuracy external reference is available, compare:

\[ \hat{\mathbf x} \]

against:

\[ \mathbf x_{\text{ref}}. \]

The reference should be substantially more accurate than the estimator under test.

Otherwise apparent error may actually come from the reference.

4.2.19.156 Independent Reference Principle

A validation reference should be as independent as possible from the estimator's own sensors.

If the same sensor contributes to both the estimator and the truth reference, the comparison may be overly favourable.

4.2.19.157 Performance Requirement Traceability

Each tuning choice should connect to a requirement.

For example:

\[ R_{\text{camera}} \]

may be justified by measured pixel noise and required pose accuracy.

Similarly:

\[ Q_{\text{bias}} \]

may be justified by IMU characterization.

This improves design traceability.

4.2.19.158 Tuning Documentation

A good tuning record should include:

• parameter name,
• numerical value,
• units,
• source,
• physical interpretation,
• test evidence,
• applicable mission phase,
• date/version.

This prevents tuning from becoming undocumented trial and error.

4.2.19.159 Version Control of Tuning

Changes to:

\[ Q, \quad R, \quad P_0, \]

or gating thresholds should be version controlled.

A small change can materially alter filter behaviour.

Regression testing should accompany tuning changes.

4.2.19.160 Filter Performance Regression Tests

A regression suite should verify that software changes do not unexpectedly degrade:

• RMSE,
• NIS,
• NEES,
• convergence time,
• rejection rate,
• numerical stability.

This is particularly important after modifying measurement models or Jacobians.

4.2.19.161 Automated Performance Metrics

Simulation scripts should automatically compute:

\[ \mathrm{RMSE}, \]

\[ \mathrm{MAE}, \]

\[ \mathrm{NIS}, \]

\[ \mathrm{NEES}, \]

\[ T_{\text{conv}}, \]

rather than relying only on visual inspection.

This enables objective comparison across versions.

4.2.19.162 Plots Required for Filter Assessment

Useful plots include:

• truth vs estimate,
• estimation error,
• \(3\sigma\) covariance bounds,
• innovation,
• normalized innovation,
• NIS,
• NEES,
• bias states,
• covariance states,
• sensor-validity flags,
• Kalman gains,
• measurement rejection.

These plots together reveal much more than state trajectories alone.

4.2.19.163 State Error with Covariance Bounds

Plot:

\[ \tilde x_i \]

together with:

\[ \pm3\sqrt{P_{ii}}. \]

This directly shows whether reported uncertainty matches actual error.

It is one of the most informative estimator plots.

4.2.19.164 Innovation Plot

Plot each component of:

\[ \mathbf r_k. \]

Look for:

• nonzero mean,
• drift,
• periodic patterns,
• abrupt spikes,
• changing variance.

These signatures often reveal modelling problems quickly.

4.2.19.165 Normalized Residual Plot

Define:

\[ \tilde r_i = \frac{ r_i }{ \sqrt{S_{ii}} }. \]

For a well-tuned approximately Gaussian scalar residual:

\[ \tilde r_i \]

should typically remain within a few standard deviations.

This makes residuals from different sensors easier to compare.

4.2.19.166 NIS Time History

Plot:

\[ \mathrm{NIS}_k \]

against chi-square limits.

Occasional excursions are expected.

Persistent high or low values indicate covariance mismatch.

4.2.19.167 NEES Time History

In simulation, plot:

\[ \mathrm{NEES}_k \]

against expected statistical bounds.

This reveals periods of overconfidence or conservatism.

It is particularly valuable after manoeuvres and sensor transitions.

4.2.19.168 Monte Carlo Envelope Plot

Across many runs, plot percentile envelopes.

For example:

\[ 5\%, 50\%, 95\%. \]

This shows how performance varies statistically instead of showing only one realization.

4.2.19.169 Error Histogram

Plot the distribution of estimation errors.

Compare against a Gaussian model if Gaussian assumptions are used.

Heavy tails may suggest that:

• outliers are present,
• noise is non-Gaussian,
• covariance is incomplete.

4.2.19.170 Q-Q Plot

A quantile-quantile plot can compare residual distribution against a Gaussian distribution.

Significant deviation in the tails may indicate heavy-tailed measurement errors.

This may motivate robust fusion.

4.2.19.171 Innovation Spectrum

Frequency-domain analysis can reveal periodic residual structure.

For example, a spectral peak may indicate:

• vibration,
• wheel disturbance,
• periodic sensor error,
• thermal cycling.

Such effects may be hidden in time-domain statistics.

4.2.19.172 Autocorrelation Plot

Plot:

\[ C_r(\tau). \]

A well-modelled innovation should show little correlation at nonzero lag.

Persistent correlation indicates missing temporal structure.

4.2.19.173 Performance Before and After Tuning

A good tuning report should show:

\[ \text{Baseline} \]

versus

\[ \text{Tuned Configuration}. \]

Compare:

• RMSE,
• NIS,
• NEES,
• convergence,
• rejection rate.

This demonstrates objectively what changed.

4.2.19.174 Avoid Tuning on One Dataset

If the filter is tuned on one dataset and evaluated on the same dataset, performance may be overly optimistic.

Use separate:

\[ \boxed{ \text{Tuning Dataset} } \]

and

\[ \boxed{ \text{Validation Dataset}. } \]

This principle is similar to training and testing separation in machine learning.

4.2.19.175 Cross-Scenario Validation

Tune using several scenarios and validate on different ones.

For example:

• different initial states,
• different trajectories,
• different geometry,
• different noise realizations.

This reduces overfitting of estimator tuning to one scenario.

4.2.19.176 Over-Tuned Filter

A filter may perform extremely well on one exact simulation because \(Q\) and \(R\) have been tuned to that scenario.

But small changes may cause large degradation.

This is estimator overfitting.

A robust filter should perform well over a range of conditions.

4.2.19.177 Robust Tuning

Robust tuning seeks values that provide acceptable performance across uncertainty rather than the best performance in one nominal case.

Conceptually:

\[ \boxed{ \text{Robust Tuning} = \text{Good Performance Across Many Conditions}. } \]

This is usually more valuable for flight systems than a narrowly optimal configuration.

4.2.19.178 Parameter Margin

If acceptable tuning exists only for:

\[ 0.99Q_0 < Q < 1.01Q_0, \]

the system is very sensitive.

A healthier design may tolerate:

\[ 0.5Q_0 < Q < 2Q_0. \]

This tuning margin is an important robustness indicator.

4.2.19.179 Sensitivity Gradient

For metric J,

\[ \frac{\partial J}{\partial q_i} \]

measures sensitivity to a tuning parameter.

Large magnitude means small parameter changes strongly affect performance.

Sensitivity gradients can guide which parameters need accurate characterization.

4.2.19.180 Uncertainty Budget

Estimator performance can be decomposed conceptually into contributions from:

• sensor noise,
• bias,
• timing,
• geometry,
• model uncertainty,
• initial uncertainty.

This forms a navigation uncertainty budget.

The budget helps identify which improvement would provide the largest benefit.

4.2.19.181 Measurement Error Budget

For a sensor:

\[ \sigma_{\text{total}}^2 = \sigma_{\text{noise}}^2 + \sigma_{\text{cal}}^2 + \sigma_{\text{timing}}^2 + \sigma_{\text{geometry}}^2 +\cdots \]

when independence assumptions are reasonable.

This can provide an initial basis for \(R\).

4.2.19.182 Process Error Budget

Similarly:

\[ Q_{\text{total}} = Q_{\text{environment}} + Q_{\text{actuator}} + Q_{\text{model}} + Q_{\text{bias}}. \]

This makes process-noise tuning traceable to physical uncertainty sources.

4.2.19.183 Performance Allocation

A mission-level navigation requirement can be allocated among error sources.

For example:

\[ \sigma_{\text{nav}}^2 = \sigma_{\text{sensor}}^2 + \sigma_{\text{model}}^2 + \sigma_{\text{timing}}^2 +\cdots \]

approximately under independence assumptions.

This can guide sensor selection and model fidelity.

4.2.19.184 Tuning and Sensor Selection

If performance remains above requirement even after reasonable tuning, the problem may not be the estimator.

It may indicate that:

• sensor accuracy is insufficient,
• update rate is too low,
• geometry is poor,
• observability is inadequate.

Thus tuning analysis can reveal hardware requirements.

4.2.19.185 Tuning and Dynamic-Model Fidelity

If innovation remains systematically correlated, increasing model fidelity may be more effective than changing \(Q\).

Possible improvements include:

• higher-order orbital dynamics,
• better disturbance models,
• actuator models,
• sensor lever arms,
• thermal effects.

4.2.19.186 Tuning and State Augmentation

Persistent residual structure may suggest a missing state.

For example:

\[ \text{innovation drift} \]

may indicate an unmodelled sensor bias.

Adding the bias state can be preferable to indefinitely inflating \(R\).

Again, observability must be verified.

4.2.19.187 Performance Under Non-Gaussian Noise

Gaussian covariance metrics may not fully describe heavy-tailed sensor behaviour.

Then evaluate:

• percentile errors,
• rejection frequency,
• robust loss,
• tail probability.

A filter can have acceptable RMS error but unacceptable tail behaviour.

4.2.19.188 Tail Risk

For safety-related navigation, consider:

\[ P( |e|>e_{\text{critical}} ). \]

This probability may matter more than average RMSE.

Monte Carlo or importance sampling may be needed to estimate rare-event behaviour.

4.2.19.189 Fault Injection Testing

Inject known faults such as:

• step bias,
• ramp drift,
• frozen data,
• noise increase,
• packet dropout,
• timing offset.

Measure:

\[ P_D, \quad P_{FA}, \quad T_D. \]

This evaluates not only estimation performance but also fault-tolerant fusion.

4.2.19.190 Step-Bias Test

Inject:

\[ b(t) = \begin{cases} 0,&t

Observe:

• innovation response,
• state error,
• detector response,
• sensor isolation,
• recovery.

This is a standard FDIR test.

4.2.19.191 Ramp-Fault Test

Inject:

\[ b(t) = d(t-t_f) \]

after fault onset.

Slow ramps are harder to detect than steps.

They are useful for evaluating CUSUM or trend-based monitors.

4.2.19.192 Noise-Increase Test

Change:

\[ R_{\text{actual}} \rightarrow \alpha R_{\text{actual}} \]

with:

\[ \alpha>1. \]

Observe whether the estimator and health monitor recognize degraded measurement quality.

4.2.19.193 Frozen-Sensor Test

Set:

\[ z_k=z_{k_f} \]

for

\[ k>k_f. \]

Test whether temporal-validity logic identifies the stale data.

4.2.19.194 Dropout Test

Remove measurements for:

\[ T_d. \]

Observe covariance growth and recovery.

The filter should not continue reducing covariance as though measurements were still present.

4.2.19.195 False Measurement Test

For perception sensors, inject an incorrect target or feature association.

This tests whether residual gating and robust fusion prevent catastrophic state corruption.

4.2.19.196 Performance Under Reacquisition

After dropout, restore the sensor.

Measure:

\[ T_{\text{reacquire}}, \]

\[ T_{\text{reconverge}}. \]

Also verify that the first returning measurement does not cause an unsafe state jump.

4.2.19.197 Performance Under Sensor Switching

Switch between redundant sensors.

Check:

• state continuity,
• covariance continuity,
• residual behaviour,
• bias transients.

A healthy sensor switch should not create artificial navigation jumps.

4.2.19.198 Bias Mismatch Between Redundant Sensors

Suppose:

\[ z_A=x+b_A, \]

\[ z_B=x+b_B. \]

Switching between them can introduce a step:

\[ \Delta z=b_B-b_A. \]

Calibration or transition logic may be required.

4.2.19.199 Performance After Reconfiguration

After isolating a sensor, assess the new estimator configuration.

Important questions:

• Which states remain observable?
• How quickly does covariance grow?
• Does accuracy remain within requirement?
• Is navigation still operationally valid?

This connects tuning directly with degraded modes.

4.2.19.200 Relative-Navigation Performance Example

For general spacecraft relative navigation, useful performance metrics may include:

\[ \mathrm{RMSE}_{r_{rel}}, \]

\[ \mathrm{RMSE}_{v_{rel}}, \]

relative attitude error,

and relative angular-rate error.

These are only application examples; the same evaluation framework applies to absolute navigation and attitude determination.

4.2.19.201 Position Error Norm

For relative or absolute position:

\[ \boxed{ e_r = \| \mathbf r-\hat{\mathbf r} \|. } \]

Report:

• RMS,
• maximum,
• percentiles,
• covariance consistency.

4.2.19.202 Velocity Error Norm

Similarly:

\[ \boxed{ e_v = \| \mathbf v-\hat{\mathbf v} \|. } \]

Velocity performance is often especially important for control and propagation.

4.2.19.203 Attitude Error

For quaternions:

\[ \delta q = q_{\text{true}} \otimes \hat q^{-1}. \]

The attitude-error angle can be computed from:

\[ \boxed{ \delta\theta = 2\cos^{-1} ( |\delta q_0| ). } \]

Using the absolute scalar part handles the quaternion sign equivalence.

4.2.19.204 Angular-Rate Error

Define:

\[ \boxed{ e_\omega = \| \boldsymbol\omega - \hat{\boldsymbol\omega} \|. } \]

This should be assessed separately from attitude error.

A filter can have good attitude but poor instantaneous rate estimation.

4.2.19.205 Bias-Estimation Error

For gyro bias:

\[ e_{b_g} = \| \mathbf b_g-\hat{\mathbf b}_g \|. \]

For accelerometer bias:

\[ e_{b_a} = \| \mathbf b_a-\hat{\mathbf b}_a \|. \]

Bias performance may determine long-term navigation drift.

4.2.19.206 Sensor Calibration Error Metrics

For estimated scale factors or misalignments:

\[ e_s = s-\hat s, \]

\[ e_{\theta_s} = \| \delta\boldsymbol\theta_s - \delta\hat{\boldsymbol\theta}_s \|. \]

These states may converge much more slowly than navigation states.

4.2.19.207 Time-Offset Estimation Error

For a clock-bias state:

\[ e_t = \delta t-\widehat{\delta t}. \]

This should be assessed when timing is estimated online.

Millisecond-level timing errors can matter significantly in fast dynamics.

4.2.19.208 Performance by State Group

Useful groups include:

\[ \text{Translation}, \]

\[ \text{Attitude}, \]

\[ \text{Biases}, \]

\[ \text{Calibration}, \]

\[ \text{Timing}. \]

A filter may perform well in one group and poorly in another.

Therefore one overall scalar score can hide important deficiencies.

4.2.19.209 Performance by Axis

Do not always report only vector norm.

For position, evaluate:

\[ e_x, \quad e_y, \quad e_z. \]

One axis may be much weaker because of sensor geometry.

Norm-only reporting can hide directional weakness.

4.2.19.210 Radial and Transverse Performance

In orbital or relative navigation, it can be useful to resolve error into meaningful directions such as:

• radial,
• along-track,
• cross-track.

This reveals geometry-dependent behaviour that inertial Cartesian metrics may hide.

4.2.19.211 Covariance Projection

For any unit direction \(\mathbf u\),

\[ \boxed{ \sigma_u^2 = \mathbf u^T \mathbf P \mathbf u. } \]

This allows uncertainty to be evaluated along physically meaningful directions.

For example, line-of-sight uncertainty can be different from lateral uncertainty.

4.2.19.212 Error Ellipsoid

For 3-D position covariance:

\[ ( \mathbf r-\hat{\mathbf r} )^T \mathbf P_r^{-1} ( \mathbf r-\hat{\mathbf r} ) = c. \]

The ellipsoid axes reveal directional confidence.

Plotting the ellipsoid can help interpret sensor geometry.

4.2.19.213 Confidence Ellipsoid Volume

A metric proportional to:

\[ \sqrt{ \det( \mathbf P_r ) } \]

represents uncertainty volume.

This can be useful for comparing configurations.

However, directional requirements should still be checked separately.

4.2.19.214 Performance Under Sensor Geometry Change

As geometry changes, covariance shape may rotate.

Therefore monitor:

\[ \mathbf P(t) \]

rather than only steady-state scalar values.

Time-varying covariance behaviour is expected in many spacecraft problems.

4.2.19.215 Tuning and Measurement Jacobian

An incorrect measurement Jacobian

\[ \mathbf H \]

can mimic poor tuning.

Symptoms include:

• inconsistent innovations,
• slow convergence,
• unexpected cross-coupling.

Before adjusting \(Q\) or \(R\), verify analytical or numerical Jacobians.

4.2.19.216 Jacobian Verification

Compare analytical Jacobian:

\[ H_{analytical} \]

with finite-difference approximation:

\[ H_{FD} \approx \frac{ h(x+\epsilon e_i)-h(x) }{ \epsilon }. \]

They should agree within numerical tolerance.

This is a fundamental EKF verification step.

4.2.19.217 Dynamic Jacobian Verification

Similarly verify:

\[ F = \frac{\partial f}{\partial x}. \]

Incorrect state-transition Jacobians can produce wrong covariance even when the nominal state propagates correctly.

4.2.19.218 Covariance vs State Accuracy Debugging

If state estimate looks good but covariance is wrong, inspect:

• \(F\),
• \(H\),
• \(Q\),
• \(R\),
• reset transformations,
• time discretization.

State accuracy alone does not validate covariance propagation.

4.2.19.219 Measurement-Model Residual Debugging

If one sensor has persistently biased innovation, inspect:

• frame convention,
• sign convention,
• sensor extrinsics,
• timing,
• lever arm,
• calibration,
• expected measurement equation.

Do not immediately retune \(R\).

4.2.19.220 Filter-Tuning Workflow

A practical workflow is:

\[ \boxed{ \begin{array}{c} \text{Build Physically Correct Models}\\ \downarrow\\ \text{Characterize Sensors}\\ \downarrow\\ \text{Initialize }P_0,Q,R\\ \downarrow\\ \text{Run Nominal Simulation}\\ \downarrow\\ \text{Inspect Errors + Innovations}\\ \downarrow\\ \text{Check NIS / NEES}\\ \downarrow\\ \text{Perform Sensitivity Sweeps}\\ \downarrow\\ \text{Run Monte Carlo}\\ \downarrow\\ \text{Test Faults + Dropouts}\\ \downarrow\\ \text{Validate on Independent Data} \end{array} } \]

This is much more reliable than manual trial-and-error tuning.

4.2.19.221 Step 1 — Verify Deterministic Models First

Before tuning stochastic parameters:

• verify units,
• verify coordinate frames,
• verify quaternion convention,
• verify measurement equations,
• verify time handling,
• verify sensor alignment,
• verify state propagation.

Otherwise \(Q\) and \(R\) may be used to hide deterministic errors.

4.2.19.222 Step 2 — Establish Sensor Noise Models

Use:

• datasheets,
• bench tests,
• recorded data,
• calibration campaigns.

Estimate realistic:

\[ R. \]

Include correlations and operating-condition dependence where relevant.

4.2.19.223 Step 3 — Establish Process Uncertainty

Identify uncertain dynamics.

Examples:

• external disturbances,
• actuator execution,
• bias evolution,
• model approximations.

Map these into:

\[ Q. \]

4.2.19.224 Step 4 — Validate Initialization

Before steady-state tuning, ensure:

\[ P_0 \]

matches the actual initial error distribution.

Otherwise convergence behaviour may distort later tuning results.

4.2.19.225 Step 5 — Inspect Raw Innovation

Plot:

\[ r_k. \]

Check:

• mean,
• drift,
• variance,
• correlation.

Large structural residual errors should be fixed before statistical tuning.

4.2.19.226 Step 6 — Inspect NIS

Compute:

\[ \mathrm{NIS}. \]

If systematically high, investigate overconfidence or model mismatch.

If systematically low, investigate excessive covariance.

4.2.19.227 Step 7 — Inspect NEES

In simulation, compute:

\[ \mathrm{NEES}. \]

Check whether state errors are statistically compatible with covariance.

This helps distinguish accuracy problems from covariance problems.

4.2.19.228 Step 8 — Perform Parameter Sweeps

Vary:

\[ Q, \quad R, \quad P_0. \]

Observe sensitivity.

Identify regions where performance is robust.

4.2.19.229 Step 9 — Run Monte Carlo

Vary uncertain parameters across realistic distributions.

Measure:

• success rate,
• error percentiles,
• NIS/NEES statistics,
• convergence time.

This is the primary statistical validation stage.

4.2.19.230 Step 10 — Test Off-Nominal Conditions

Include:

• dropouts,
• outliers,
• faults,
• timing offsets,
• geometry degradation,
• model mismatch.

A filter should not be tuned only for nominal operation.

4.2.19.231 Step 11 — Validate Against Independent Data

Use data not used during tuning.

This verifies that the tuning generalizes.

4.2.19.232 Step 12 — Freeze and Document Baseline Tuning

Once validated:

\[ Q, \quad R, \quad P_0, \]

thresholds and mode-dependent settings should become a controlled baseline.

Subsequent changes require regression testing.

4.2.19.233 Example Tuning Diagnosis: High NIS and High NEES

Suppose:

\[ \mathrm{NIS} \gg m \]

and

\[ \mathrm{NEES} \gg n. \]

This suggests the filter is overconfident.

Possible causes:

• \(Q\) too small,
• \(R\) too small,
• model errors,
• unmodelled bias.

The correct response is diagnosis, not automatically increasing both matrices.

4.2.19.234 Example: High NIS but Acceptable NEES

If NIS is high but state consistency is acceptable, the problem may be localized to the measurement model or \(R\).

Possible issues include:

• sensor covariance underestimated,
• measurement outliers,
• wrong measurement Jacobian.

4.2.19.235 Example: Acceptable NIS but High NEES

This can occur when measurements appear consistent but the state covariance is still too optimistic.

Possible causes include:

• unobservable state directions,
• state-transition covariance error,
• incorrect process noise,
• covariance-reset mistakes.

This is why both metrics are valuable.

4.2.19.236 Example: Low NIS and Low NEES

Both being systematically low indicates excessive conservatism.

Possible causes:

• \(Q\) too large,
• \(R\) too large,
• \(P_0\) too large.

The estimator may meet integrity requirements but fail unnecessarily strict accuracy requirements.

4.2.19.237 Example: Good RMSE but High NEES

The filter estimate may be close to truth, but covariance is even smaller.

Therefore the filter is overconfident.

This can be dangerous because future gating and integrity logic depend on covariance.

4.2.19.238 Example: Poor RMSE but Correct NEES

The estimator may be statistically honest but simply not accurate enough.

This indicates insufficient information or excessive noise rather than inconsistency.

Potential solutions include:

• better sensor,
• higher update rate,
• improved geometry,
• improved model.

4.2.19.239 Example: Biased Innovation

If:

\[ E[r]\neq0, \]

check deterministic issues first.

Likely causes:

• sensor bias,
• timing,
• frame error,
• calibration error,
• model bias.

Changing \(R\) alone is rarely the correct solution.

4.2.19.240 Example: Correlated Innovation

If residual autocorrelation is strong, inspect dynamic modelling.

Possible causes:

• missing bias state,
• unmodelled disturbance,
• filtered sensor output,
• time offset,
• inadequate process model.

4.2.19.241 Example: Frequent Gating Rejection

Possible causes:

• gate too tight,
• \(R\) underestimated,
• \(Q\) underestimated,
• sensor outliers,
• model mismatch,
• poor initialization.

Review the entire consistency chain.

4.2.19.242 Example: Slow Bias Convergence

Possible causes:

• bias covariance too small,
• bias process noise too small,
• weak observability,
• insufficient external reference,
• poor excitation.

Do not assume the bias model alone is the cause.

4.2.19.243 Example: Excessively Noisy Estimate

Possible causes:

• \(R\) too small,
• \(Q\) too large,
• measurement noise underestimated,
• update rate too high relative to correlated noise.

The estimator may be following sensor noise too aggressively.

4.2.19.244 Example: Estimate Lags Dynamics

Possible causes:

• \(Q\) too small,
• \(R\) too large,
• process model too rigid,
• update latency.

The filter may be over-trusting prediction or underusing measurements.

4.2.19.245 Example: Covariance Shrinks but Error Does Not

This is a classic sign of inconsistency.

Potential causes:

• wrong \(Q\),
• wrong \(R\),
• unobservable state,
• incorrect Jacobian,
• deterministic model error.

This situation should be treated seriously.

4.2.19.246 Example: Covariance Continues Growing

Possible causes:

• sensor dropout,
• weak observability,
• process noise too large,
• measurement uncertainty too large,
• rejected measurements.

This may be correct behaviour rather than a bug.

The question is whether remaining navigation performance is acceptable.

4.2.19.247 Tuning in Information Form

The measurement information contribution is:

\[ \mathbf Y_z = \mathbf H^T \mathbf R^{-1} \mathbf H. \]

Reducing \(R\) increases information.

Increasing process noise reduces confidence in propagated information.

Thus tuning can also be interpreted as managing information flow.

4.2.19.248 Covariance Is Not Just a Plot

Covariance affects:

• Kalman gain,
• innovation covariance,
• gating,
• navigation validity,
• FDIR,
• guidance permissions,
• sensor reweighting.

Therefore covariance quality is a system-level requirement.

4.2.19.249 Navigation Performance Must Be State-Specific

Different states may have different requirements.

For example:

\[ \sigma_r \]

may satisfy the requirement while:

\[ \sigma_v \]

does not.

A single overall RMS number is insufficient.

4.2.19.250 Performance Must Be Phase-Specific

Different mission phases require different accuracy and update rates.

A navigation solution adequate for coarse orbit knowledge may be insufficient for precision relative navigation.

Therefore performance should be evaluated relative to the current operational requirement.

4.2.19.251 Performance Must Be Probabilistic

Because sensor noise and disturbances are random, filter performance is a distribution.

Thus:

\[ \boxed{ \text{One Simulation Run} \neq \text{Filter Performance Characterization}. } \]

Monte Carlo or equivalent statistical analysis is necessary.

4.2.19.252 Performance Must Be Reproducible

Record:

• random seeds,
• software version,
• tuning version,
• sensor configuration,
• truth model,
• scenario parameters.

This allows results to be reproduced and compared.

4.2.19.253 Performance Must Be Traceable

Every reported metric should identify:

• scenario,
• time interval,
• state definition,
• reference frame,
• units,
• sample population.

This prevents ambiguous statements such as:

“Position error is 2 cm.”

Without context, such a number is incomplete.

4.2.19.254 Reporting RMSE Properly

Specify whether RMSE is:

• per axis,
• vector norm,
• steady-state only,
• entire trajectory,
• one run,
• Monte Carlo mean.

For example:

\[ \mathrm{RMSE}_{x} \]

is not the same as:

\[ \mathrm{RMSE}_{\|\mathbf r\|}. \]

4.2.19.255 Reporting Covariance Properly

Specify whether uncertainty is:

\[ 1\sigma, \quad 2\sigma, \quad 3\sigma. \]

Also specify frame and axis.

For example:

\[ 3\sigma_{radial} \]

is clearer than “position uncertainty.”

4.2.19.256 Reporting Monte Carlo Results

A good summary may include:

• number of runs,
• parameter distributions,
• success rate,
• median,
• \(95^\text{th}\) percentile,
• \(99^\text{th}\) percentile,
• maximum observed value.

This makes statistical conclusions transparent.

4.2.19.257 Statistical Confidence in Monte Carlo

A Monte Carlo result itself has uncertainty.

If only:

\[ N=10 \]

runs are performed, estimating a

\[ 99\% \]

success probability is not meaningful.

Large probability claims require sufficient sample count or specialized rare-event analysis.

4.2.19.258 Monte Carlo Sample Size

The required sample count depends on the probability being estimated.

For ordinary performance characterization, hundreds or thousands of runs may be useful.

For very rare failures, standard Monte Carlo may become computationally inefficient.

4.2.19.259 Rare-Event Analysis

For low-probability critical events, methods such as:

• importance sampling,
• subset simulation,
• worst-case analysis,

may be more efficient than brute-force Monte Carlo.

This becomes important for high-integrity navigation systems.

4.2.19.260 Confidence Intervals on Performance Metrics

Report uncertainty in estimated statistics.

For example:

\[ \hat p \pm \Delta p. \]

Similarly, confidence intervals can be computed for mean RMSE or failure probability.

This prevents over-interpreting limited simulation samples.

4.2.19.261 Comparing Two Tuning Configurations

Suppose configurations A and B produce:

\[ \mathrm{RMSE}_A < \mathrm{RMSE}_B \]

but:

\[ \mathrm{NEES}_A \gg n \]

while:

\[ \mathrm{NEES}_B \approx n. \]

Configuration A is more accurate in the tested run but less statistically trustworthy.

Therefore tuning selection should consider the complete performance picture.

4.2.19.262 Filter Tuning Is an Engineering Trade

Typical trade-offs include:

\[ \text{Responsiveness} \leftrightarrow \text{Noise Suppression}, \]

\[ \text{Accuracy} \leftrightarrow \text{Robustness}, \]

\[ \text{Fault Sensitivity} \leftrightarrow \text{False Alarm Rate}, \]

\[ \text{Model Trust} \leftrightarrow \text{Measurement Trust}. \]

There is rarely one universally optimal setting.

4.2.19.263 Avoid Manual Tuning Without Statistics

Visual tuning can be useful for initial intuition.

But final tuning should be supported by:

\[ \mathrm{NIS}, \]

\[ \mathrm{NEES}, \]

\[ \mathrm{RMSE}, \]

\[ \text{Monte Carlo statistics}. \]

Otherwise the filter may simply be tuned to look aesthetically smooth.

4.2.19.264 Smoothness Is Not a Performance Metric

A smooth estimate can be badly wrong.

A noisy-looking estimate can be statistically correct if the true state itself contains high-frequency motion.

Therefore visual smoothness should never replace quantitative assessment.

4.2.19.265 Tuning and Control Interaction

The controller consumes estimated states.

If the estimator is noisy, control commands may become noisy.

If the estimator lags, control response may be delayed.

Thus filter tuning should also be evaluated in closed-loop simulation.

4.2.19.266 Closed-Loop Performance

In closed loop:

\[ \text{Estimator Error} \rightarrow \text{Control Command} \rightarrow \text{Vehicle Motion} \rightarrow \text{New Measurements}. \]

The estimator and controller interact.

A tuning that performs well offline may behave differently when coupled to control.

4.2.19.267 Navigation-Induced Control Effort

One useful metric is additional control effort caused by estimation noise.

For command:

\[ u_k, \]

evaluate:

\[ \sum_k \|u_k\|. \]

Excessive estimator noise can increase actuator activity unnecessarily.

4.2.19.268 Estimator-Induced Limit Cycling

Noisy state estimates near control thresholds can cause repeated switching or pulsing.

Therefore filter performance should be examined near:

• deadbands,
• mode thresholds,
• actuator minimum impulse logic.

This is an interaction effect rather than a pure estimator metric.

4.2.19.269 Tuning and Guidance Interaction

Guidance decisions may depend on estimated state and covariance.

For example:

\[ \text{proceed} \]

or

\[ \text{hold}. \]

A conservative filter may delay mission progression.

An overconfident filter may allow progression too early.

Therefore covariance tuning influences autonomy.

4.2.19.270 Tuning and Sensor Scheduling

Some sensors may be activated only when uncertainty exceeds a threshold.

For example:

\[ \operatorname{tr}(P) > P_{\text{threshold}} \]

may trigger a high-accuracy sensor.

Thus estimator tuning can influence sensor duty cycle and power consumption.

4.2.19.271 Performance Versus Power

High-rate sensor use can improve navigation but consume more power.

Therefore performance assessment may include:

\[ \boxed{ \text{Navigation Accuracy} \leftrightarrow \text{Sensor Power}. } \]

This is particularly relevant for small spacecraft.

4.2.19.272 Performance Versus Computation

Higher-rate filtering or more complex algorithms may improve performance but increase CPU load.

Thus:

\[ \boxed{ \text{Estimator Sophistication} \leftrightarrow \text{Real-Time Resources}. } \]

A slightly less accurate algorithm may be preferable if it is substantially more robust and computationally efficient.

4.2.19.273 Performance Under Processor Load

Test estimator timing when the flight computer is heavily loaded.

A filter that meets deadlines in isolation may miss them during integrated operation.

Timing jitter can itself degrade sensor fusion.

4.2.19.274 Performance Under Packet Delay

Introduce communication delay.

Verify:

• timestamp handling,
• delayed-update logic,
• state age,
• covariance consistency.

This links back to Section 4.2.14.

4.2.19.275 Performance Under Asynchronous Measurements

Sensors may update independently.

Verify that filter results remain consistent when update ordering changes.

This is especially important when several measurements share nearly identical timestamps.

4.2.19.276 Deterministic Replay Test

Replay exactly the same input sequence twice.

The estimator should produce identical outputs if designed deterministically.

This is useful for regression testing and debugging.

4.2.19.277 Numerical Reproducibility

Different processor architectures or compiler options can produce small floating-point differences.

Safety-critical testing should determine whether such differences remain bounded and operationally insignificant.

4.2.19.278 Filter Performance Under Long Duration

A filter that behaves well for minutes may accumulate errors over hours or days.

Long-duration tests should check:

• numerical drift,
• covariance growth,
• bias tracking,
• model accumulation error.

4.2.19.279 Long-Term Bias Tracking

Slowly varying biases require long simulations.

Short tests may falsely suggest perfect calibration.

Therefore bias-state validation should use time scales appropriate to the physical sensor.

4.2.19.280 Thermal-Cycle Testing

If sensor bias varies with temperature, simulate or test thermal cycles.

Evaluate whether:

• fixed calibration is sufficient,
• bias states track the change,
• covariance remains consistent.

4.2.19.281 Vibration-Condition Testing

Measurement noise can increase under vibration.

A realistic performance campaign should evaluate both quiet and high-vibration conditions.

The same \(R\) may not be appropriate in both.

4.2.19.282 Environmental Robustness

Navigation sensors may be affected by:

• glare,
• shadows,
• albedo,
• multipath,
• radiation,
• target reflectivity.

Performance assessment should cover the environmental envelope relevant to the sensor.

4.2.19.283 Sensor-Degradation Trend

Performance should also be assessed as sensor quality gradually degrades.

For example:

\[ \sigma_z(t)\uparrow. \]

This tests whether adaptive covariance or fault-detection logic responds appropriately.

4.2.19.284 Acceptance Thresholds

For every major metric, define an acceptance criterion.

For example:

\[ \mathrm{RMSE}_r < r_{\max}, \]

\[ \overline{\mathrm{NIS}} \in [\gamma_L,\gamma_U], \]

\[ T_{\text{conv}} < T_{\max}. \]

Without thresholds, testing produces plots but not a clear verification result.

4.2.19.285 Pass/Fail Logic

A test should state whether:

\[ \boxed{ \text{Requirement Satisfied} } \]

or

\[ \boxed{ \text{Requirement Not Satisfied}. } \]

Qualitative statements such as “performance looks good” should not replace requirement-based verification.

4.2.19.286 Margin to Requirement

If a requirement is:

\[ e

define margin:

\[ M = e_{\max} - e_{\text{observed}}. \]

Small positive margin may indicate sensitivity to uncertainty.

Large margin provides robustness.

4.2.19.287 Statistical Margin

For percentile requirements, compare:

\[ e_{99} \]

with:

\[ e_{\text{req}}. \]

Margin can be expressed as:

\[ M_{99} = e_{\text{req}} - e_{99}. \]

This is more meaningful than comparing only mean performance.

4.2.19.288 Performance Summary Table

A useful report may summarize:

This makes performance traceable.

4.2.19.289 Tuning Summary Table

For each filter mode, document:

This is far better than undocumented constants inside code.

4.2.19.290 Common Mistake: Tune Until Estimate Matches Truth

In simulation it is tempting to adjust \(Q\) and \(R\) until the estimate follows truth closely.

But the filter does not know truth in flight.

Tuning should instead represent actual uncertainty.

Otherwise the estimator may be unintentionally fitted to the simulated trajectory.

4.2.19.291 Common Mistake: Reduce \(R\) to Improve Accuracy

Reducing \(R\) makes the filter trust measurements more.

But if sensor noise has not actually decreased, this creates overconfidence.

Thus:

\[ \boxed{ R \text{ should describe the sensor, not desired performance}. } \]

4.2.19.292 Common Mistake: Increase \(Q\) Until Filter Follows Measurements

This may hide dynamic-model errors.

A better process is:

1. verify model,
2. quantify uncertainty,
3. derive \(Q\),
4. test consistency.

4.2.19.293 Common Mistake: Tune One State Independently

Covariance is coupled.

Changing one process-noise term can influence several states through dynamics and cross-covariance.

Therefore always inspect the complete estimator response.

4.2.19.294 Common Mistake: Ignore Correlation

Ignoring measurement or state correlation can produce artificially optimistic covariance.

This is especially important in multi-sensor fusion.

4.2.19.295 Common Mistake: Ignore Sensor Internal Filtering

If a sensor output is already filtered, consecutive measurements may be correlated.

Treating them as independent raw measurements can double-count information.

4.2.19.296 Common Mistake: Evaluate Only Nominal Case

A filter can perform perfectly under nominal conditions and fail immediately under:

• larger initial error,
• sensor dropout,
• geometry degradation.

Robustness testing is essential.

4.2.19.297 Common Mistake: Evaluate Only Mean Error

Mean error can hide:

• rare large errors,
• directional weakness,
• convergence failures.

Always examine tails and percentiles.

4.2.19.298 Common Mistake: Ignore Covariance

A good-looking state trajectory with wrong covariance is not a fully validated estimator.

Covariance drives subsequent decision logic.

4.2.19.299 Common Mistake: Tune Gating Before Covariance

Innovation gating assumes meaningful \(S\).

If \(Q\), \(R\), or \(P\) are wrong, gate behaviour will also be wrong.

Tune and validate covariance first.

4.2.19.300 Common Mistake: Use the Same Noise Realization Repeatedly

Using only one random seed can create misleading confidence.

Monte Carlo should vary independent noise realizations.

4.2.19.301 Common Mistake: Compare Filters with Different Inputs

An algorithm comparison is unfair if one estimator receives better sensor data or a different initialization.

All comparisons should use equivalent conditions.

4.2.19.302 Common Mistake: Ignore Computation

An estimator that requires more time than the available update period cannot be deployed, regardless of estimation accuracy.

Real-time performance belongs in the assessment.

4.2.19.303 Common Mistake: Use One Global \(R\)

Different operating conditions can have very different measurement quality.

A fixed covariance may be inadequate across the full envelope.

Use state-, quality-, or mode-dependent covariance when physically justified.

4.2.19.304 Common Mistake: Let Adaptive Tuning Hide Faults

Adaptive covariance should not keep increasing until every bad measurement appears statistically acceptable.

At some point, abnormal behaviour should trigger fault logic.

Thus:

\[ \boxed{ \text{Adaptation} \neq \text{Unlimited Excuse for Bad Data}. } \]

4.2.19.305 Common Mistake: Ignore Filter Recovery

Testing only normal operation misses important behaviour after:

• sensor return,
• processor restart,
• manoeuvre,
• fault isolation.

Recovery performance should be quantified.

4.2.19.306 Performance Verification Hierarchy

A useful hierarchy is:

\[ \boxed{ \begin{array}{c} \text{Unit Tests}\\ \downarrow\\ \text{Single-Sensor Tests}\\ \downarrow\\ \text{Integrated Filter Tests}\\ \downarrow\\ \text{Monte Carlo}\\ \downarrow\\ \text{Fault Injection}\\ \downarrow\\ \text{Closed-Loop SIL}\\ \downarrow\\ \text{HIL}\\ \downarrow\\ \text{System Validation} \end{array} } \]

Each level catches different classes of problems.

4.2.19.307 Unit-Level Performance Tests

At unit level verify:

• matrix dimensions,
• covariance symmetry,
• measurement residual computation,
• Jacobians,
• discretization,
• update equations.

These tests establish mathematical correctness before system-level tuning.

4.2.19.308 Single-Sensor Validation

Test each sensor model independently.

Verify:

\[ z-h(x) \]

statistics under controlled truth.

This prevents sensor-model errors from being hidden inside multi-sensor fusion.

4.2.19.309 Integrated Filter Validation

Combine sensors and verify:

• proper update ordering,
• covariance reduction,
• cross-state correction,
• asynchronous updates,
• sensor handover.

Only after this should large-scale tuning begin.

4.2.19.310 Closed-Loop Validation

Connect:

\[ \text{Navigation} \rightarrow \text{Guidance} \rightarrow \text{Control} \rightarrow \text{Dynamics}. \]

This reveals whether estimation performance is sufficient for actual vehicle behaviour.

4.2.19.311 Hardware Validation

Finally, test:

• real timing,
• real packet structures,
• actual noise,
• interface delays,
• sensor restart behaviour.

Hardware often reveals assumptions that simulation overlooked.

4.2.19.312 Tuning Freeze Criteria

Tuning should be considered mature when:

• nominal requirements are met,
• statistical consistency is acceptable,
• sensitivity margins are acceptable,
• Monte Carlo performance is stable,
• fault tests pass,
• independent datasets confirm results.

Only then should the tuning be frozen for a controlled baseline.

4.2.19.313 Retuning Triggers

Retuning may be required after changes in:

• sensor hardware,
• sensor firmware,
• estimator model,
• state vector,
• update rate,
• mission environment,
• vehicle mass properties,
• processing algorithm.

Tuning should not be assumed portable across major system changes.

4.2.19.314 Filter Performance Review Questions

A technical review should ask:

1. Where did \(Q\) come from?
2. Where did \(R\) come from?
3. Is \(P_0\) statistically justified?
4. Are innovations zero mean?
5. Are innovations white?
6. Is NIS consistent?
7. Is NEES consistent?
8. What is convergence time?
9. What is worst-case error?
10. What is the \(95^\text{th}\) or \(99^\text{th}\) percentile?
11. What happens after sensor dropout?
12. What happens after reacquisition?
13. What happens if noise is twice expected?
14. What happens under poor geometry?
15. What happens if one sensor is lost?
16. Does the filter remain numerically stable?
17. Does it meet real-time deadlines?

If these questions cannot be answered, tuning is not yet fully validated.

4.2.19.315 Complete Tuning and Assessment Chain

The complete logic is:

\[ \boxed{ \begin{array}{c} \text{Physical Models}\\ +\\ \text{Sensor Characterization}\\ \downarrow\\ P_0,\ Q,\ R\\ \downarrow\\ \text{Filter Implementation}\\ \downarrow\\ \text{Innovation Analysis}\\ \downarrow\\ \text{NIS / NEES}\\ \downarrow\\ \text{Accuracy Metrics}\\ \downarrow\\ \text{Sensitivity Analysis}\\ \downarrow\\ \text{Monte Carlo}\\ \downarrow\\ \text{Fault / Dropout Testing}\\ \downarrow\\ \text{Closed-Loop Verification}\\ \downarrow\\ \text{Validated Navigation Performance} \end{array} } \]

4.2.19.316 Key Takeaways

1. Filter tuning determines how uncertainty is represented and how prediction and measurements are weighted.

\[ \boxed{ P_0,\ Q,\ R } \]

are central tuning quantities.

2. **

\(Q\) should represent uncertainty in the dynamic model.**

It should not simply be increased to make the filter more responsive.

3. **

\(R\) should represent actual measurement uncertainty.**

It should not simply be reduced to make the filter follow the sensor more closely.

4. Innovation statistics are fundamental tuning diagnostics.

A healthy innovation should be approximately zero mean and statistically consistent with \(\mathbf S\).

5. NIS tests consistency in measurement space.

\[ \boxed{ \mathrm{NIS} = r^TS^{-1}r. } \]

6. NEES tests consistency in state space when truth is available.

\[ \boxed{ \mathrm{NEES} = \tilde x^TP^{-1}\tilde x. } \]

7. Accuracy and consistency are different.

A filter can have low RMSE while still being dangerously overconfident.

8. Filter performance should include more than RMSE.

Evaluate:

• bias,
• percentiles,
• maximum error,
• convergence time,
• recovery time,
• rejection rate,
• availability,
• consistency.

9. Monte Carlo analysis is essential.

One simulation trajectory is not sufficient to characterize estimator performance.

10. Robust tuning seeks acceptable performance across uncertainty rather than optimum performance in one nominal case.

11. Sensor geometry, observability, and update rate influence tuning performance.

Tuning cannot create information that the sensors and dynamics do not provide.

12. Deterministic model errors should not be hidden by covariance inflation.

Correct frames, timing, calibration, lever arms, and measurement models first.

13. **

\(Q\)**, \(R\), gating, FDIR, and robust weighting are interconnected.

They should be assessed as one estimation architecture.

14. Performance should be verified across nominal, degraded, and faulted sensor configurations.

15. Numerical stability is part of estimator performance.

Covariance symmetry, positive semidefiniteness, conditioning, and real-time execution must be verified.

16. Performance testing should progress from unit tests to SIL, Monte Carlo, fault injection, closed-loop simulation, and HIL.

17. Relative navigation can be mentioned as one application, but the tuning principles are general to spacecraft navigation and state estimation.

18. A filter should be considered successfully tuned only when it is

\[ \boxed{ \text{Accurate} + \text{Consistent} + \text{Robust} + \text{Numerically Stable} + \text{Operationally Valid}. } \]

The central principle is:

\[ \boxed{ \text{A good estimator is not the one that merely produces a smooth state estimate; it is the one whose errors, uncertainty, residuals, and behaviour remain statistically credible across the conditions in which the spacecraft must operate.} } \]

Next: 4.2.20 Filter Consistency and Performance Metrics

After establishing process-noise and measurement-noise tuning, sensitivity, robustness, and estimator performance-assessment methods, the next section focuses specifically on filter consistency and the statistical metrics used to determine whether estimated uncertainty agrees with actual estimator behaviour.