Maximum Likelihood and MAP Estimation

4.3.6 Maximum Likelihood and MAP Estimation

Least-squares estimation was introduced earlier as an optimization problem:

\[ \boxed{ \hat{\mathbf x} = \arg\min_{\mathbf x} \mathbf r(\mathbf x)^T \mathbf R^{-1} \mathbf r(\mathbf x) } \]

where

\[ \mathbf r(\mathbf x) = \mathbf z-h(\mathbf x) \]

is the measurement residual.

A natural question is:

Why is this particular quadratic cost statistically meaningful?

The answer comes from probability theory.

If the measurement error is Gaussian,

\[ \boxed{ \mathbf v \sim \mathcal N(\mathbf 0,\mathbf R) } \]

then the probability of observing a particular measurement vector \(\mathbf z\), assuming the state has value \(\mathbf x\), is described by the likelihood

\[ \boxed{ p(\mathbf z|\mathbf x). } \]

The state that makes the observed measurements most probable is the Maximum Likelihood Estimate, or MLE:

\[ \boxed{ \hat{\mathbf x}_{ML} = \arg\max_{\mathbf x} p(\mathbf z|\mathbf x). } \]

Under Gaussian measurement noise, maximizing likelihood is equivalent to minimizing the weighted least-squares cost.

Thus,

\[ \boxed{ \text{Gaussian Maximum Likelihood} \Longleftrightarrow \text{Weighted Least Squares}. } \]

If prior information about the state is also available,

\[ p(\mathbf x), \]

Bayes' theorem gives

\[ \boxed{ p(\mathbf x|\mathbf z) = \frac{ p(\mathbf z|\mathbf x)p(\mathbf x) }{ p(\mathbf z) }. } \]

The state that maximizes the posterior probability is the Maximum A Posteriori estimate, or MAP:

\[ \boxed{ \hat{\mathbf x}_{MAP} = \arg\max_{\mathbf x} p(\mathbf x|\mathbf z). } \]

Under Gaussian likelihood and Gaussian prior assumptions,

\[ \boxed{ \text{MAP} \Longleftrightarrow \text{Prior-Constrained Weighted Least Squares}. } \]

This section therefore provides the probabilistic interpretation behind much of the estimation theory developed in Sections 4.3.1–4.3.5.

4.3.6.1 From Deterministic Optimization to Statistical Estimation

Suppose the measurement model is

\[ \boxed{ \mathbf z = h(\mathbf x) + \mathbf v. } \]

A deterministic estimation viewpoint asks:

Which value of \mathbf x produces the smallest measurement mismatch?

A probabilistic viewpoint asks:

Which value of \mathbf x makes the actual observed measurements most statistically plausible?

These viewpoints become closely related once a probability model for

\[ \mathbf v \]

is specified.

Thus,

\[ \boxed{ \text{Measurement Model} + \text{Noise Distribution} \rightarrow \text{Likelihood Function}. } \]

The assumed error distribution determines the statistical meaning of the resulting estimator.

4.3.6.2 What Is a Likelihood?

Suppose measurements \(\mathbf z\) have already been observed.

The quantity

\[ \boxed{ L(\mathbf x) = p(\mathbf z|\mathbf x) } \]

is called the likelihood function.

It is considered as a function of the unknown state or parameter \(\mathbf x\), while the observed data \(\mathbf z\) are held fixed.

A large value of

\[ L(\mathbf x) \]

means that the candidate state \(\mathbf x\) makes the observed measurement relatively plausible under the adopted statistical model.

4.3.6.3 Probability versus Likelihood

Probability and likelihood use the same mathematical expression but answer different questions.

Probability

Given a fixed state \(\mathbf x\),

\[ p(\mathbf z|\mathbf x) \]

describes how likely different measurements are.

Likelihood

Given an observed measurement \(\mathbf z\),

\[ L(\mathbf x) = p(\mathbf z|\mathbf x) \]

is considered as a function of candidate states.

Therefore,

\[ \boxed{ \text{Probability varies the data;} \qquad \text{Likelihood varies the hypothesis or parameter}. } \]

Likelihood is not itself generally a probability distribution over \(\mathbf x\).

Engineering interpretation: Likelihood should be read as a relative support function over candidate states for the data that were actually observed. Its numerical value becomes a probability density value only with respect to the measurement variable, not the parameter being varied.

4.3.6.4 Measurement Probability Model

Consider

\[ \mathbf z = h(\mathbf x) + \mathbf v. \]

If the measurement-noise distribution is known,

\[ p_{\mathbf v}(\mathbf v), \]

then because

\[ \mathbf v = \mathbf z-h(\mathbf x), \]

the likelihood is determined by the residual:

\[ \boxed{ p(\mathbf z|\mathbf x) = p_{\mathbf v} [ \mathbf z-h(\mathbf x) ]. } \]

Thus the residual enters statistical estimation naturally.

4.3.6.5 Scalar Gaussian Measurement Model

Suppose

\[ z=h(x)+v \]

with

\[ v \sim \mathcal N(0,\sigma^2). \]

Then

\[ \boxed{ p(z|x) = \frac{1} {\sqrt{2\pi\sigma^2}} \exp \left[ -\frac{ (z-h(x))^2 }{ 2\sigma^2 } \right]. } \]

The likelihood becomes largest when

\[ z-h(x) \]

is small relative to the measurement standard deviation.

Thus,

\[ \boxed{ \text{Likelihood depends on residual size relative to expected noise}. } \]

4.3.6.6 Multivariate Gaussian Measurement Model

For vector measurements,

\[ \mathbf z = h(\mathbf x) + \mathbf v \]

with

\[ \mathbf v \sim \mathcal N(\mathbf0,\mathbf R), \]

the likelihood is

\[ \boxed{ p(\mathbf z|\mathbf x) = \frac{ 1 }{ (2\pi)^{m/2} |\mathbf R|^{1/2} } \exp \left[ -\frac12 \mathbf r(\mathbf x)^T \mathbf R^{-1} \mathbf r(\mathbf x) \right] } \]

where

\[ \boxed{ \mathbf r(\mathbf x) = \mathbf z-h(\mathbf x). } \]

The covariance \(\mathbf R\) controls the statistical scale and correlation of the residuals.

4.3.6.7 Mahalanobis Distance

The quantity

\[ \boxed{ d_M^2 = \mathbf r^T \mathbf R^{-1} \mathbf r } \]

is the squared Mahalanobis distance of the residual.

Unlike the ordinary Euclidean norm,

\[ \mathbf r^T\mathbf r, \]

the Mahalanobis distance accounts for

• different measurement variances,
• different measurement units,
• measurement correlations.

Therefore,

\[ \boxed{ \text{Gaussian Likelihood} \propto \exp \left( -\frac12d_M^2 \right). } \]

Engineering interpretation: The Mahalanobis distance can also be interpreted geometrically. Residuals are measured in units of the assumed uncertainty ellipse, so directions with large expected variance are penalized less than equally sized residuals in tightly known directions.

4.3.6.8 Why Covariance Appears in the Likelihood

Suppose one observation has very small uncertainty and another has large uncertainty.

The same numerical residual should not be considered equally significant for both.

The Gaussian likelihood automatically accounts for this through

\[ \mathbf R^{-1}. \]

For independent observations,

\[ \mathbf R = \operatorname{diag} ( \sigma_1^2, \ldots, \sigma_m^2 ). \]

Then

\[ \boxed{ \mathbf r^T \mathbf R^{-1} \mathbf r = \sum_{i=1}^{m} \frac{r_i^2}{\sigma_i^2}. } \]

Thus each error is judged relative to its expected variance.

4.3.6.9 Likelihood Surface

The likelihood can be considered as a surface over the state space:

\[ L(\mathbf x). \]

For a simple estimation problem, this surface may have one clear maximum.

For nonlinear problems, it may contain

• multiple peaks,
• broad flat regions,
• narrow peaks,
• asymmetric shapes.

Thus,

\[ \boxed{ \text{Nonlinear Measurement Model} \rightarrow \text{Potentially Nonlinear Likelihood Surface}. } \]

The shape of the likelihood reflects the information contained in the observations.

4.3.6.10 Maximum Likelihood Estimation

The Maximum Likelihood Estimate is

\[ \boxed{ \hat{\mathbf x}_{ML} = \arg\max_{\mathbf x} p(\mathbf z|\mathbf x). } \]

It selects the state that makes the observed data most likely under the assumed measurement model.

MLE therefore depends on

• the observation model,
• the noise distribution,
• the unknown-state parameterization,
• the actual observations.

4.3.6.11 Why Log-Likelihood Is Used

Likelihoods from many measurements are usually products of probabilities.

For independent measurements,

\[ L(\mathbf x) = \prod_{k=1}^{N} p(\mathbf z_k|\mathbf x). \]

Products of many small probabilities can become numerically tiny.

Taking the logarithm gives

\[ \boxed{ \ell(\mathbf x) = \ln L(\mathbf x). } \]

Because logarithm is monotonic,

\[ \boxed{ \arg\max L(\mathbf x) = \arg\max \ell(\mathbf x). } \]

4.3.6.12 Product Becomes Sum

For independent measurements,

\[ L(\mathbf x) = \prod_{k=1}^{N} p(\mathbf z_k|\mathbf x). \]

Taking the logarithm,

\[ \boxed{ \ell(\mathbf x) = \sum_{k=1}^{N} \ln p(\mathbf z_k|\mathbf x). } \]

This transformation is both mathematically convenient and numerically stable.

Thus,

\[ \boxed{ \text{Likelihood Products} \rightarrow \text{Log-Likelihood Sums}. } \]

4.3.6.13 Gaussian Log-Likelihood

For

\[ \mathbf z \sim \mathcal N ( h(\mathbf x), \mathbf R ), \]

the log-likelihood is

\[ \boxed{ \ell(\mathbf x) = -\frac12 \mathbf r^T \mathbf R^{-1} \mathbf r - \frac12 \ln|\mathbf R| - \frac{m}{2}\ln(2\pi). } \]

If \(\mathbf R\) does not depend on \(\mathbf x\), the last two terms are constant with respect to the state.

Therefore maximizing \ell is equivalent to minimizing

\[ \boxed{ \mathbf r^T \mathbf R^{-1} \mathbf r. } \]

4.3.6.14 Negative Log-Likelihood

Optimization algorithms usually minimize rather than maximize.

Define the negative log-likelihood:

\[ \boxed{ \mathcal L(\mathbf x) = -\ln p(\mathbf z|\mathbf x). } \]

For a Gaussian model,

\[ \boxed{ \mathcal L(\mathbf x) = \frac12 \mathbf r^T \mathbf R^{-1} \mathbf r + \frac12\ln|\mathbf R| + \frac{m}{2}\ln(2\pi). } \]

Thus maximum likelihood becomes

\[ \boxed{ \hat{\mathbf x}_{ML} = \arg\min_{\mathbf x} \mathcal L(\mathbf x). } \]

4.3.6.15 Connection Between Gaussian MLE and Weighted Least Squares

If \(\mathbf R\) is independent of \(\mathbf x\),

\[ \ln|\mathbf R| \]

and

\[ m\ln(2\pi) \]

are constants.

Therefore,

\[ \boxed{ \hat{\mathbf x}_{ML} = \arg\min_{\mathbf x} \mathbf r^T \mathbf R^{-1} \mathbf r. } \]

But this is exactly weighted least squares.

Hence,

\[ \boxed{ \text{Gaussian MLE} = \text{Weighted Least Squares}. } \]

This gives WLS its statistical interpretation.

Engineering interpretation: This equivalence is one of the most important bridges in estimation theory: the WLS objective is not merely convenient algebra; under the Gaussian model it is the negative log-likelihood up to additive and positive scale constants.

4.3.6.16 Ordinary Least Squares as a Special MLE

Suppose

\[ \mathbf R = \sigma^2\mathbf I. \]

Then

\[ \mathbf R^{-1} = \frac{1}{\sigma^2}\mathbf I. \]

The ML objective becomes

\[ \frac{1}{\sigma^2} \mathbf r^T\mathbf r. \]

Because \(1/\sigma^2\) is a positive constant,

\[ \boxed{ \hat{\mathbf x}_{ML} = \arg\min \mathbf r^T\mathbf r. } \]

Therefore ordinary least squares is a special case of Gaussian maximum likelihood with equal independent measurement variance.

4.3.6.17 Multiple Independent Measurements

Suppose

\[ \mathbf z_k = h_k(\mathbf x)+\mathbf v_k \]

with independent Gaussian noise:

\[ \mathbf v_k \sim \mathcal N ( \mathbf0, \mathbf R_k ). \]

The joint likelihood is

\[ \boxed{ L(\mathbf x) = \prod_{k=1}^{N} p(\mathbf z_k|\mathbf x). } \]

The negative log-likelihood becomes

\[ \boxed{ \mathcal L = \frac12 \sum_{k=1}^{N} \mathbf r_k^T \mathbf R_k^{-1} \mathbf r_k + \text{constant}. } \]

Again, this is weighted least squares.

4.3.6.18 Correlated Measurements

If measurement errors are correlated, the joint covariance is not block diagonal.

Instead,

\[ \mathbf R = \begin{bmatrix} \mathbf R_{11} & \mathbf R_{12} \\ \mathbf R_{21} & \mathbf R_{22} \end{bmatrix}. \]

The Gaussian likelihood uses the complete covariance:

\[ \boxed{ L \propto \exp \left[ -\frac12 \mathbf r^T \mathbf R^{-1} \mathbf r \right]. } \]

Ignoring correlation changes the assumed likelihood and can cause the estimator to overestimate available independent information.

4.3.6.19 If Covariance Depends on the State

The simple WLS equivalence assumes that \(\mathbf R\) is independent of \(\mathbf x\).

If

\[ \mathbf R=\mathbf R(\mathbf x), \]

the negative log-likelihood contains

\[ \boxed{ \frac12 \ln |\mathbf R(\mathbf x)|. } \]

That term can no longer be discarded.

Therefore,

\[ \boxed{ \text{State-Dependent Covariance} \Rightarrow \text{Gaussian MLE Is Not Just Residual WLS}. } \]

This distinction is important in heteroscedastic measurement models.

Engineering interpretation: When covariance depends on the state or parameter, both the residual-weighting term and the normalization term change with the candidate solution. Dropping the determinant term would therefore change the statistical objective.

4.3.6.20 Likelihood for Non-Gaussian Noise

Maximum likelihood does not require Gaussian noise.

If the residual follows another probability distribution,

\[ p_{\mathbf v}(\mathbf v), \]

then

\[ \boxed{ L(\mathbf x) = p_{\mathbf v} [ \mathbf z-h(\mathbf x) ]. } \]

Different distributions generate different optimization costs.

Thus,

\[ \boxed{ \text{Noise Distribution} \rightarrow \text{Loss Function}. } \]

4.3.6.21 Laplace Noise and Absolute Residuals

Suppose scalar measurement errors follow a Laplace distribution:

\[ p(v) \propto \exp \left( -\frac{|v|}{b} \right). \]

Then the negative log-likelihood is proportional to

\[ \boxed{ |r|. } \]

Therefore maximum likelihood becomes an \(L_1\)-type estimation problem rather than least squares.

This illustrates an important principle:

\[ \boxed{ \text{Quadratic Loss Is Not Universal; It Comes From Gaussian Error Assumptions}. } \]

4.3.6.22 Heavy-Tailed Distributions

Outlier-prone observations may sometimes be better represented by heavy-tailed distributions.

Examples include

• Student-t,
• Cauchy,
• mixture distributions.

These distributions penalize extreme residuals differently from Gaussian models.

Their negative log-likelihoods can resemble robust loss functions.

Thus probabilistic modelling provides one formal basis for robust estimation.

4.3.6.23 Maximum Likelihood for Parameter Estimation

The unknown need not be a state.

Suppose measurements depend on parameter \(\theta\):

\[ \boxed{ z_k = h_k(\theta) + v_k. } \]

Then

\[ \boxed{ \hat\theta_{ML} = \arg\max_\theta p( z_1,\ldots,z_N|\theta ). } \]

Examples may include estimation of

• calibration constants,
• measurement biases,
• dynamic coefficients,
• force-model parameters,
• timing offsets.

4.3.6.24 State and Parameter Estimation Together

Define an augmented unknown vector:

\[ \boxed{ \boldsymbol{\xi} = \begin{bmatrix} \mathbf x\\ \boldsymbol{\theta} \end{bmatrix}. } \]

Then the likelihood becomes

\[ \boxed{ L(\boldsymbol{\xi}) = p( \mathbf z | \mathbf x,\boldsymbol{\theta} ). } \]

The ML estimate is

\[ \boxed{ \hat{\boldsymbol{\xi}}_{ML} = \arg\max_{\boldsymbol{\xi}} L(\boldsymbol{\xi}). } \]

However, successful optimization does not guarantee that all parameters are independently identifiable.

4.3.6.25 Identifiability and Likelihood Shape

If two parameters produce nearly the same measurement effect, the likelihood may have an elongated ridge.

For example,

\[ h(\theta_1,\theta_2) \approx h(\theta_1+\Delta,\theta_2-c\Delta). \]

Then many parameter combinations yield similar likelihood.

This produces

• strong parameter correlation,
• shallow curvature,
• large uncertainty.

Thus,

\[ \boxed{ \text{Flat Likelihood Direction} \rightarrow \text{Weak Parameter Identifiability}. } \]

Engineering interpretation: A ridge-like likelihood is the probabilistic analogue of poor conditioning in least squares. The optimizer may identify one combination of parameters well while leaving an orthogonal combination weakly determined.

4.3.6.26 Local Maximum versus Global Maximum

A nonlinear likelihood may have several local maxima.

An optimizer may converge to

\[ \hat{\mathbf x}_{local} \]

rather than the global ML solution.

Therefore,

\[ \boxed{ \text{Optimizer Convergence} \neq \text{Guaranteed Global Maximum Likelihood}. } \]

Initial-condition testing, global search methods, or problem-specific initialization may be required.

4.3.6.27 Likelihood Ratio

The relative support for two candidate states can be compared through the likelihood ratio:

\[ \boxed{ \Lambda = \frac{ L(\mathbf x_1) }{ L(\mathbf x_2) }. } \]

If

\[ \Lambda>1, \]

the observed measurements are more likely under \(\mathbf x_1\) than under \(\mathbf x_2\), given the assumed statistical model.

In practice, log-likelihood differences are often more numerically convenient.

4.3.6.28 Log-Likelihood Difference

Define

\[ \Delta\ell = \ell(\mathbf x_1) - \ell(\mathbf x_2). \]

Then

\[ \boxed{ \frac{ L(\mathbf x_1) }{ L(\mathbf x_2) } = e^{\Delta\ell}. } \]

Thus log-likelihood differences provide a convenient way to compare candidate parameter values.

4.3.6.29 Score Function

The gradient of the log-likelihood is called the score:

\[ \boxed{ \mathbf s(\mathbf x) = \nabla_{\mathbf x} \ell(\mathbf x). } \]

At an interior ML optimum,

\[ \boxed{ \mathbf s( \hat{\mathbf x}_{ML} ) = \mathbf0. } \]

The score indicates how the likelihood changes for small perturbations of the candidate state.

4.3.6.30 Gaussian Score Function

For constant \(\mathbf R\),

\[ \ell = -\frac12 \mathbf r^T \mathbf R^{-1} \mathbf r + C. \]

Since

\[ \mathbf r = \mathbf z-h(\mathbf x), \]

and

\[ \mathbf H = \frac{\partial h}{\partial\mathbf x}, \]

the score is

\[ \boxed{ \nabla\ell = \mathbf H^T \mathbf R^{-1} \mathbf r. } \]

Therefore the ML stationarity condition is

\[ \boxed{ \mathbf H^T \mathbf R^{-1} \mathbf r = \mathbf0. } \]

This is the same optimality condition encountered in nonlinear weighted least squares.

4.3.6.31 Fisher Information

The Fisher Information Matrix measures the expected curvature of the log-likelihood:

\[ \boxed{ \mathbf I(\mathbf x) = E \left[ \mathbf s(\mathbf x) \mathbf s(\mathbf x)^T \right]. } \]

Equivalently, under regular conditions,

\[ \boxed{ \mathbf I(\mathbf x) = - E \left[ \nabla^2 \ell(\mathbf x) \right]. } \]

It quantifies how much information the measurements are expected to provide about the unknown state.

Engineering interpretation: The Fisher matrix describes expected local information, not guaranteed information from one particular data realization. Its eigenstructure reveals expected strong and weak parameter directions.

4.3.6.32 Fisher Information for Gaussian Measurements

For a Gaussian measurement with constant covariance,

\[ \boxed{ \mathbf I = \mathbf H^T \mathbf R^{-1} \mathbf H. } \]

This is exactly the information matrix encountered in weighted least squares.

Therefore,

\[ \boxed{ \text{WLS Information Matrix} = \text{Gaussian Fisher Information} } \]

under the relevant local linear assumptions.

This is an important connection between optimization and statistical estimation.

4.3.6.33 Information from Independent Measurements

For independent observations,

\[ \boxed{ \mathbf I_{total} = \sum_{k=1}^{N} \mathbf I_k. } \]

For Gaussian measurements,

\[ \boxed{ \mathbf I_{total} = \sum_{k=1}^{N} \mathbf H_k^T \mathbf R_k^{-1} \mathbf H_k. } \]

Thus independent statistical information adds.

4.3.6.34 Cramér–Rao Lower Bound

For an unbiased estimator under appropriate regularity conditions,

\[ \boxed{ \operatorname{Cov} ( \hat{\mathbf x} ) \succeq \mathbf I(\mathbf x)^{-1}. } \]

This is the Cramér–Rao Lower Bound, or CRLB.

It states that no unbiased estimator can have covariance smaller than the inverse Fisher information in the matrix sense, under the assumptions used to derive the bound.

Engineering interpretation: The matrix inequality means that the difference between an estimator covariance and the inverse Fisher information is positive semidefinite. The bound therefore applies to every direction in the parameter space, not merely to diagonal variances.

4.3.6.35 Interpretation of the Cramér–Rao Bound

The CRLB is not automatically the actual estimator covariance.

Instead, it is a theoretical lower bound.

Thus,

\[ \boxed{ \mathbf I^{-1} = \text{Best Possible Local Covariance Bound Under the Assumed Model} } \]

rather than a guarantee that a particular estimator achieves that uncertainty.

An efficient estimator may approach the CRLB under suitable conditions.

4.3.6.36 MLE and Asymptotic Behaviour

Under regular conditions and with sufficiently informative data, MLE has important asymptotic properties.

It is often

• consistent,
• asymptotically unbiased,
• asymptotically efficient,
• asymptotically Gaussian.

Conceptually,

\[ \boxed{ \hat{\mathbf x}_{ML} \approx \mathcal N ( \mathbf x_{true}, \mathbf I^{-1} ) } \]

for sufficiently large data sets under appropriate assumptions.

These are asymptotic results, not guarantees for every finite spacecraft data set.

4.3.6.37 Prior Probability

MLE uses only the likelihood.

If prior information exists, represent it as

\[ \boxed{ p(\mathbf x). } \]

The prior may encode knowledge from

• previous navigation solutions,
• calibration,
• physical limits,
• historical measurements,
• engineering knowledge.

The prior is specified before incorporating the current measurement set.

4.3.6.38 Bayesian Posterior

Bayes' theorem gives

\[ \boxed{ p(\mathbf x|\mathbf z) = \frac{ p(\mathbf z|\mathbf x) p(\mathbf x) }{ p(\mathbf z) }. } \]

The posterior combines

\[ \boxed{ \text{Current Measurement Information} + \text{Prior Information}. } \]

This posterior distribution represents updated knowledge of the unknown state after observing the data.

4.3.6.39 Evidence

The denominator

\[ \boxed{ p(\mathbf z) = \int p(\mathbf z|\mathbf x) p(\mathbf x) \,d\mathbf x } \]

is the evidence or marginal likelihood.

It normalizes the posterior:

\[ \int p(\mathbf x|\mathbf z) \,d\mathbf x = 1. \]

For MAP estimation, the evidence does not depend on \(\mathbf x\) and therefore does not affect the maximizing state.

4.3.6.40 Maximum A Posteriori Estimation

The MAP estimate is

\[ \boxed{ \hat{\mathbf x}_{MAP} = \arg\max_{\mathbf x} p(\mathbf x|\mathbf z). } \]

Using Bayes' theorem,

\[ \hat{\mathbf x}_{MAP} = \arg\max_{\mathbf x} p(\mathbf z|\mathbf x) p(\mathbf x). \]

Therefore,

\[ \boxed{ \text{MAP} = \text{Likelihood} + \text{Prior}. } \]

4.3.6.41 Log-Posterior

Taking logarithms,

\[ \boxed{ \ln p(\mathbf x|\mathbf z) = \ln p(\mathbf z|\mathbf x) + \ln p(\mathbf x) - \ln p(\mathbf z). } \]

Since

\[ p(\mathbf z) \]

does not depend on \(\mathbf x\),

\[ \boxed{ \hat{\mathbf x}_{MAP} = \arg\max_{\mathbf x} \left[ \ln p(\mathbf z|\mathbf x) + \ln p(\mathbf x) \right]. } \]

4.3.6.42 Negative Log-Posterior

MAP can equivalently be written as a minimization:

\[ \boxed{ \hat{\mathbf x}_{MAP} = \arg\min_{\mathbf x} \left[ -\ln p(\mathbf z|\mathbf x) - \ln p(\mathbf x) \right]. } \]

Thus,

\[ \boxed{ \text{MAP Cost} = \text{Negative Log-Likelihood} + \text{Negative Log-Prior}. } \]

This provides a general probabilistic interpretation of prior-constrained optimization.

4.3.6.43 Gaussian Prior

Suppose

\[ \boxed{ \mathbf x \sim \mathcal N ( \bar{\mathbf x}, \mathbf P_0 ). } \]

Then

\[ p(\mathbf x) \propto \exp \left[ -\frac12 ( \mathbf x-\bar{\mathbf x} )^T \mathbf P_0^{-1} ( \mathbf x-\bar{\mathbf x} ) \right]. \]

Therefore,

\[ \boxed{ -\ln p(\mathbf x) = \frac12 ( \mathbf x-\bar{\mathbf x} )^T \mathbf P_0^{-1} ( \mathbf x-\bar{\mathbf x} ) + C. } \]

4.3.6.44 Gaussian MAP Cost

With Gaussian measurement noise and Gaussian prior,

\[ \boxed{ \begin{aligned} J_{MAP}(\mathbf x) ={}& [ \mathbf z-h(\mathbf x) ]^T \mathbf R^{-1} [ \mathbf z-h(\mathbf x) ] \\ &+ ( \mathbf x-\bar{\mathbf x} )^T \mathbf P_0^{-1} ( \mathbf x-\bar{\mathbf x} ). \end{aligned} } \]

Therefore,

\[ \boxed{ \text{Gaussian MAP} \Longleftrightarrow \text{Prior-Constrained Weighted Least Squares}. } \]

Engineering interpretation: This equation also explains why a prior can stabilize an otherwise weak problem: prior curvature adds directly to measurement curvature. That numerical stabilization should still be distinguished from information actually supplied by the new measurements.

4.3.6.45 Difference Between ML and MAP

MLE solves

\[ \boxed{ \hat{\mathbf x}_{ML} = \arg\max p(\mathbf z|\mathbf x). } \]

MAP solves

\[ \boxed{ \hat{\mathbf x}_{MAP} = \arg\max p(\mathbf z|\mathbf x)p(\mathbf x). } \]

Therefore,

\[ \boxed{ \text{MLE Uses Measurement Information Only} } \]

while

\[ \boxed{ \text{MAP Uses Measurement Information + Prior Information}. } \]

4.3.6.46 When ML and MAP Are Similar

If the measurement likelihood is very informative compared with the prior,

\[ p(\mathbf z|\mathbf x) \]

is sharply concentrated.

Then the prior may have relatively little influence.

In that case,

\[ \boxed{ \hat{\mathbf x}_{MAP} \approx \hat{\mathbf x}_{ML}. } \]

Similarly, a very broad weak prior may produce little difference between ML and MAP.

4.3.6.47 When MAP Is Strongly Influenced by the Prior

If measurements provide weak information,

\[ \mathbf H^T \mathbf R^{-1} \mathbf H \]

may be small or poorly conditioned.

Then the prior can strongly affect the solution.

A tight prior covariance means

\[ \mathbf P_0^{-1} \]

is large.

Therefore,

\[ \boxed{ \text{Precise Prior} \rightarrow \text{Strong MAP Influence}. } \]

4.3.6.48 Incorrect Prior

A prior can improve estimation only when it is meaningful.

An incorrect but overly confident prior may bias the MAP solution.

Thus,

\[ \boxed{ \text{Overconfident Incorrect Prior} \rightarrow \text{Potentially Biased Posterior}. } \]

Prior covariance should represent actual uncertainty rather than a desire to force the estimate toward a preferred value.

4.3.6.49 Weak Prior

A weak Gaussian prior can be represented by large covariance:

\[ \mathbf P_0 \rightarrow \text{large}. \]

Then

\[ \mathbf P_0^{-1} \rightarrow \text{small}. \]

The MAP objective approaches the ML objective.

Therefore,

\[ \boxed{ \text{Weak Prior} \rightarrow \text{MAP Approaches ML}. } \]

4.3.6.50 Uniform Prior

If the prior is constant over the admissible region,

\[ p(\mathbf x) = C, \]

then

\[ \boxed{ \hat{\mathbf x}_{MAP} = \hat{\mathbf x}_{ML} } \]

provided the ML solution lies within that region.

A uniform prior therefore adds no relative preference within its support.

4.3.6.51 MAP Is Not the Posterior Mean

The MAP estimate is the mode of the posterior:

\[ \boxed{ \hat{\mathbf x}_{MAP} = \operatorname{mode} [ p(\mathbf x|\mathbf z) ]. } \]

The posterior mean is

\[ \boxed{ E[ \mathbf x|\mathbf z ]. } \]

For a Gaussian posterior, mean and mode coincide.

For a skewed or multimodal posterior, they may differ substantially.

Thus,

\[ \boxed{ \text{MAP} \neq \text{Posterior Mean} } \]

in general.

Engineering interpretation: For a symmetric unimodal Gaussian posterior, mean, median, and mode coincide. Once the posterior becomes skewed or multimodal, a single MAP point can be a poor summary of the full uncertainty.

4.3.6.52 MAP versus Bayesian Estimation

MAP returns a single point estimate.

Full Bayesian inference considers the entire posterior distribution:

\[ \boxed{ p(\mathbf x|\mathbf z). } \]

The posterior can describe

• uncertainty,
• asymmetry,
• multiple plausible solutions,
• correlations.

Therefore MAP is one summary of Bayesian inference, not the complete posterior.

4.3.6.53 Posterior Covariance

If the posterior is approximately Gaussian near its maximum,

\[ \boxed{ \mathbf P_{post} \approx [ \nabla^2 ( -\ln p(\mathbf x|\mathbf z) ) ]^{-1}_{\hat{\mathbf x}_{MAP}}. } \]

For Gaussian likelihood and Gaussian prior with local linearization,

\[ \boxed{ \mathbf P_{post} \approx \left( \mathbf H^T \mathbf R^{-1} \mathbf H + \mathbf P_0^{-1} \right)^{-1}. } \]

This is the same posterior covariance form encountered earlier.

4.3.6.54 Posterior Information

Define

\[ \boxed{ \boldsymbol{\Lambda}_{post} = \mathbf P_{post}^{-1}. } \]

Then

\[ \boxed{ \boldsymbol{\Lambda}_{post} = \mathbf P_0^{-1} + \mathbf H^T \mathbf R^{-1} \mathbf H. } \]

Thus,

\[ \boxed{ \text{Posterior Information} = \text{Prior Information} + \text{Measurement Information}. } \]

This unifies MAP, recursive estimation, and weighted least squares.

4.3.6.55 MAP and Regularization

Suppose a prior has the form

\[ p(\mathbf x) \propto \exp \left[ -\lambda \|\mathbf L\mathbf x\|^2 \right]. \]

Then the negative log-prior becomes a regularization penalty:

\[ \boxed{ \lambda \|\mathbf L\mathbf x\|^2. } \]

Therefore some regularization methods have a Bayesian interpretation.

However, the probabilistic interpretation is valid only if the penalty corresponds to an explicitly assumed prior.

4.3.6.56 Gaussian Prior and Quadratic Regularization

For

\[ \mathbf x \sim \mathcal N ( \mathbf0, \mathbf P_0 ), \]

the negative log-prior is

\[ \boxed{ \mathbf x^T \mathbf P_0^{-1} \mathbf x. } \]

If

\[ \mathbf P_0^{-1} = \lambda\mathbf I, \]

the prior term becomes

\[ \boxed{ \lambda \|\mathbf x\|^2. } \]

Thus ridge-type regularization corresponds to a zero-mean Gaussian prior under this interpretation.

4.3.6.57 Non-Gaussian Priors

The prior need not be Gaussian.

Possible priors may represent

• positivity,
• bounded ranges,
• sparsity,
• multimodal knowledge,
• heavy-tailed parameter distributions.

Different priors produce different MAP objective functions.

Therefore,

\[ \boxed{ \text{Prior Distribution} \rightarrow \text{Regularization Structure in MAP Optimization}. } \]

4.3.6.58 Hard Bounds versus Prior Probability

Suppose

\[ \theta_{\min} \le \theta \le \theta_{\max}. \]

A hard bound declares all values outside the interval inadmissible.

A prior instead assigns probability density within the admissible space.

These are conceptually different:

\[ \boxed{ \text{Constraint} \neq \text{Prior}. } \]

Both may appear together in an estimation problem.

4.3.6.59 Dynamic Maximum Likelihood

For dynamic state estimation,

\[ \mathbf x_k = f( \mathbf x_{k-1}, \boldsymbol{\theta} ) \]

and

\[ \mathbf z_k = h_k( \mathbf x_k ) + \mathbf v_k. \]

Measurements collected across time produce likelihood

\[ \boxed{ L( \mathbf x_0, \boldsymbol{\theta} ) = p( \mathbf z_1, \ldots, \mathbf z_N | \mathbf x_0, \boldsymbol{\theta} ). } \]

The reference state and dynamic parameters can be estimated by maximizing this likelihood.

4.3.6.60 Dynamic Gaussian Likelihood

For conditionally independent Gaussian measurements,

\[ \boxed{ L \propto \prod_{k=1}^{N} \exp \left[ -\frac12 \mathbf r_k^T \mathbf R_k^{-1} \mathbf r_k \right]. } \]

Therefore,

\[ \boxed{ -\ln L = \frac12 \sum_{k=1}^{N} \mathbf r_k^T \mathbf R_k^{-1} \mathbf r_k + C. } \]

This is exactly the batch dynamic weighted least-squares objective introduced earlier.

4.3.6.61 Dynamic MAP Estimation

If a prior on the reference state and parameters is included,

\[ p( \mathbf x_0, \boldsymbol{\theta} ), \]

then

\[ \boxed{ p( \mathbf x_0, \boldsymbol{\theta} | \mathbf z_{1:N} ) \propto p( \mathbf z_{1:N} | \mathbf x_0, \boldsymbol{\theta} ) p( \mathbf x_0, \boldsymbol{\theta} ). } \]

The MAP solution balances trajectory-fit likelihood with prior knowledge.

4.3.6.62 Connection to Batch Differential Correction

For Gaussian observations, maximizing the dynamic likelihood is equivalent to minimizing

\[ \sum_k \mathbf r_k^T \mathbf R_k^{-1} \mathbf r_k. \]

This is precisely the nonlinear batch least-squares problem solved by differential correction.

Therefore,

\[ \boxed{ \text{Batch Differential Correction} = \text{Gaussian Dynamic Maximum Likelihood Optimization} } \]

under the corresponding assumptions.

Engineering interpretation: Differential correction is therefore best viewed as a numerical method for solving a dynamic Gaussian maximum-likelihood problem when its residual structure is exploited through local linearization.

4.3.6.63 Connection to Recursive Bayesian Estimation

Section 4.3.4 developed

\[ p( \mathbf x_k | Z_k ). \]

MAP estimation selects the most probable state from this posterior.

Recursive Bayesian estimation, however, propagates the complete posterior distribution conceptually.

Thus,

\[ \boxed{ \text{MAP} = \text{Posterior Mode} } \]

while

\[ \boxed{ \text{Bayesian Filter} = \text{Recursive Posterior Propagation}. } \]

4.3.6.64 General Orbit-Determination Interpretation

Suppose the unknown reference orbit is

\[ \mathbf x_0 = \begin{bmatrix} \mathbf r_0\\ \mathbf v_0 \end{bmatrix}. \]

Tracking data are obtained at multiple epochs.

For each candidate reference state:

1. propagate the orbit,
2. predict the tracking data,
3. form residuals,
4. evaluate the likelihood.

The ML orbit is

\[ \boxed{ \hat{\mathbf x}_{0,ML} = \arg\max_{\mathbf x_0} p( \mathbf z_{1:N} | \mathbf x_0 ). } \]

If a prior orbit and covariance are available, the corresponding estimate becomes MAP.

4.3.6.65 General Relative-Navigation Example

Consider a general relative-navigation problem with

\[ \mathbf x_{\mathrm{rel}} = \begin{bmatrix} \mathbf r_{\mathrm{rel}}\\ \mathbf v_{\mathrm{rel}} \end{bmatrix}. \]

Suppose the measurement set contains

• range,
• angular direction,
• relative velocity-related observations.

The likelihood may be written as

\[ \boxed{ p( \mathbf z | \mathbf x_{\mathrm{rel}} ). } \]

For Gaussian observations,

\[ \boxed{ -\ln L \propto \sum_k \mathbf r_k^T \mathbf R_k^{-1} \mathbf r_k. } \]

The ML relative state is the state that best explains the measurement set statistically.

If a prior relative-navigation estimate is available,

\[ p( \mathbf x_{\mathrm{rel}} ), \]

the result becomes a MAP estimate.

This is a general RPO estimation concept and is not specific to any particular mission.

4.3.6.66 Geometry and Likelihood

Measurement geometry affects the shape of the likelihood.

For example, a single range measurement may define many states with similar range.

The likelihood may therefore be elongated along poorly constrained directions.

Additional measurements with different geometry can narrow the likelihood.

Thus,

\[ \boxed{ \text{Measurement Geometry} \rightarrow \text{Likelihood Shape} \rightarrow \text{State Uncertainty}. } \]

Engineering interpretation: Likelihood geometry and physical measurement geometry are directly connected: narrow likelihood curvature corresponds to strong information, while elongated or flat regions correspond to weakly constrained state combinations.

4.3.6.67 Multiple Hypotheses

Some nonlinear measurements may support several physically plausible states.

Then the likelihood can become multimodal:

\[ \boxed{ L(\mathbf x) \text{ has multiple peaks}. } \]

A single covariance around one solution may then be inadequate.

This is important for ambiguous angular or feature-based navigation measurements.

4.3.6.68 MAP in a Multimodal Posterior

If the posterior contains multiple modes,

\[ p(\mathbf x|\mathbf z) \]

may have several separated peaks.

MAP selects only the highest one.

Thus,

\[ \boxed{ \text{MAP May Hide Other Significant Posterior Modes}. } \]

For strongly ambiguous problems, full posterior or multi-hypothesis methods may be preferable.

Engineering interpretation: In a multimodal posterior, a local covariance around the MAP mode says nothing about the probability mass surrounding the other modes. This is a key limitation of representing a complex posterior by one state and one covariance matrix.

4.3.6.69 Nuisance Parameters

Some unknown quantities affect the measurements but are not the main quantities of interest.

These are often called nuisance parameters.

Examples include

• unknown bias,
• scale factor,
• offset,
• environmental parameter.

Suppose

\[ L( \mathbf x,\boldsymbol{\eta} ) \]

depends on desired state \(\mathbf x\) and nuisance parameter \(\boldsymbol{\eta}\).

Both may need to be handled properly rather than simply ignoring \(\boldsymbol{\eta}\).

4.3.6.70 Profile Likelihood

One way to handle nuisance parameters is profile likelihood.

For each candidate \(\mathbf x\), maximize over \(\boldsymbol{\eta}\):

\[ \boxed{ L_p(\mathbf x) = \max_{\boldsymbol{\eta}} L( \mathbf x,\boldsymbol{\eta} ). } \]

Then estimate

\[ \mathbf x \]

using the profile likelihood.

This is useful when uncertainty in nuisance parameters materially affects the desired state estimate.

4.3.6.71 Marginalization versus Profiling

Bayesian estimation may integrate nuisance parameters:

\[ \boxed{ p(\mathbf x|\mathbf z) = \int p( \mathbf x,\boldsymbol{\eta} | \mathbf z ) \,d\boldsymbol{\eta}. } \]

This is called marginalization.

By contrast, profile likelihood maximizes over nuisance parameters.

Thus,

\[ \boxed{ \text{Profiling} \neq \text{Marginalization}. } \]

They represent different approaches to uncertainty in nuisance variables.

4.3.6.72 Bias Estimation

Suppose

\[ \mathbf z = h(\mathbf x) + \mathbf b + \boldsymbol{\eta}. \]

Ignoring \(\mathbf b\) may bias the state estimate.

Instead,

\[ \boldsymbol{\xi} = \begin{bmatrix} \mathbf x\\ \mathbf b \end{bmatrix} \]

may be estimated jointly.

The likelihood becomes

\[ \boxed{ p( \mathbf z | \mathbf x,\mathbf b ). } \]

Successful bias estimation still depends on identifiability.

4.3.6.73 Noise Parameters Can Also Be Estimated

Sometimes the covariance itself contains unknown parameters.

For example,

\[ \mathbf R = \sigma^2\mathbf I \]

with unknown \(\sigma^2\).

Then the likelihood depends on both

\[ \mathbf x \]

and

\[ \sigma^2. \]

The determinant term

\[ \ln|\mathbf R| \]

becomes essential and cannot be discarded.

Thus,

\[ \boxed{ \text{Estimating Noise Parameters Requires the Full Gaussian Likelihood}. } \]

Engineering interpretation: This is why the normalization term of the Gaussian likelihood matters whenever noise level is being estimated. Without it, the optimizer could artificially increase the variance to reduce the weighted residual penalty without paying the corresponding probability-normalization cost.

4.3.6.74 Example: Unknown Gaussian Variance

For independent scalar residuals,

\[ r_i \sim \mathcal N ( 0,\sigma^2 ), \]

the log-likelihood is

\[ \boxed{ \ell = -\frac{N}{2} \ln(2\pi) - \frac{N}{2} \ln\sigma^2 - \frac{1}{2\sigma^2} \sum_i r_i^2. } \]

The variance therefore influences both

• residual weighting,
• normalization of probability density.

4.3.6.75 Likelihood and Outliers

A Gaussian likelihood assigns exponentially decreasing probability to large residuals.

Therefore, extreme outliers can strongly influence the ML solution.

If real measurement errors are heavy-tailed, a Gaussian likelihood may be a poor statistical model.

This connects maximum likelihood directly to robust estimation.

4.3.6.76 Model Mismatch

MLE is optimal only relative to the assumed likelihood.

If the assumed measurement distribution is wrong,

\[ p_{\text{assumed}} (\mathbf z|\mathbf x) \neq p_{\text{actual}} (\mathbf z|\mathbf x), \]

the resulting estimate can be biased or inefficient.

Therefore,

\[ \boxed{ \text{Statistical Optimality Is Always Model-Dependent}. } \]

Engineering interpretation: The phrase 'optimal' in ML therefore always means optimal for the adopted probability model. Model checking is part of estimation, not something separate from it.

4.3.6.77 Incorrect Covariance

If \(\mathbf R\) is underestimated, measurements are treated as more precise than they really are.

If \(\mathbf R\) is overestimated, measurements are underweighted.

Thus,

\[ \boxed{ \text{Incorrect Covariance} \rightarrow \text{Incorrect Likelihood Shape}. } \]

This can distort both estimates and reported uncertainty.

4.3.6.78 Measurement Independence Assumption

A common simplification is

\[ p( \mathbf z_1,\ldots,\mathbf z_N|\mathbf x ) = \prod_k p( \mathbf z_k|\mathbf x ). \]

This requires conditional independence.

If the observations share common errors,

• clock bias,
• calibration error,
• preprocessing bias,

then this product may not correctly represent the joint likelihood.

4.3.6.79 Whitening and Likelihood

If

\[ \mathbf R = \mathbf L\mathbf L^T, \]

define the whitened residual

\[ \boxed{ \tilde{\mathbf r} = \mathbf L^{-1} \mathbf r. } \]

Then

\[ \mathbf r^T \mathbf R^{-1} \mathbf r = \tilde{\mathbf r}^T \tilde{\mathbf r}. \]

Thus the Gaussian negative log-likelihood becomes Euclidean least squares in whitened coordinates.

4.3.6.80 Optimization of ML and MAP Objectives

The ML or MAP objective may be nonlinear.

Possible methods include

• Gauss–Newton,
• Newton methods,
• Levenberg–Marquardt,
• trust-region methods,
• gradient-based optimization,
• constrained optimization.

The statistical formulation defines the objective.

The numerical optimizer determines how that objective is solved.

Therefore,

\[ \boxed{ \text{Estimator Definition} \neq \text{Optimization Algorithm}. } \]

4.3.6.81 MLE versus Optimization Method

MLE does not mean “use one particular solver.”

The estimate is defined by

\[ \boxed{ \hat{\mathbf x}_{ML} = \arg\max L(\mathbf x). } \]

Gauss–Newton or LM are merely methods for finding that maximum when the likelihood corresponds to a nonlinear least-squares structure.

This distinction is important.

4.3.6.82 MAP versus Optimization Method

Similarly,

\[ \boxed{ \hat{\mathbf x}_{MAP} = \arg\max p(\mathbf x|\mathbf z) } \]

defines the estimator.

The posterior mode may be computed using

• Newton,
• Gauss–Newton,
• LM,
• constrained solvers,
• other optimization methods.

Thus MAP is a statistical criterion, not a specific algorithm.

4.3.6.83 Gradient of Gaussian Negative Log-Likelihood

For constant covariance,

\[ \mathcal L = \frac12 \mathbf r^T \mathbf R^{-1} \mathbf r. \]

Then

\[ \boxed{ \nabla\mathcal L = - \mathbf H^T \mathbf R^{-1} \mathbf r. } \]

At the ML solution,

\[ \boxed{ \mathbf H^T \mathbf R^{-1} \mathbf r = \mathbf0. } \]

This again matches the nonlinear WLS optimality condition.

4.3.6.84 Hessian and Information

The Hessian of the negative log-likelihood describes local curvature.

For Gaussian nonlinear measurement models,

\[ \nabla^2\mathcal L \]

contains

\[ \mathbf H^T \mathbf R^{-1} \mathbf H \]

plus residual-weighted second-derivative terms.

Gauss–Newton approximates it using

\[ \boxed{ \mathbf H^T \mathbf R^{-1} \mathbf H. } \]

This is closely connected to Fisher information.

4.3.6.85 Observed Information versus Expected Information

The negative Hessian of the log-likelihood evaluated using actual observations is sometimes called observed information.

The expectation of that quantity is Fisher information:

\[ \boxed{ \mathbf I = - E[ \nabla^2\ell ]. } \]

Observed and expected information can differ for finite data sets.

4.3.6.86 Confidence Region from Likelihood Curvature

Near a well-behaved ML solution,

\[ \ell(\mathbf x) \]

may be approximated quadratically.

Then the uncertainty region is approximately ellipsoidal.

The local covariance is related to inverse likelihood curvature:

\[ \boxed{ \mathbf P \approx \mathbf I^{-1}. } \]

This is closely connected to the normal-matrix covariance used in least squares.

4.3.6.87 Likelihood-Based Uncertainty versus Covariance Approximation

For strongly nonlinear likelihoods, an inverse Hessian provides only a local uncertainty approximation.

The full likelihood may be

• skewed,
• multimodal,
• bounded,
• non-Gaussian.

Thus,

\[ \boxed{ \text{Inverse Hessian} \neq \text{Complete Global Uncertainty Description}. } \]

Likelihood profiling or sampling methods may reveal more structure.

4.3.6.88 Priors and Identifiability

A prior can make an otherwise singular MAP optimization numerically well posed.

For example,

\[ \mathbf H^T \mathbf R^{-1} \mathbf H \]

may be rank deficient while

\[ \mathbf H^T \mathbf R^{-1} \mathbf H + \mathbf P_0^{-1} \]

is nonsingular.

However,

\[ \boxed{ \text{Prior-Constrained Solution} \neq \text{Measurement Identifiability}. } \]

The data may still contain little independent information about that state component.

Engineering interpretation: A prior can resolve a numerical singularity, but it does not change the fact that the measurements alone were unable to constrain that direction. Reporting prior and data information separately prevents this distinction from being hidden.

4.3.6.89 Information from Prior versus Information from Data

It is useful to distinguish

\[ \boxed{ \boldsymbol{\Lambda}_{prior} = \mathbf P_0^{-1} } \]

from

\[ \boxed{ \boldsymbol{\Lambda}_{data} = \mathbf H^T \mathbf R^{-1} \mathbf H. } \]

Then

\[ \boxed{ \boldsymbol{\Lambda}_{post} = \boldsymbol{\Lambda}_{prior} + \boldsymbol{\Lambda}_{data}. } \]

This makes it clear whether a well-constrained posterior comes mainly from observations or prior assumptions.

4.3.6.90 Prior Dominance

Suppose

\[ \boldsymbol{\Lambda}_{prior} \gg \boldsymbol{\Lambda}_{data}. \]

Then the posterior is dominated by prior information.

In such a case, reporting only the posterior covariance may hide the fact that the measurements added little new information.

Therefore prior and measurement contributions should sometimes be examined separately.

4.3.6.91 Data Dominance

If

\[ \boldsymbol{\Lambda}_{data} \gg \boldsymbol{\Lambda}_{prior}, \]

then the posterior is largely determined by measurements.

This often occurs when

• many informative measurements are available,
• measurement covariance is small,
• geometry is strong.

Then MAP approaches ML.

4.3.6.92 Prior Sensitivity Testing

A useful validation is to repeat the MAP estimation with plausible alternative priors.

If the solution changes significantly, the result is prior-sensitive.

This does not automatically mean the method is wrong, but it indicates that the data alone may not strongly determine the state.

4.3.6.93 Measurement-Model Sensitivity Testing

MLE and MAP are also sensitive to the assumed likelihood model.

Useful tests include changing

• measurement covariance,
• distribution family,
• correlation assumptions,
• outlier handling.

Large changes in the solution indicate dependence on statistical modelling assumptions.

4.3.6.94 Constraints in ML and MAP

Optimization may include hard physical constraints:

\[ g(\mathbf x)=0, \]

or

\[ c(\mathbf x)\le0. \]

Then

\[ \boxed{ \hat{\mathbf x}_{ML} = \arg\max_{\mathbf x\in\mathcal C} L(\mathbf x). } \]

Similarly for MAP.

Constraints restrict the feasible state space but are conceptually different from prior probability.

4.3.6.95 Parameter Transformations

For a positive parameter,

\[ \theta>0, \]

one may use

\[ \gamma=\ln\theta. \]

Then

\[ \theta=e^\gamma. \]

The likelihood can be optimized in \gamma-space.

However, parameter transformation affects the prior density if performing Bayesian inference.

Thus transformations must be handled carefully when interpreting MAP estimates.

4.3.6.96 MAP Is Not Invariant to Parameterization

MLE is generally invariant under one-to-one transformations when transformed appropriately.

MAP is more subtle because probability density values depend on parameterization.

A reparameterization introduces a Jacobian factor in the prior/posterior density.

Therefore,

\[ \boxed{ \text{MAP Point Estimate Can Depend on Parameterization}. } \]

This is an important conceptual limitation of MAP.

Engineering interpretation: MAP depends on the density value at a point, and densities transform with a Jacobian under reparameterization. The most probable point can therefore move even when the underlying probability content is unchanged.

4.3.6.97 MLE Invariance

If

\[ \phi=g(\theta) \]

is one-to-one and

\[ \hat\theta_{ML} \]

is the maximum-likelihood estimate, then

\[ \boxed{ \hat\phi_{ML} = g( \hat\theta_{ML} ). } \]

This is known as the invariance property of MLE.

4.3.6.98 Posterior Predictive Distribution

Bayesian estimation can also predict future measurements.

Given posterior

\[ p(\mathbf x|\mathbf z), \]

the predictive distribution for future observation \(\mathbf z^\star\) is

\[ \boxed{ p( \mathbf z^\star|\mathbf z ) = \int p( \mathbf z^\star|\mathbf x ) p( \mathbf x|\mathbf z ) \,d\mathbf x. } \]

This incorporates uncertainty in the estimated state.

MAP alone does not fully represent this uncertainty.

4.3.6.99 Model Checking with Likelihood

A high likelihood is meaningful only relative to the assumed model.

Residual and likelihood diagnostics should examine whether the observed errors behave as predicted.

For Gaussian models, useful checks include

• residual mean,
• normalized residual magnitude,
• correlation,
• tail behaviour.

Poor diagnostics may indicate that the chosen likelihood is inappropriate.

4.3.6.100 Likelihood Does Not Prove the Model Is Correct

The ML estimate is the best state within the assumed model family.

If the measurement model is incorrect, the highest-likelihood parameter value may still be physically wrong.

Therefore,

\[ \boxed{ \text{Maximum Likelihood} \neq \text{Proof of Model Correctness}. } \]

This is analogous to the earlier principle that small least-squares residuals do not prove model validity.

4.3.6.101 Overfitting and Likelihood

Adding more parameters can increase the maximum likelihood because the model gains additional flexibility.

Therefore,

\[ \boxed{ \text{Higher Maximum Likelihood} \neq \text{Automatically Better Model}. } \]

Model complexity must be considered separately.

This motivates model-selection tools such as information criteria, though those are distinct from basic state estimation.

4.3.6.102 Likelihood and Residual Analysis

For a Gaussian likelihood, residual analysis remains essential because

\[ L \]

is directly determined by weighted residuals.

After estimation, examine

• raw residuals,
• normalized residuals,
• trends,
• correlations,
• outliers,
• measurement-type dependence.

A single likelihood value can hide structured model deficiencies.

4.3.6.103 Numerical Underflow

Directly multiplying many probabilities can cause underflow:

\[ L = \prod_{k=1}^{N} p_k \rightarrow 0 \]

numerically even when relative likelihood information remains meaningful.

Therefore use

\[ \boxed{ \ell = \sum_k \ln p_k. } \]

Log-likelihood should generally be preferred in software.

Engineering interpretation: In practical software, log-likelihood evaluation also makes it easier to compare hypotheses by differences in log-likelihood rather than ratios of extremely small likelihood values.

4.3.6.104 Determinants in Gaussian Likelihood

For covariance \(\mathbf R\), the Gaussian normalization includes

\[ |\mathbf R|^{-1/2}. \]

Rather than directly computing

\[ |\mathbf R|, \]

numerically stable methods use matrix factorizations.

If

\[ \mathbf R = \mathbf L\mathbf L^T, \]

then

\[ \boxed{ \ln|\mathbf R| = 2 \sum_i \ln L_{ii}. } \]

This is numerically safer.

4.3.6.105 Avoid Explicit Matrix Inversion

Expressions such as

\[ \mathbf r^T \mathbf R^{-1} \mathbf r \]

should generally not be implemented by explicitly forming

\[ \mathbf R^{-1}. \]

Instead solve

\[ \mathbf R\mathbf y = \mathbf r \]

and compute

\[ \boxed{ \mathbf r^T\mathbf y. } \]

This improves numerical robustness.

4.3.6.106 Cholesky Evaluation of Gaussian Likelihood

For positive-definite covariance,

\[ \mathbf R = \mathbf L\mathbf L^T. \]

Solve

\[ \mathbf L\mathbf y = \mathbf r. \]

Then

\[ \boxed{ \mathbf r^T \mathbf R^{-1} \mathbf r = \mathbf y^T\mathbf y. } \]

This gives a stable way to evaluate the Mahalanobis term and log determinant.

4.3.6.107 Scaling

Optimization variables may contain different units:

• position,
• velocity,
• bias,
• dimensionless coefficients.

Poor scaling can distort numerical optimization of the likelihood or posterior.

Use scaled variables:

\[ \boxed{ \mathbf x = \mathbf S_x \tilde{\mathbf x}. } \]

Scaling changes numerical behaviour but not the underlying physical information if handled consistently.

4.3.6.108 Likelihood Gradient Verification

If analytical likelihood gradients are implemented, compare them against finite differences or automatic differentiation.

Errors in

\[ \nabla\ell \]

can cause

• slow convergence,
• incorrect convergence,
• optimizer divergence.

Derivative verification remains as important in ML/MAP optimization as in nonlinear least squares.

4.3.6.109 Numerical Hessian or Approximate Hessian

Optimization may use

• exact Hessian,
• Gauss–Newton approximation,
• quasi-Newton approximation,
• Fisher information approximation.

The choice depends on

• model structure,
• available derivatives,
• conditioning,
• computational cost.

Again,

\[ \boxed{ \text{Statistical Estimator} \neq \text{Numerical Solver}. } \]

4.3.6.110 Fisher Scoring

A Newton-like method can replace the observed Hessian with expected Fisher information.

The update conceptually becomes

\[ \boxed{ \delta\mathbf x = \mathbf I(\mathbf x)^{-1} \mathbf s(\mathbf x). } \]

This is known as Fisher scoring.

It provides another link between likelihood optimization and information-based estimation.

4.3.6.111 General Comparison: LS, WLS, MLE, and MAP

Their relationship can be summarized as

\[ \boxed{ \text{OLS} \subset \text{WLS} \subset \text{Gaussian MLE} } \]

and

\[ \boxed{ \text{Gaussian MAP} = \text{Gaussian MLE} + \text{Gaussian Prior}. } \]

Engineering interpretation: The table should be read as a hierarchy of assumptions rather than as entirely separate methods. OLS and WLS are optimization forms; under Gaussian assumptions they are also ML estimators, while MAP extends the likelihood with prior information.

4.3.6.112 MLE versus MAP versus Posterior Mean

MLE

\[ \boxed{ \hat{\mathbf x}_{ML} = \arg\max p(\mathbf z|\mathbf x). } \]

Uses likelihood only.

MAP

\[ \boxed{ \hat{\mathbf x}_{MAP} = \arg\max p(\mathbf x|\mathbf z). } \]

Uses likelihood and prior.

Posterior mean

\[ \boxed{ \hat{\mathbf x}_{PM} = E[ \mathbf x|\mathbf z ]. } \]

Uses the complete posterior distribution.

For Gaussian posterior distributions, MAP and posterior mean coincide.

4.3.6.113 Practical MLE Workflow

A disciplined maximum-likelihood workflow is:

1. Define unknown states and parameters.
2. Define the observation model.
3. Define the measurement-error distribution.
4. Define covariance and correlations.
5. Construct the likelihood.
6. Prefer log-likelihood for computation.
7. Derive or compute gradients if required.
8. Verify derivatives.
9. Choose an initial estimate.
10. Select an appropriate optimizer.
11. Monitor likelihood increase or negative-log-likelihood decrease.
12. Check convergence.
13. Examine Hessian or Fisher information.
14. Examine identifiability.
15. Compute uncertainty approximations.
16. Analyse residuals.
17. Test statistical assumptions.
18. Test sensitivity to initialization.
19. Test sensitivity to covariance assumptions.
20. Verify physical plausibility.

Engineering interpretation: The workflow should be iterative at the modelling level. If residual diagnostics or sensitivity tests reveal that the likelihood family, covariance, or independence assumptions are poor, revise the statistical model rather than only changing the optimizer.

4.3.6.114 Practical MAP Workflow

A MAP workflow extends MLE:

1. Define unknown state and parameters.
2. Define measurement likelihood.
3. Define prior distribution.
4. Verify that the prior represents genuine prior knowledge.
5. Form posterior:

\[ p(\mathbf x|\mathbf z) \propto p(\mathbf z|\mathbf x) p(\mathbf x). \]

1. Convert to negative log-posterior.
2. Scale variables appropriately.
3. Select an optimizer.
4. Solve for the posterior mode.
5. Examine prior and measurement information separately.
6. Compute local posterior covariance if appropriate.
7. Test sensitivity to prior assumptions.
8. Check residual structure.
9. Check parameter correlation.
10. Validate against independent data where possible.

4.3.6.115 Complete Maximum-Likelihood Architecture

\[ \boxed{ \begin{array}{c} \text{Unknown State / Parameters}\\ \mathbf x \\ \downarrow\\ \text{Measurement Model}\\ \hat{\mathbf z} = h(\mathbf x) \\ \downarrow\\ \text{Residual}\\ \mathbf r = \mathbf z-\hat{\mathbf z} \\ \downarrow\\ \text{Noise Distribution}\\ p_{\mathbf v}(\mathbf v) \\ \downarrow\\ \text{Likelihood}\\ L(\mathbf x) = p(\mathbf z|\mathbf x) \\ \downarrow\\ \text{Log-Likelihood}\\ \ell(\mathbf x) \\ \downarrow\\ \text{Optimization}\\ \hat{\mathbf x}_{ML} = \arg\max\ell(\mathbf x) \\ \downarrow\\ \text{Residual / Information / Uncertainty Validation} \end{array} } \]

4.3.6.116 Complete MAP Architecture

\[ \boxed{ \begin{array}{c} \text{Prior Knowledge}\\ p(\mathbf x) \\ + \\ \text{Measurement Likelihood}\\ p(\mathbf z|\mathbf x) \\ \downarrow\\ \text{Bayes' Theorem}\\ p(\mathbf x|\mathbf z) \propto p(\mathbf z|\mathbf x)p(\mathbf x) \\ \downarrow\\ \text{Posterior} \\ \downarrow\\ \text{Maximize Posterior}\\ \hat{\mathbf x}_{MAP} \\ \downarrow\\ \text{Posterior Curvature / Covariance / Validation} \end{array} } \]

4.3.6.117 Key Equations

Measurement model

\[ \boxed{ \mathbf z = h(\mathbf x) + \mathbf v } \]

Likelihood

\[ \boxed{ L(\mathbf x) = p(\mathbf z|\mathbf x) } \]

Maximum likelihood estimate

\[ \boxed{ \hat{\mathbf x}_{ML} = \arg\max_{\mathbf x} p(\mathbf z|\mathbf x) } \]

Log-likelihood

\[ \boxed{ \ell(\mathbf x) = \ln p(\mathbf z|\mathbf x) } \]

Gaussian likelihood

\[ \boxed{ p(\mathbf z|\mathbf x) = \frac{ 1 }{ (2\pi)^{m/2} |\mathbf R|^{1/2} } \exp \left[ -\frac12 \mathbf r^T \mathbf R^{-1} \mathbf r \right] } \]

Gaussian negative log-likelihood

\[ \boxed{ -\ell = \frac12 \mathbf r^T \mathbf R^{-1} \mathbf r + \frac12\ln|\mathbf R| + \frac{m}{2}\ln(2\pi) } \]

Mahalanobis distance

\[ \boxed{ d_M^2 = \mathbf r^T \mathbf R^{-1} \mathbf r } \]

Gaussian ML-WLS equivalence

\[ \boxed{ \hat{\mathbf x}_{ML} = \arg\min \mathbf r^T \mathbf R^{-1} \mathbf r } \]

Bayesian posterior

\[ \boxed{ p(\mathbf x|\mathbf z) = \frac{ p(\mathbf z|\mathbf x) p(\mathbf x) }{ p(\mathbf z) } } \]

MAP estimate

\[ \boxed{ \hat{\mathbf x}_{MAP} = \arg\max_{\mathbf x} p(\mathbf x|\mathbf z) } \]

MAP from likelihood and prior

\[ \boxed{ \hat{\mathbf x}_{MAP} = \arg\max_{\mathbf x} p(\mathbf z|\mathbf x) p(\mathbf x) } \]

Gaussian prior

\[ \boxed{ \mathbf x \sim \mathcal N ( \bar{\mathbf x}, \mathbf P_0 ) } \]

Gaussian MAP cost

\[ \boxed{ \begin{aligned} J_{MAP} ={}& \mathbf r^T \mathbf R^{-1} \mathbf r \\ &+ ( \mathbf x-\bar{\mathbf x} )^T \mathbf P_0^{-1} ( \mathbf x-\bar{\mathbf x} ) \end{aligned} } \]

Fisher information

\[ \boxed{ \mathbf I(\mathbf x) = E [ \mathbf s\mathbf s^T ] } \]

Gaussian Fisher information

\[ \boxed{ \mathbf I = \mathbf H^T \mathbf R^{-1} \mathbf H } \]

Cramér–Rao bound

\[ \boxed{ \operatorname{Cov} ( \hat{\mathbf x} ) \succeq \mathbf I^{-1} } \]

Posterior information

\[ \boxed{ \mathbf P_{post}^{-1} = \mathbf P_0^{-1} + \mathbf H^T \mathbf R^{-1} \mathbf H } \]

Local posterior covariance

\[ \boxed{ \mathbf P_{post} \approx \left( \mathbf P_0^{-1} + \mathbf H^T \mathbf R^{-1} \mathbf H \right)^{-1} } \]

4.3.6.118 Key Takeaways

Maximum likelihood introduces a probabilistic interpretation of estimation.

The likelihood is

\[ \boxed{ L(\mathbf x) = p(\mathbf z|\mathbf x). } \]

MLE selects

\[ \boxed{ \hat{\mathbf x}_{ML} = \arg\max L(\mathbf x). } \]

For Gaussian measurement noise,

\[ \boxed{ p(\mathbf z|\mathbf x) \propto \exp \left[ -\frac12 \mathbf r^T \mathbf R^{-1} \mathbf r \right]. } \]

Therefore,

\[ \boxed{ \text{Gaussian Maximum Likelihood} \Longleftrightarrow \text{Weighted Least Squares}. } \]

This is why the familiar WLS cost

\[ \mathbf r^T \mathbf R^{-1} \mathbf r \]

is statistically meaningful.

Taking logarithms does not change the location of the maximum:

\[ \boxed{ \arg\max L = \arg\max \ln L. } \]

The log-likelihood is usually preferred numerically because products become sums.

MAP estimation introduces a prior:

\[ \boxed{ p(\mathbf x|\mathbf z) \propto p(\mathbf z|\mathbf x) p(\mathbf x). } \]

Thus,

\[ \boxed{ \text{Posterior} \propto \text{Likelihood} \times \text{Prior}. } \]

For Gaussian likelihood and Gaussian prior,

\[ \boxed{ \text{MAP} \Longleftrightarrow \text{Prior-Constrained Weighted Least Squares}. } \]

The posterior information matrix is

\[ \boxed{ \mathbf P_{post}^{-1} = \mathbf P_0^{-1} + \mathbf H^T \mathbf R^{-1} \mathbf H. } \]

Therefore,

\[ \boxed{ \text{Posterior Information} = \text{Prior Information} + \text{Measurement Information}. } \]

Fisher information provides a deeper statistical interpretation of the same matrix:

\[ \boxed{ \mathbf I = \mathbf H^T \mathbf R^{-1} \mathbf H } \]

for Gaussian linearized observations.

The Cramér–Rao bound provides a theoretical lower limit on covariance for unbiased estimators under appropriate assumptions:

\[ \boxed{ \operatorname{Cov} ( \hat{\mathbf x} ) \succeq \mathbf I^{-1}. } \]

MLE and MAP are estimation criteria, not numerical algorithms.

Gauss–Newton, LM, Newton, and trust-region methods are ways of solving those optimization problems.

Most importantly,

\[ \boxed{ \text{An Estimate Is Only Statistically Optimal Relative to the Probability Model Assumed}. } \]

If the measurement model, covariance, correlation, likelihood distribution, or prior is wrong, an ML or MAP solution may still be physically incorrect.

4.3.6.119 What Comes Next

Maximum likelihood and MAP estimation determine a state or parameter value from measurements and, in the MAP case, prior information.

However, many spacecraft applications require not only an estimate at the current or reference epoch but also an improved reconstruction of past states using measurements obtained later.

Filtering typically estimates

\[ \boxed{ \hat{\mathbf x}_{k|k} } \]

using measurements available up to time \(k\).

But after additional observations become available through time \(N\), we may ask:

\[ \boxed{ \text{Can future measurements improve our estimate of the state at an earlier time?} } \]

The answer is yes.

Instead of only moving forward in time,

\[ \boxed{ \text{Forward Filtering} } \]

we can also use information backward:

\[ \boxed{ \text{Forward Filter} + \text{Backward Information} \rightarrow \text{Smoothed Trajectory}. } \]

This leads directly to

\[ \boxed{ \textbf{4.3.7 Smoothing and Retrodiction} } \]

where the next section should develop:

• filtering versus smoothing,
• retrodiction,
• fixed-interval smoothing,
• fixed-lag smoothing,
• fixed-point smoothing,
• Rauch–Tung–Striebel smoothing,
• forward/backward processing,
• smoother gain,
• smoothed covariance,
• trajectory reconstruction,
• use of future measurements to improve past-state estimates,
• and the relationship between smoothing and batch estimation.

Next: 4.3.7 Smoothing and Retrodiction

Maximum likelihood and MAP estimation determine the state or parameter value that best explains the measurements under an assumed statistical model. In many trajectory and navigation applications, however, later measurements can also improve estimates of states at earlier epochs.

The next section develops filtering versus smoothing, retrodiction, fixed-interval, fixed-lag, and fixed-point smoothing, Rauch–Tung–Striebel smoothing, forward/backward information processing, smoother gains, smoothed covariance, trajectory reconstruction, and the relationship between smoothing and batch estimation.