Beyond the IEKF: Lie-group INS/DVL fusion and honest subsea covariance

A new filter for an old dead-reckoning problem

A paper posted to arXiv in September 2026 by Arihant Lunawat, Pieter van Goor, Frank Dellaert and Stefan B. Williams – Equivariant Filter Design for Acoustic and Depth Aided Inertial Navigation Systems – takes aim at the estimator sitting at the heart of every GPS-denied AUV. The problem it addresses is one we know intimately from running subsea navigation: an inertial measurement unit propagates the state, a Doppler velocity log constrains velocity against the seabed, and a pressure sensor pins depth. The open question has always been how best to blend them.

The authors’ contribution is a Tangent-Group Equivariant Filter (TG-EqF). They argue that the widely used Invariant Extended Kalman Filter (IEKF) has a structural flaw: it appends the IMU biases to the state as a plain Euclidean extension, which breaks the group-affine property the IEKF relies on for its clean, state-independent error dynamics. Their fix carries the biases inside the geometry of the state space. The reported numbers are a reduction of 18-25% in attitude, velocity and position error against both a Two-Frame-Group IEKF and a Multiplicative EKF in Monte Carlo simulation, with offline AUV field data showing reduced position drift. The result they lead with, though, is not the error reduction. It is consistency: the filter’s Average Normalized Estimation Error Squared (ANEES) stays closer to its nominal value of one than the alternatives.

That last point is the one we think is worth unpacking, because it is where the engineering value sits for offshore work.

Why the geometry of the state matters

Start with what a standard EKF does when it linearises. It expands the non-linear motion and measurement models about the current estimate, and the quality of that expansion depends on how good the estimate already is. When the estimate is poor – long dead-reckoning runs, aggressive attitude changes, aiding dropouts – the linearisation point is wrong, and the covariance the filter propagates no longer describes the real error. In our experience the filter becomes overconfident precisely when it should not be.

The IEKF, following Barrau and Bonnabel, gets around much of this by posing the navigation state on a matrix Lie group – rotation, velocity and position bundled into an extended pose. For a system with “group-affine” dynamics, the error defined on that group evolves independently of the state estimate. The linearisation error effectively vanishes for the navigation states, which is why the IEKF earned its place in subsea and aerospace INS design. The theory is elegant and, as we have seen ourselves, the field results are real.

The complication is the IMU biases. Gyro and accelerometer biases are not part of that neat group structure, and the standard move is to bolt them on as ordinary vector states. That bolt-on destroys the group-affine property the whole construction depended on. The exact log-linear error dynamics become approximate again, and – this is the part the paper stresses, and the part we would underline – the covariance drifts away from the true error alongside the estimate. You lose the very consistency guarantee that justified going to a Lie group in the first place.

The Equivariant Filter framework generalises the idea. Instead of forcing the biases into a group that cannot hold them, the Tangent-Group symmetry carries the biases within the state geometry so the symmetry is respected across the whole state, biases included. The consequence the authors claim is concrete: zero linearisation error in the navigation states, second-order error only in the biases, a DVL output model whose update leaves only third-order linearisation error, and a direct pressure output. In plain terms, the estimator behaves the same way regardless of where it sits in state space, and the covariance it reports keeps describing the actual error for longer and across a wider operating envelope.

The real prize is a covariance you can trust

This is the point we press hardest on with any survey or positioning engineer. In subsea navigation the estimated covariance is not a diagnostic curiosity that sits in a log file. It is the quantity almost every downstream decision runs on.

When an INS decides whether to accept or reject an acoustic aiding fix – a USBL update, an LBL range, a sparse GNSS fix at the surface – it does so through a statistical gate built on that covariance. A Mahalanobis test compares the innovation against the predicted uncertainty. If the covariance is too small, the filter rejects perfectly good fixes because they look statistically implausible, and unaided drift grows unchecked. If it is too large, the filter swallows outliers and the position jumps. Get the covariance wrong and the aiding logic that is supposed to bound your drift becomes the thing that lets it run. The discipline here is the same one that underpins rigorous integrity checking in offshore GNSS positioning: the reported uncertainty has to be honest before any gate built on it means anything.

The consistency measure the paper uses, ANEES, is in our view the right yardstick for this. Normalised estimation error squared compares the actual error against the covariance the filter claims; averaged and normalised, a consistent filter returns a value near one. Above one, the filter is overconfident. Below one, it is needlessly conservative. A filter that posts a 20% lower RMS error but drifts to an ANEES of three or four is, to our minds, more dangerous than a slightly less accurate filter that stays near one, because it lies about how much to trust it. The TG-EqF’s claim to stay close to one is the claim that its uncertainty output can be used as an uncertainty output.

That feeds straight into deliverable QA. IHO S-44 Edition 6 defines total horizontal and total vertical uncertainty at a specified confidence level for each order of survey, and the total propagated uncertainty you report against those thresholds is built up from the navigation solution’s covariance. A filter whose covariance degrades faster than its true error is quietly reporting a TPU that does not hold at that confidence level. Your compliance statement against Order 1a or Special Order is then resting on a number the estimator cannot actually back. The same logic applies to construction support, where position confidence bounds decide whether a touchdown or a structure placement is inside tolerance, and to any case where the INS acts as a reference input and its reported quality drives weighting or voting.

Where the claim stops and the field begins

The paper is careful, and we would urge readers to be equally careful about the boundary of what it demonstrates. Two of its three pillars are Monte Carlo simulation and a Two-Frame-Group IEKF and MEKF benchmark; the field component is offline analysis that corroborates the simulation. That is a sound way to develop and evidence a filter. It is not a qualification programme, and we would not read it as one.

Several practical points bound the result:

• The gain scales with IMU grade and mission duration. Biases are the whole story here. For a navigation-grade FOG or RLG unit with small, stable biases, the advantage of carrying biases inside the geometry is real but smaller. For a tactical or MEMS-class IMU whose biases wander, and for the long GPS-denied legs where those biases dominate the drift budget, we expect the structural benefit to be most pronounced. Match the expected improvement to the sensor you actually fly.
• No estimator geometry rescues a bad measurement. The equivariant DVL output model reduces linearisation error in the update. It does nothing about a DVL that loses bottom-lock over soft sediment or beyond its altitude range, that slips into water-track without a clean flag, or that carries a sound-speed error. A systematic velocity scale error of even a few tenths of a percent integrates directly into position drift, and it does so regardless of how cleverly the filter is posed.
• Lever arms and boresight still dominate. The offsets between IMU, DVL and pressure sensor, and the DVL-to-IMU misalignment, are in our experience frequently the largest single contributors to real-world drift. They live outside the estimator’s symmetry and have to be resolved by calibration.
• Consistency also depends on tuning. A well-tuned MEKF can look consistent inside the envelope it was tuned for. The structural argument for the EqF is that it holds consistency across a wider range of conditions without retuning, which is exactly the property we want when a single filter has to serve transit, survey lines, turns and aiding gaps in one dive.

There is also the standing question of what you do when the DVL simply stops helping. When bottom-lock drops out for extended periods, no INS/DVL geometry can hold the solution alone, and we and other teams increasingly look at optical aiding to buy position confidence without acoustics or at other bounding fixes. A better-conditioned covariance makes those aiding sources easier to fuse correctly, but it does not remove the need for them.

What to do with this now

We would treat this development as a filter-core question, not a change to your sensor suite. The recommendations below are the things we would have a positioning lead act on with existing hardware.

1. Interrogate your INS vendor on the estimator, specifically the bias handling. Ask whether the fusion is EKF, MEKF, IEKF or an equivariant design, and ask directly how the IMU biases enter the state. If the answer is a Euclidean bias appended to a Lie-group pose, you now know its consistency weakness by design.
2. Demand consistency evidence, not just accuracy. Request NEES/ANEES or normalised innovation squared (NIS) plots over representative GPS-denied durations, not only RMS position error. A lower RMS figure is worth less than a covariance that stays near nominal through an aiding dropout. Make consistency a line item in the acceptance criteria.
3. Test through the failure envelope you actually face. In factory or on-site trials, force DVL bottom-lock loss for a defined interval, introduce aiding gaps, and drive turns. Where a reference exists – an LBL box-in or surface GNSS – compare the reported covariance growth against true error. That comparison, not a clean straight-line run, is what tells us whether the uncertainty is honest.
4. Re-check your outlier-gating thresholds against real covariance behaviour. If the filter has been running overconfident, your Mahalanobis gate has been silently mis-set and you may have been rejecting valid aiding. Re-derive the gate from measured covariance behaviour.
5. Hold the fundamentals. Calibrate and record IMU-DVL-pressure lever arms and boresight, log DVL sound-speed and scale factor, and monitor bottom-lock quality throughout the dive. The estimator improvement is real, but even a small velocity scale error will still write itself into your position drift.
6. Trace filter covariance through to your S-44 reporting. Confirm that the covariance feeding your TPU actually supports the confidence claim for the order you are certifying, under the conditions the survey was flown, not the conditions the filter was tuned in.

Our honest summary is that a better-posed filter buys you two things of unequal value. The 18-25% error reduction is welcome. The covariance you can trust is the one that changes how we write our integrity case.

---

Based on: Equivariant Filter Design for Acoustic and Depth Aided Inertial Navigation Systems