Onboard video: H.264, bandwidth and staying in sync

What video is actually for

A camera is the cheapest instrument on the aircraft and often the most persuasive. It shows the store separating, the flap moving, the flow cone, the panel that flexed, the hand that moved the switch a second before the parameter changed. None of that needs a sensor, a calibration or a wiring diagram.

It is also the only channel whose usefulness collapses quietly. A stream that was recorded at the wrong exposure, out of focus, or with a timing offset nobody measured still plays back perfectly. It just cannot be used as evidence. Video is easy to acquire and hard to acquire well, which is the opposite of most instrumentation problems.

Bit rate is the whole problem

Raw video is out of the question. A single 1080p25 channel uncompressed is roughly 1 Gbit/s, which is the entire budget of a fast recorder for one camera. So everything about onboard video is a negotiation about how much compression you can accept.

Two observations follow from that table. First, video dominates the storage budget: four 1080p cameras for a three-hour sortie is more data than every bus and analog channel on the aircraft combined. Second, storage is not the constraint people expect it to be — a modern cartridge swallows all of it, and the real constraints are the telemetry link and the review time of whoever has to watch it.

What H.264 does, and what it costs you

H.264 gets its compression by describing most frames as changes from other frames. An I-frame is coded on its own. A P-frame says what moved since the last one. A B-frame interpolates between a past and a future frame, which is why it compresses so well — and why the encoder has to hold frames and emit them out of order.

A group of pictures is the repeating pattern of those frames. Inside a GOP, only the I-frame stands alone; everything else depends on its neighbours. That single fact drives all the consequences that matter on a test aircraft.

• Seeking is only accurate to a GOP unless the decoder replays from the previous I-frame. With a two-second GOP, "go to the moment of the event" lands up to two seconds away.
• A corrupted I-frame damages every frame that depends on it — a single bad block on a link or a medium costs a whole GOP, not a frame.
• B-frames add latency, because the encoder cannot emit a frame it has not yet seen the future of. For a live downlink, configure without them.
• Variable bit rate is right for storage and wrong for a telemetry link, where the budget is fixed and a burst simply does not fit.

Intra-only: when every frame must stand alone

The alternative is to compress each frame independently — JPEG 2000, or H.264 configured all-intra. You pay five to ten times the bit rate and you get properties that are hard to obtain any other way: every frame is a picture in its own right, seeking is frame-accurate, an error damages one frame and stops there, and quality does not breathe with the motion in the scene.

That is the right choice whenever a frame is evidence rather than illustration: separation events, structural failures, anything you will step through frame by frame, and anything a third party will re-analyse. It is also the natural fit for image recording in a crash-protected installation, where a truncated recording has to remain readable up to its last complete frame.

For a cockpit view over a three-hour sortie, inter-frame compression is obviously right. For the four seconds around a store release, it is obviously wrong. Most campaigns need both, and the useful question is per camera, not per aircraft.

Keeping video and data in the same timeline

One frame at 25 fps is 40 milliseconds. Half a frame of timing error is 20 milliseconds — invisible when you watch the video, and glaring when you put it next to a parameter sampled at 1 kHz. Getting that alignment right is the difference between video that supports an analysis and video that undermines it.

The rule is simple and frequently broken: the time stamp belongs to the moment of capture, not to the moment the compressed frame comes out of the encoder. Encoders buffer, reorder and vary their delay with scene content; a stamp taken at the output inherits all of that.

The container then has to preserve it. In a Chapter 10 recording, video goes into its own packets carrying the frame timing; the D.T.MUX format marks each frame boundary and its length inside the block, so a reader can resynchronise on the next frame instead of losing the rest of the channel — and a header field carries the offset of the first image in the block, which is what allows sub-block alignment.

With more than one camera, add the question of alignment between them. Two independently clocked cameras drift, and a stereo or multi-view analysis needs them locked to a common reference — either genlocked, or at minimum stamped by the same acquisition unit rather than by themselves.

Record, stream, or both

A telemetry link that comfortably carries a few megabits per second of PCM cannot also carry four 1080p cameras. The usual architecture is therefore asymmetric: full-rate video recorded onboard, and a reduced stream — lower resolution, lower frame rate, one camera at a time, switchable from the ground — sent down so the flight-test engineers can see what they are talking about.

On a bench or in an integration hall the calculation changes: Ethernet is free, and streaming the full rate to a ground station gives a live view with no compromise at all. The recorder is then a safety net rather than the primary path.

Cameras and installation

• Lock the exposure, the gain and the white balance. Automatic exposure hunting through a cloud makes two frames of the same event incomparable, and auto-gain quietly adds noise you will later mistake for detail.
• Fix the focus. Nothing on a test aircraft is at a distance that changes, and an autofocus is a mechanism waiting to be shaken.
• Mind the mounting. A camera is a lever arm on a bracket: it sees the vibration of the structure it is bolted to, and a resonance in the bracket ruins the image without anything failing.
• Cable length, shielding and connectors are the usual cause of an intermittent camera — and of an EMC finding in the chamber.
• Cold and condensation at altitude, then a warm descent, is what fogs a lens. A sealed housing with a controlled window is the answer.
• Qualify the camera like any other airborne equipment. A commercial module in a machined housing is a common and legitimate approach, but the environmental evidence is still yours to produce.

On the ground

Video earns its place at replay, next to the parameters. A cursor that moves through both at once, a burnt-in time and GPS overlay for a report, a clip extracted around an event with the same time reference as the data: that is what turns a recording into an argument in a review.

Which is the same requirement as everywhere else in an instrumentation chain — one clock, honestly recorded. Video simply makes the errors visible, because a human eye is very good at noticing that the switch moved after the parameter it caused.