I always used to chuckle when I started working with software developers that would sometimes blame issues in the software on "Bad Data". I would picture a bunch of 1s and Zeros lurking in the shadows, drinking and smoking and just waiting until no one is looking...
Of course, that is not what we are talking about here. Data obviously is not good or bad, but if you work with data long enough (or work with flight data analysts and developers), you will, no doubt, hear the term.
What we are really talking about when we refer to "bad data" is data that does not represent what actually happened on the aircraft. In this blog, I will go over some common examples but in a follow-up blog, I will discuss some of the ways we can deal with these issues.
So, what exactly causes bad data?
Commonly, and especially in older recording systems, you might see data "spikes". In the example below there is a spike in both N2 and Pressure Altitude shortly after Engine 1 is shut down (that spike in the data in the right quarter of the chart).
This is pretty common when the aircraft is powering down (or up). But these data spikes can occur at any phase of flight, especially in older aircraft. They do not necessarily mean that something is wrong with the recording system. It just means that, for whatever reason, an incorrect value was sent to the recorder. Many times, multiple parameters will be affected by these power "glitches" as shown in the image above.
No Computed Data (NCD) and No Data Update (NDU)
Sometimes, the Flight Data Acquisition Unit (FDAU) may send "error patterns" to the flight recorder under certain circumstances. For example, if the FDAU is expecting data from an airspeed sensor, but none is being sent (NCD), it may send an error pattern to the recorder. Likewise, if the data has not changed over a period of time (NDU), it may also send an error pattern.
The error pattern is typically just the parameter value alternating between its maximum and minimum value every frame (4 seconds). Most modern data analysis systems should be able to compensate for these, but this could mean that there is an issue on the aircraft.
I would argue that imprecise data is not really "bad", but it is worth mentioning because it needs to be understood and compensated for.
One great example is that of Inertial Navigation System (INS) or Inertial Reference System (IRS) "drift". Most, if not all, modern aircraft today use GPS for positional data but some older aircraft still report INS/IRS position.
This data is very accurate at departure (where it is set by the crew), but its accuracy suffers as the flight progresses. The error is acceptable for navigation purposes but it will typically be unusable for determining locations where more precision is required, such as touch down point and possibly even landing runway.
Again, this is not a problem with the system - just a limitation. But if you are working with flight data, you need to be aware of these limitations.
Conversion Errors (Errors With the Data frame)
Some errors can be a little harder to detect. For example, there could be errors in how the FRED file was programmed. If you are new to my blogs, FRED stands for Flight Recorder Electronic Documentation. It is the file that tells the software how to convert 1s and 0s in the flight data into useful engineering units such as Airspeed and Altitude.
The FRED file must be produced by a human being somewhere along the line (even if the FRED file is generated from ARINC 767 data, someone still had to define it somewhere). Because humans are involved, it is prone to typos and other bugs.
I have added another example below. In this example, the Altitude (in green) drops to zero every other second or so. This could be an issue with how the parameter is defined in the FRED file. It could be that altitude occupies twice as many subframes as it should.
But is this an issue with the FRED file, or the documentation that was used to program the FRED file?
Even if the FRED file itself is coded perfectly according to the aircraft documentation, there is a possibility that the documentation itself has errors.
These types of errors are sometimes obvious, for example, altitude could be double what is expected. But sometimes they are very hard to detect. Imagine if altitude was off by only 10%. That could be harder to catch.
Aircraft Mechanical Issues
Finally, there are times when there really is a mechanical issue with a sensor on the aircraft. It is rare, but it still happens. We have seen issues with individual sensors, such as accelerometers as well as issues with the FDR and FDAU themselves.
Most regulatory bodies mandate that the flight recorder system be checked for reasonableness every 12-24 months. This is an important task because, even though these systems are extremely reliable, like any system, they can be prone to failure.
We have looked at some of the common causes of "bad data". In my next blog, I will take a look at some of the ways we can handle these issues in a software system.
What we are really talking about when we refer to "bad data" is data that does not represent what actually happened on the aircraft.