Which airline ratings are the most trustworthy?

Every week, it seems a company or media organization claims it has the definitive list of the best and worst airlines. But what are the best airline ratings?

If you’re a numbers person, you can easily eliminate some of them. For example, one survey claims to poll nearly 20 million people online. Highly unlikely. Others don’t even tell you how many passengers were surveyed, or how they qualified these participants. And few — very few — use hard performance data to back up their results.

By the way, Harrison’s case was not difficult. After four luggage-less days, she just wanted Air Canada to pay for the extra clothes and toiletries she had to buy, plus cover her cell phone bill.

“I don’t think this is too much to ask for after all of the time and effort I have gone through to get my luggage,” she told me.

I shared the Air Canada executive contacts from my consumer advocacy site and offered to help Harrison see her claim through on my help forum.

How do I know the best airline ratings?

I’ve covered the airline industry for years, both as a journalist and a consumer advocate. My nonprofit consumer advocacy organization collects consumer complaints and reports them every month. I’ve been able to overlay the complaints data with the surveys to determine which ones are talking to actual consumers — and which ones are interviewing random people online.

Here are the top 25 companies since the beginning of 2019. Note: these are not one-off questions sent to my advocacy site. Consumers must complete a detailed form to get help. Last month, we received nearly 1,000 requests for assistance, a new record.

Yep, that’s a lot of airline complaints. And look, there’s Air Canada at number 19!

Here are the best airline ratings and here’s how they do it

Based on my observations, there are only a handful of airline quality measures to which I regularly refer.

Airline Quality Rating

The annual Airline Quality Rating (AQR), which will be released tomorrow morning, is a compilation of government data on airline performance. The rating is a multifactor examination of the airlines based on mishandled baggage, consumer complaints, on-time performance and involuntary denied boardings, reported to the U.S. Department of Transportation.

American Customer Satisfaction Index

The American Customer Satisfaction Index (ACSI), which normally releases its airline index in late April, is a rigorously researched measure of customer sentiment. Each year, ACSI surveys 300,000 customers, using the results to benchmark customer satisfaction with more than 400 companies in 46 industries and 10 economic sectors — including airlines.

Routehappy

Routehappy doesn’t rate airlines. Instead, it gives you the information you need to rate an airline. When it’s integrated into a booking engine like Expedia, Google Flights, or Hipmunk, you can quickly pull up comprehensive information on flight amenities such as Wi-Fi, in-seat power, entertainment, seats, food, beverages, and aircraft type. I like Routehappy because airline quality is personal, and in the end, it’s your seat and onboard amenities that matter the most.

Is that it? Well, no. There are numerous other airline ratings, and a few of them are worth paying attention to. Most of them are complete nonsense, of course. To find out how to separate the wheat from the chaff, I consulted the experts.

How to tell if you’ve found the best airline ratings

“Travelers should definitely read the methodology to understand how the study defined quality,” advises Haley Berg, an economist at Hopper, a flight search engine. “You want to make sure the variables they looked at match what you value most when selecting a flight.”

But that can be tricky. If you take mostly short-haul flights, you may be willing to compromise on comfort, but on-time arrival may be a priority. If you’re a business traveler, you may be less price-sensitive but care about service.

“It’s easier to utilize these rankings when they’re only looking at the factor you care about most instead of an overall ranking,” says Berg.

David Van Amburg, managing director of the ACSI, says you should also look at who’s behind the study.

“Any study directly affiliated with an airline should be viewed suspiciously,” he says.

If possible, find out what the survey asked. “Any of the popular one-question studies implemented to measure consumer experiences should be viewed with suspicion as well; they’re cheap and easy, which means the data probably isn’t as reliable,” he adds.

Finally, check the survey’s track record.

“Look at longevity,” says Brent Bowen, the Embry-Riddle Aeronautical University professor who publishes the AQR. “There seems to be a top 10 rating of everything today. Look to see if the survey is just a popularity poll or hard data.”

If you start applying these hard templates to the airline ratings out there, they start to fade to irrelevance. Some reader polls don’t bother to ask if the respondents even flew on the airlines they’re rating. Others have only been around a few years or were published on a whim by a blogger who wanted to make a quick buck. And others are commercial enterprises that want to sell their awards to the industry but do the bare minimum to keep out of trouble with the statistics police.

How to use the best airline ratings

A minimalist approach to the best airline ratings may help you select the best flight. Use a few trusted sources to make your next booking decision — if you have a choice of carriers — and you can’t go wrong. Which is to say, check the ACSI, AQR and the Routehappy data before reserving a seat on the most convenient flight for you.