Baseball has been a game of numbers and keeping stats from the start. The New York Morning News printed the first baseball box score in 1845. Other newspapers followed suit with printed tables of statistics after each game.

One of the earliest baseball outcomes measured was the defensive error. Philadelphia shortstop Billy Shindle made 122 of them in 1890. That remains the single-season record. Herman Long holds the major league career record with 1096 errors committed between 1889 and 1904. Those records appear safe unless players start using gloves from the 19th century.

“An error is a mistake by a fielder that allows a batter to reach base, or a runner to advance an extra base, or allows an at bat to continue after the batter should have been put out.” (Baseball-Reference) An official scorer judges whether an error has occurred based on if the play could have been made by “ordinary effort.”

For decades, broadcasters and fans have used the number of errors committed and that stat’s first cousin fielding percentage as the criteria to evaluate the defensive skill of players.

The profound weaknesses in that practice are obvious. Skills such as defensive range and arm strength remain unconsidered. In fact, a player with an above average range exposes himself to the real possibility of more errors.

Consider this example: Two fielders are confronted with an identical set of 100 ground balls. Fielder A gets to 60 of them and makes a perfect fielding play each time. 10 times his arm is too weak to beat the runner. Fielder A recorded 50 outs on the 100 ground balls. The number of errors he commits is zero and his fielding percentage a pristine 1.000.

Fielder B gets to all 100 ground balls, but 10 bounce off his glove or roll through his legs. He also makes five inaccurate throws to first base. The official scorer assigns 15 errors to Fielder B and his fielding percentage is .850. But Fielder B has recorded 85 outs, compared to 50 by Fielder A.

Fielder B is more valuable to his team, but to the average viewer – and traditional box score reader – he looks much worse.

Errors would be an incomplete and misleading indicator of defensive skill even if they were determined in an objective way. Instead, they are based on what a human being thinks should have happened. Bill James has pointed out that a baseball error is the only major statistic in sports determined that way.

Think of the questionable scoring decisions – whether due to an honest mistake or hometown bias – we see all the time. If you doubt that those judgments are subjective, consider that players can call the press box and have scoring decisions overturned. And yet, these decisions are the foundation not only for traditional defensive stats, but also in calculating ERA, one of the most trusted measurements used to evaluate pitchers.

Errors and fielding percentage aren’t meaningless. In fact, they are important components of evaluating defensive skill. But alone they fall far short in providing a complete picture.

Accurate measurements are crucial as defense is being stressed by teams more than ever. Players like Jason Heyward and Elvis Andrus have earned massive contracts in large part because of the defensive skill they offer.

To correct the shortcomings in traditional defensive stats, we’ve seen the creation of several advanced measurements (metrics). Stats like Ultimate Zone Rating and Defensive Runs Saved among others have become available to the public at sites like FanGraphs.

Many fans and broadcasters question the reliability of the new defensive metrics. Variables like range and arm strength are hard to measure. There is no agreement on which defensive metric(s) is best, and different metrics produce dissimilar results. Sample sizes are pretty small considering the tiny number of difficult batted balls hit to each fielder in a season. A couple unlucky or injury-induced bad plays can make a big difference. And even the folks who created the metrics warn against taking them as gospel.

Skepticism is understandable. Let’s try to demystify what’s going on.

New defensive metrics attempt to answer two simple questions: How many plays should the defender have made and how many did he make?

Ultimate Zone Rating (UZR) is set on a scale where 0 (zero) is league average. A positive UZR means the player was better than average as a defender, a negative UZR means below average. UZR is expressed as the number of runs a player saved or cost his team due to defense. If you want a detailed explanation, read this post by Michael Lichtman, who came up with UZR.

UZR uses a zone-based method. The field is broken up into different zones and players are assigned value based on how many plays they make in each zone, compared to how often other players make the play. When Billy Hamilton makes a catch in centerfield that no other CF snags, he gets more credit than for a routine play. UZR uses many years of data when deciding how much to award or penalize a player. It’s also park adjusted.

Defensive Runs Saved (DRS) is calculated by The Fielding Bible, run by John Dewan. DRS is similar to UZR in many ways. DRS uses the same scale, with zero as league average, uses zones and adjusts for park factors. The biggest difference is that DRS uses a one-year sample when comparing plays. DRS also uses smaller zones because they think that makes their ratings more precise. DRS also factors in defensive positioning.

Both UZR and DRS are based on every play in each game. People employed by Baseball Info Solutions watch game video and code players and zones. They determine ball velocity and angle using objective criteria. But humans don’t estimate the difficulty of a play. A computer algorithm compares one play to others.

But even that limited role for humans is about to change. Statcast – MLB’s revolutionary, state-of-the-art tracking technology – is now measuring defensive data much more precisely than we’ve ever had before. Its data will take the small residual human element out of coding. UZR and DRS can use Statcast data to refine estimates of the exact amount of ground a player cover, as well as the velocity and angle of the hit ball. Statcast also provides an exact location for all defensive players at the time of the pitch.

“Statcast is taking the zone-based fielding models further and further,” said Sam Grossman, Reds assistant general manager, to our Redleg Nation Q&A group earlier this month. “We’re doing a lot of things we couldn’t do two years ago when we just had the zone and where we thought the player started. It gives us a much more accurate view of what a guy’s range is.”

Yes, Statcast still has data gaps that must be plugged. But it’s a gigantic stride forward in measurement, with improvements each year.

With that context, let’s talk Eugenio Suarez and what to make of his defensive ability.

Suarez has a positive UZR score in 2016. His range (+5.5 runs) and double play (+0.4 runs) skills have offset his negative error (-3.1 runs) factor. Arm strength – and that’s a plus for Suarez – isn’t measured for infielders. Overall, Suarez has a positive 3.4 UZR when extrapolated to a 150-game season. Suarez has a +1 runs DRS score. That stands in stark contrast to his extremely negative ratings from 2015 when he played shortstop.

There’s sense to be made from all that. The skills that are portable from Suarez’s time at shortstop – range and turning double plays – are his strengths at 3B relative to others who play that position. Suarez gets to more balls. But he’s still the same bad-hands, erratic-throws guy that he was at shortstop for the Reds last year.

Suarez leads all NL players in errors committed with 22 (Billy Shindle can rest easy). Suarez was similarly error-prone at shortstop, with 19 in just 96 games for the Reds in 2015. Errors are glaring; range relative to other 3B is less obvious, even to those of us who watch every night. That explains the paradox of how Suarez often can look so bad in the field but yet have positive advanced metrics. Moving his shaky glove from SS to 3B has helped increase his net defensive value to the team.

It’s also important to note that Suarez’s 2016 numbers are all pretty small in absolute terms. We’re dealing with defensive runs above average, not wins. The rule of thumb is that a 10-run swing equals one win. Most individual runs don’t impact the win or loss in a game. In a depressed run environment, maybe 9 runs per win is right.

A UZR or DRS of +5 is considered “above average” but not “great.” It takes a score of +10 runs to be “great.” A player with a +15 UZR/DRS score is considered Gold Glove caliber. Billy Hamilton’s UZR/150 is 16.8 runs and his DRS is +14.

A defensive player with a UZR or DRS that’s +/- a few runs should be considered simply average. That’s Eugenio Suarez at third base.