How We Test Tablets

Choosing the right tablet is complicated. You have to choose among at least five operating systems. You have to decide which size is right for you. You need to figure out which system has the apps you're looking for.

But performance is also an important aspect of a new tablet. You don't want a sluggish browser that's going to leave you drumming your fingers waiting for pages to load. You want a camera that takes good pictures and smooth video. And you don't want a battery that takes a whole day to recharge. To help you evaluate tablet performance and make an informed choice, the PCWorld Lab has developed a series of tests based on the same kinds of tasks you'd perform with your tablet - browsing the web, taking and viewing pictures and video, playing games and recharging the battery.

Video: How We Test Tablets

We're able to run most of these tests in equivalent ways on tablets with different operating systems, allowing us to provide objective data to help you choose among models running iOS, Android, WebOS and other operating systems.

Tablets are ideal for casual web surfing, so it's important to know how quickly a tablet's browser can load complex websites featuring plenty of images and video. Much of the what you see in modern interactive web pages is created through JavaScript, so our analysts use the Sunspider JavaScript benchmark to test the performance of the most up-to-date version of the tablet's built-in browser. Next, we access a local webserver on our private network via Wi-Fi and time how long it takes the tablet to load a copy of a standard PCWorld HDTV index page, and to load a 30MB Web page with a mix of text, tables and large images. Our lab analysts conduct both tests twice to ensure consistency, and if results vary by more than 10 percent we test a third time and drop the outlying result.

We also test tablet start time by recording how long it takes a tablet that is powered off to display the main operating system screen. Every second counts when you're regularly flipping your tablet on and off, so we start timing the tablet with a stopwatch as soon as the power button is pressed and repeat three times to ensure accuracy.

Tablets are also great for watching movies or listening to music while you're out and about, so it's important to know how easy it will be to load your device up with your favorite media. To test file transfer speed, we connect the tablet to a desktop computer running 64-bit Windows 7 Ultimate with a 2.8 GHz Intel Core i7 processor and 4GB of DDR3 RAM. We test both the tablet's read and write transfer speeds by copying and pasting 3GB of text, audio and video files from the tablet to the PC and vice-versa.

These file transfer tests are designed to test the tablet's built-in storage throughput. For directly comparable results, the tablet must be able to connect to the desktop, and mount as a new drive that's accessible in Windows file manager. Since Apple's iOS requires iTunes to transfer files to the iPad, we don't include it in these tests. Each test is conducted twice to ensure accuracy.

Mobile gaming is another popular tablet pastime, so our lab team tests the graphical capabilities of every tablet by running both the Egypt and Pro versions of GLBenchmark 2.1. We run the tests first with anti-aliasing on and later with anti-aliasing off, repeating both tests three times and averaging the results to produce four different scores measured in frames per second.

Of course owning a tablet that holds all your favorite movies and plays all your favorite games is pointless if the battery can't last through one long plane trip. Our lab analysts evaluate a tablet's battery life during typical use by playing a video file at maximum brightness with audio set at a volume level loud enough to be heard clearly in an office setting. The video file is a high-definition animated trailer that plays at 720p continuously in a loop until the battery gives out, at which point the total battery life will be logged by either our automated timer application (on Android tablets) or one of our trusty testers standing by with stopwatch in hand.

After the tablet's battery is completely drained our lab team times how long it takes the device to recharge to 100 percent power when plugged into an AC power outlet in our office.

Every tablet that includes a camera undergoes further testing in our digital camera testing area.

First, our lab analysts shoot a still life on automatic settings in an artificial daylight environment with the flash off to test exposure and color levels. If the tablet includes a flash we take an additional shot with the flash on. We then shoot an image of a target resolution chart to test how well the camera sensor can capture sharp details. If the tablet can capture video, we shoot two test videos of an object in motio: one in a well-lit room and another under low-light conditions.

We print out all test images on a Fujifilm Pictrography 3500 printer that is calibrated before every print job. We print the images in an 8-by-10-inch format and they're then judged by a rotating panel of judges under 5000K flood lights. Each judge rates an image with one of 5 word scores: Poor, Fair, Good, Very Good or Superior. All videos are judged on the same scale on a 30-inch HP LP3065 LCD monitor pre-calibrated using a Pantone ColorMunki. Finally, we perform a subjective test of the tablet's display by comparing how an image looks in dim lighting on several other tablets as well as the calibrated 30-inch HP LCD monitor.

After our lab team has finished testing a tablet, all results are double-checked and entered into our database for use in product reviews and analysis.

For comprehensive coverage of the Android ecosystem, visit Greenbot.com.

Subscribe to the Tablet Tips & Trends Newsletter

Comments