FPS Benchmark: NVidia ShadowPlay vs. AMD GVR vs. FRAPS

FRAPS has been in the gameplay capture business for over a decade now, inarguably serving as the best solution for early gameplay video recordings. The advent of casual streaming and competitive eSports has finally pushed recorded game content to widespread consumption. ISP-provided datarates have mostly stabilized to usable levels, which helps in production and consumption of high bit-rate content.

ShadowPlay was announced as a FRAPS alternative last year by nVidia, and is only compatible with nVidia devices. The tool uses an integrated H.264 video encoder on Kepler and Maxwell hardware, ensuring most the performance drag is loaded on the GPU rather than the CPU; moreover, it's being loaded on specific components of the GPU that are built for video encoding and largely unused while gaming.

FRAPS does no live encoding and only records raw data output, which theoretically means it will have the best quality (lossless), but also demands the most resources in storage and CPU cycles.

Raptr's Gaming Evolved - GVR is being provided alongside AMD driver packages now, supplying AMD users with their own live-encoding software. The GVR -- or "Game Video Recorder" -- can be deployed on AMD, NVidia, and some integrated mobile CPU solutions. Similar to ShadowPlay, GVR uses on-GPU encoders (VCE compliant with AMD) to load the GPU instead of the CPU. This also produces significantly smaller file sizes than the raw, lossless AVI captured by FRAPS.

I decided to benchmark the three utilities to show the true performance hit from using capture software. This will also explore whether or not it makes sense to use GVR with nVidia devices, considering it is compatible and could theoretically replace ShadowPlay if performance were solid enough. In this ShadowPlay vs. FRAPS vs. GVR benchmark, we'll look at framerate hits from capture, percent (delta) advantage between AMD & NVidia devices, and space requirements.

Test Methodology 

The standard GN 2013 test bench was used for all tests. We used two cards, two games, and three software utilities for this benchmark. The tests were conducted in a fashion that shows performance degradation against uncaptured gameplay (baseline). We benchmarked unfiltered gameplay against recorded gameplay with each tool.

There's a problem, though -- AMD and nVidia GPUs can't be compared linearly with raw FPS output due to architectural and performance differences. In order to show which software suite performs best on each GPU, we had to normalize the comparative results and present them as "Percent FPS Degradation" rather than strictly "FPS."

For single-GPU tests, raw FPS can be shown to bring things down to earth. We did both types of tests -- comparative and single-GPU.

Storage and capture quality differences were also tested. All tests were run for two minutes at 50Mbps (GVR, ShadowPlay). FRAPS has an unalterable bit-rate that is discussed below.

The GRID: Autosport FPS benchmark (full game bench here) was conducted a custom "max" setting. Our tuned "max" setting started from the baseline "ultra" preset, but then added Global Illumination, SSAO, and advanced lighting. Metro: Last Light used Very High settings with High tessellation and no SSAO.

All tests were conducted three times for parity, each using the built-in GRID: Autosport graphics benchmark tool and Metro: Last Light benchmark tool. The tools can be accessed through the graphics options and looped until cancelation, at which point an XML file is compiled with results. We used FRAPS' benchmark utility for real-time measurement of the framerate, then used FRAFS to analyze the 1% high, 1% low, min, max, and average FPS; FRAFS was further used to dissect the frametime performance of the GPUs, though that will not be covered here (as it is more related to drivers and GPUs than to GRID). These numbers were matched against internally-captured XML results.

NVidia 340.43 beta drivers were used for all tests conducted on nVidia's GPUs. The 340.43 beta introduced performance improvements specifically for GRID. AMD 14.7 drivers were used for the AMD cards, which contain no notes of GRID-specific improvements.

The system was kept in a constant thermal environment (21C - 22C at all times) while under test. 4x4GB memory modules were kept overclocked at 2133MHz. All case fans were set to 100% speed and automated fan control settings were disabled for purposes of test consistency and thermal stability.

A 120Hz display was connected for purposes of ensuring frame throttles were a non-issue. The native resolution of the display is 1920x1080. V-Sync was completely disabled for this test.

We ran the following settings for all discrete devices under test (DUT):

The video cards tested include:

• AMD Radeon R9 290X 4GB (provided by CyberPower).
• NVidia GTX 780 Ti 3GB (provided by nVidia).

NVIDIA - FPS Performance - ShadowPlay vs. AMD GVR vs. FRAPS 

To the surprise of no one, nVidia appears to perform best when using ShadowPlay over the alternatives. GRID: Autosport and Metro: Last Light both see almost zero measurable impact to total FPS output when using ShadowPlay to capture gameplay. Using Raptr's GVR with an nVidia solution has a noticeable FPS hit, but nothing compared to the likes of FRAPS.

FRAPS brings both games down to borderline unplayable (assuming a desirable FPS of ~60); using FRAPS to record either of these games, you'd probably want to drop quality settings slightly to ensure a more constant 60FPS.

The takeaway here is that ShadowPlay is unbelievably optimized and allows high-quality capture with minimal degradation of the experience.

AMD - FPS Performance - GVR vs. FRAPS

Using only an AMD device -- which is incompatible with ShadowPlay -- we still see relatively high FPS output when capturing gameplay. GRID sees a ~9% performance hit with GVR enabled, Metro sees a ~3.77% performance hit with GVR enabled. In both instances, our framerate still stays well above target levels. Considering where we came from (FRAPS at 59.77% / 28.57% degradation), this is a marked improvement.

Raptr's GVR should be utilized by all gamers on AMD cards with an interest in gameplay capture.

Performance Degradation from Baseline - AMD vs. NVIDIA

"None" (no capture software) has been normalized to appear as 100% performance -- meaning you're getting 100% of your frames delivered. The software tested will degrade performance from 100% (impact to delivered frames), so the lower the percentage, the worse the framerate.

NVidia's ShadowPlay retains almost 100% performance throughput on the nVidia GTX 780 Ti we tested. Frankly, 98.21% total throughput is insanely impressive. There's no reason not to record gameplay at this point, aside from being uninteresting content or not having enough storage. AMD is incompatible with ShadowPlay and was untested.

GVR sees 90.86% performance retention (or 9.14 degradation) with an AMD device. Again, very impressive. Not quite as optimized as ShadowPlay, but still good. With an nVidia device, we see 68.21% performance retention. There's no point to using GVR with an nVidia solution, as far as I can see, unless it offers some sort of game compatibility or desirable feature that ShadowPlay does not.

FRAPS is more equal between the two, but has the biggest hits. We lose more than 50% of our total FPS throughput when capturing with FRAPS. The software still has abundant uses outside of gaming (like benchmarking), but video capture seems best left to the dedicated solutions.

The numbers in Metro are a lot better than those in GRID, but this is almost certainly due to software and/or driver optimization. Metro is a go-to benchmarking solution, so it's likely that the game saw heavy testing when developers were building GVR and ShadowPlay; it is also known that AMD and nVidia drivers have been optimized for Metro.

Storage Requirements, Quality, Bit-Rate of FRAPS, GVR, & ShadowPlay 

All tests had GVR and ShadowPlay set to 50Mbps (highest quality) data-rate capture. Each test was run for two minutes. In theory, this should produce files of about 750MB in size (or 375MB per minute). Further compression and other encoding tasks happen in the background of the software that we are unfamiliar with and will impact filesize.

GVR and ShadowPlay are effectively the same. The quality is high on both and there are few visible differences. I can't objectively criticize such similar quality output, and subjective criticism feels like it is going to be inaccurate due to test expectations / test bias -- so I'll refrain from commenting on specific qualities of capture GVR vs. ShadowPlay. All you need to know is that both GVR and ShadowPlay produce very similar, very high-quality output when captured at 50Mbps.

FRAPS is just insane. There's no live encoding going on, so capture is completely lossless and effectively "What You See is What You Get" output. A two-minute capture required 9.57GB of storage spread across 3 files, and shows why we need several TB just for GN's B-roll gameplay footage folders. To put things into perspective, assuming no other variables are encoded and we're just looking at raw datarate, this is the throughput:

• 9570MB * 8 (bits) = 76560Mb
• 76560Mb / 2 (minutes) = 38280Mb
• 38280Mb / 60 (seconds) = 638Mbps

And for this reason, FRAPS still has objectively crisper output -- but very few viewers will notice that. And, given the datarate of most internet connections and limitations of YouTube, it is entirely irrelevant anyway. Keep in mind that most ISPs in US cities provide something like 30-50Mbps at absolute best (quiet, Kansas!). Also keep in mind that these same ISPs offer something like 1-5Mbps upload. For these reasons, FRAPS data has always required editing, encoding, and rendering through third-party software -- Adobe Premiere, Sony Vegas, etc. -- to ensure a more wieldy filesize and datarate. YouTube won't even support a bit-rate that high and will degrade it anyway, so might as well just use GVR or ShadowPlay.

To put things into perspective insofar as the internet: On my (expensive, but useless) connection, a single second of 638Mbps footage would require nearly 11 minutes to upload to a server. For one second. If I had a 5-minute gameplay video at that datarate, it'd take me 53 hours to upload, and that's assuming you can find a service that will even allow such high-density video.

That said, FRAPS is technically crisper for local viewing. It's just hardly noticeable.

Conclusion 

Somewhat expectedly, you're best off using GVR for AMD solutions and ShadowPlay for nVidia solutions. AMD has worked with Raptr on development of its Gaming Evolved software, and so it is almost a certainty that AMD has provided test setups for GVR. GVR theoretically also functions on some IGPs, though we left those untested for now.

There is little reason to use FRAPS as gameplay capture going forward -- the file sizes are simply too unwieldy and need to be re-rendered anyway. The casual gamer who just wants to upload straight commentary or gameplay (without post-production) would be best off using GVR or ShadowPlay. Both require the same amount of clicks to use, for the most part, so just choose what's best for the video card.

The filesize and performance retention of Raptr / AMD's GVR is stunning, to be honest. It's a powerful start to gameplay capture done efficiently. This is even more the case for ShadowPlay, though with nVidia building specifically for its own encoding solutions, this is perhaps less surprising.

FRAPS will stick around for a while yet. It's useful with CPU-only systems that are incompatible with ShadowPlay and GVR, useful for incompatible GPUs (Fermi and older), and has uses in benchmarking. That said, I'd be shocked if AMD, Raptr, and/or nVidia don't soon develop alternatives. FCAT already exists, so that's a start. We're moving away from the Fermi and older GPUs, so those gains will soon be invalidated unless FRAPS can update to take advantage of hardware.

- Steve "Lelldorianx" Burke.