An Interview with Amit Mookerjee on AMD’s Media SDK

Making heterogeneous compute easy to use…

AMD - logoBack at AFDS12 AMD announced that they were working on a software development kit (SDK) called the Media SDK. Aimed at enabling developers to leverage the power of AMD’s hardware to accelerate media heavy applications like desktop video conferencing and wireless display. Traditionally AMD has offered the power of its unified video decode (UVD) and video codec engine (VCE) fixed function hardware blocks to developers using Microsoft’s DirectX Video Acceleration (DXVA) application programming interface (API) on Windows or the X-Video Bitstream Acceleration (XvBA) API on Linux.

The problem with these APIs from AMD’s perspective was that they didn’t truly expose potential of AMD’s fixed function hardware and they completely neglected the rest of the compute available on AMD’s heterogeneous platform. These older low-level APIs were also difficult for mainstream developers to work with and required a lot of additional development time to utilize. The solution was clear; AMD need to build its own APIs to get developers to leverage its hardware to the fullest extent.

AMD_Media_SDK_banner

To that end AMD made a public beta of its Media SDK available for download during its APU13 conference back in November. The current iteration of the Media SDK is set of Media Foundation Transforms (MFTs) and libraries for Windows 7/8/8.1 that enable developers to accelerate video decode/encode operations as well as to do GPU accelerated pre/post processing on video streams.

The beauty of AMD’s choice to use MFTs is that developers no longer have to use C++ AMP to do GPU accelerated video processing. Now you can write your program in any of the languages that Microsoft’s development platform supports like C#, Visual Basic, or Javascript and then use an MFT to leverage AMD’s hardware without being forced to specify exactly what computations are happening in the background.

AMD has also included what it calls the AMF-DEM library which allows developers to utilize a direct hardware path from the display controller to the VCE engine. With the AMF-DEM library desktop applications can capture and encode a video stream in a single step minimizing the delay between when the video stream is captured and when it’s transmitted. This is a pretty neat feature that’s almost single-handedly enabled AMD to offer the lowest latency real-time wireless display solution on the market.

AMD Media SDK Sum

We had a chance to interview the Product Manager of the Media SDK, AMD’s Amit Mookerjee, about his latest creation.

What is AMD’s vision for the media SDK? Is it a niche product aimed at developers with very performance sensitive applications (like the APP SDK) or should we expect Microsoft and Google to start using AMD’s APIs and MFTs in their core apps?

The Media SDK is created with the vision to enable developers to provide the best multimedia experiences on the AMD platform. The Media SDK is a cornerstone of AMD’s heterogeneous compute strategy whereby developers can easily leverage CPU, GPU and HW accelerators for media processing. Some key features of Media SDKv1.0 are highlighted below:

Video pre/post processing: The Media SDK provides a GPU optimized library – Video Quality MFT – for video pre/post processing. Features like shake stabilization, dynamic contrast and noise reduction can be used to dramatically improve video conferencing and playback experiences. These features may also be used in cleaning up video in editing or in transcoding applications.

Hardware accelerated video encoding and decoding: The Media SDK provides APIs for leveraging the fixed function hardware blocks – Video Compression Engine (VCE) and Unified Video Decoder (UVD) – for video encoding and decoding respectively.

Low latency encoding: AMD’s Video Compression Engine (VCE) can leverage a direct path to the display controller for low latency video encoding known as Display Encode Mode (DEM). The Media SDK provides a library – AMF-DEM – for leveraging this unique capability in low latency applications like wireless display.

Developers can also use the Media SDKv1.0 to develop Windows 8/8.1 store applications by leveraging the media HW accelerators and GPU for video processing.

AMD Media SDK Devs

AMD provides the most powerful heterogeneous compute platform in the industry today. With support for features like D3D to OpenCL inter-op and samples for building optimal media processing pipelines, the Media SDK makes it easy for multimedia developers to leverage heterogeneous compute.

AMD wants to enable a broad developer community to easily leverage the unique and differentiated multimedia capabilities on the AMD platform. The Media SDK caters to the following developer audiences:

Developers using industry standard APIs like MFT, OpenMAX , ffmpeg for media processing.

Developers that have their own custom framework or looking to leverage features and optimizations not available via industry standard APIs.

AMD Media SDK SS

AMD is working on the AMD Media Framework (AMF) show in the figure above. AMF provides a cross platform C++ API for media processing. Higher level multimedia components like MFT are built on top of AMF. Developers have the option of either using industry standard APIs like MFT or the AMF API. The strategy remains the same as we expand our support to additional operating systems. With this two pronged approach we expect the Media SDK to be attractive to a wide range of developers.

At its 2011 and 2012 developer conferences AMD brought Corel and Adobe on stage to talk about how they were using OpenCL to do computations similar to the workloads that the Media SDK targets with its video pre/post processing capabilities. In our talks with AMD the company made it clear that the Media SDK isn’t designed to replace the lower level methods that major developers have taken to utilize GPU compute for media processing applications. Rather the SDK should compliment existing options by allowing for that compute to be used in Windows Store applications and making those compute capabilities available to a wider array of programmers. More adventurous programmers can try their hand at writing their own video filters in C++ AMP and still utilize AMD’s dedicated video decode and encode hardware through the Media SDK. So whether you want to use AMD’s video filters or roll your own, the Media SDK has something to offer.

AMD’s Media SDK is about making it as easy as possible for developers to leverage the power of AMD’s heterogeneous platform in applications that transform, transmit, display, or capture video. Whether that means using fixed function hardware blocks, GPU compute, or even CPU cores AMD is leveraging the full power of its heterogeneous platform with the Media SDK to differentiate itself from Intel and Nvidia.

A sizable portion of the features in the Media SDK appear to only be useful for enabling AMD’s Wireless Display solution. How did AMD’s focus on creating a high quality wireless display implementation affect the development of the Media SDK?

As mentioned earlier the Media SDK provides the AMF-DEM library for leveraging the unique Display Encode Mode (DEM) mode capability on the AMD platform for low latency video encoding. The AMD Wireless Display Solution leverages the AMF-DEM library. Check out the Splashtop remote desktop application and the ScreenMirror application for other proof points for the AMF-DEM library. We feel that the DEM feature is a significant differentiator for AMD in latency sensitive applications.

Thanks to AMD’s Display Encode Mode (DEM) the company is the only game in-town for real-time capture and wireless display. Thus saying it’s a significant differentiator is a bit of an understatement considering the competitive landscape.

When you presented at AFDS 12 there was talk of having the beta released by the end of 2012 with the final version coming in 2013. But only now near the end of 2013 do we finally have a publicly available beta. Why was there such a large delay in the release schedule for the Media SDK beta?

We wanted to provide a high quality and easy to use SDK for our developer community and were unwilling to make the release until we concluded our challenging internal review process.

AMD Media SDK Roadmap

Currently the Media SDK supports Windows 7/8 and Windows 8 Store applications. In light of the continuing growth of the Android ecosystem and AMD’s continuing efforts to enter the tablet market should we expect to see the Media SDK add support for Android or any other operating systems in the future?

The initial focus of the Media SDK is on Windows. However we will support additional operating systems in the future. The thing to note is our two pronged strategy mentioned earlier centered on the AMF API that allows us to easily support additional operating systems and multimedia frameworks. We do not have a schedule yet for supporting additional operating systems. Stay tuned for further announcements.

AMD has big plans for enabling the use of heterogeneous compute on its platforms and the Media SDK is key to that effort. It’s important to look at this response as an acknowledgement that work is on going and that this is just the tip of the iceberg.

It appears that some of the features originally slated to be a part of the 1.0 version of the Media SDK, namely support for multiple GPUs, have been bumped back to the 1.1 version. Is there a particular reason for this change?

The Multi-GPU feature is currently alpha quality and is offered to a handful of ISVs. We did not want to delay MediaSDKv1.0 to incorporate this feature. We expect multi-GPU support to be production quality in the Media SDKv1.1 release.

This is an understandable choice as APU-based systems will likely see the biggest gain from apps that use the Media SDK’s MFTs rather than systems with multiple GPUs. It’s important to note that the Media SDK isn’t APU-only and discrete graphics will boost the performance of GPU compute operations like video pre/post processing.

One of the concerns that developers had about the Media SDK was the licensing model and how it would affect what kind of applications could take advantage of the SDK. How has AMD decided to license the Media SDK and the use of its IP?

The Media SDK libraries can be incorporated free of charge in any application. However the app provider is responsible for royalty payments on codecs to groups like MPEGLA. Please refer to Sec 2.6 of MediaSDK EULA for more details.

This is a reasonable way to handle licensing and its good to see that AMD’s taking a no cost approach.

Certain features of the Media SDK, like the video quality MFTs, are only supported on the Southern Islands family of devices. Why is that the case?

We optimized the Video Quality MFT for the latest AMD APUs and discrete GPUs. We can support older devices if we get a strong demand from the developer community.

Outside of the features coming in the version 1.1 update should we expect AMD to continue to make more fixed function hardware accessible to developers or has all of the potential in AMD’s UVD/VCE blocks already been exposed?

There will be significant enhancements to UVD and VCE to support new codecs and provide better performance. Future releases of the Media SDK will support these enhanced media accelerators. Stay tuned for further announcements.

It’s clear that AMD’s Media SDK effort has been through some trials and tribulations; but with the beta available now, the release version arriving in January, and public roadmap for a follow-up release in H2 the future of AMD’s Media SDK looks very promising. AMD has spent a lot of time talking both publicly and privately about how strong its commitment to heterogeneous computing is and how important giving developers the tools they need to leverage AMD’s hardware is. With the release of the Media SDK AMD is continuing to make good on that commitment and that vision.

Thanks to Amit Mookerjee and AMD for taking the time to answer our questions.S|A

The following two tabs change content below.
Thomas Ryan is a freelance technology writer and photographer from Seattle, living in Austin. You can also find his work on SemiAccurate and PCWorld. He has a BA in Geography from the University of Washington with a minor in Urban Design and Planning and specializes in geospatial data science. If you have a hardware performance question or an interesting data set Thomas has you covered.