Implementing Hardware Scaling Filter using Video Process Blt D3D11 DDI and Adding PIX\_FMT\_D3D11 Support to Media Foundation Encoder (mfenc)
In this article, we will discuss the implementation of a hardware scaling filter using the Video Process Blt D3D11 DDI and adding PIX\_FMT\_D3D11 support to the Media Foundation Encoder (mfenc). This article assumes that the reader has a basic understanding of Direct3D 11, Media Foundation, and video processing concepts.
Background
Direct3D 11 provides a set of APIs that allow developers to perform hardware-accelerated graphics rendering. Media Foundation, on the other hand, is a multimedia framework that enables the processing and transmission of audio and video data. Video Process Blt D3D11 DDI is an API that allows developers to perform hardware-accelerated video processing using Direct3D 11.
Implementing Hardware Scaling Filter using Video Process Blt D3D11 DDI
To implement a hardware scaling filter using Video Process Blt D3D11 DDI, we need to create a custom video processor. The custom video processor will perform the scaling operation using the Video Process Blt D3D11 DDI.
The first step in creating a custom video processor is to create a video processing object. The video processing object provides a set of APIs that allow developers to perform video processing operations. To create a video processing object, we need to call the CreateVideoProcessingObject function.
HRESULT CreateVideoProcessingObject(
_In_ REFIID clsid,
_In_ IMFAttributes *pAttributes,
_Out_ IMFVideoProcessingController **ppController,
_Out_ IMFVideoProcessor **ppVideoProcessor
);
The clsid parameter specifies the custom video processor class identifier. The pAttributes parameter specifies the attributes for the video processing object. The ppController parameter specifies the pointer to the video processing controller. The ppVideoProcessor parameter specifies the pointer to the video processing object.
Once we have created the video processing object, we need to set up the input and output media types. The input media type specifies the format of the input video data. The output media type specifies the format of the output video data. To set up the input and output media types, we need to call the SetInputType and SetOutputType functions.
HRESULT SetInputType(
_In_ DWORD dwInputStreamID,
_In_ IMFMediaType *pMediaType,
_In_ DWORD dwFlags
);
HRESULT SetOutputType(
_In_ DWORD dwOutputStreamID,
_In_ IMFMediaType *pMediaType,
_In_ DWORD dwFlags
);
The dwInputStreamID and dwOutputStreamID parameters specify the index of the input and output streams. The pMediaType parameter specifies the input or output media type. The dwFlags parameter specifies the flags for the media type.
After setting up the input and output media types, we need to create the Direct3D 11 device and device context. We can create the Direct3D 11 device and device context using the D3D11_CREATE_DEVICE function.
D3D_FEATURE_LEVEL FeatureLevel;
D3D_DRIVER_TYPE DriverType = D3D_DRIVER_TYPE_HARDWARE;
HMODULE Module = NULL;
D3D11_CREATE_DEVICE_FLAG Flag = 0;
HRESULT hr = D3D11_CREATE_DEVICE(
Module,
DriverType,
0,
Flag,
FeatureLevels,
ARRAYSIZE(FeatureLevels),
D3D11_SDK_VERSION,
&pDevice,
&pFeatureLevel,
&pImmediateContext
);
Once we have created the Direct3D 11 device and device context, we can set up the Video Process Blt D3D11 DDI. To set up the Video Process Blt D3D11 DDI, we need to call the MFCreateVideoProcessorD3D11HwDeviceManager function.
IMFDXGIDeviceManager *pDXGIDeviceManager = NULL;
HRESULT hr = MFCreateVideoProcessorD3D11HwDeviceManager(
&pDXGIDeviceManager
);
After setting up the Video Process Blt D3D11 DDI, we can perform the scaling operation. To perform the scaling operation, we need to call the videoProcessorBltD3D11 function.
HRESULT videoProcessorBltD3D11(
_In_ IMFVideoProcessor *pVideoProcessor,
_In_ DXGI\_MAPPED\_SUBRESOURCE *pSrcResource,
_In_ DXGI\_MAPPED\_SUBRESOURCE *pDstResource,
_In_ D3D11\_RECT *pSrcRect,
_In_ D3D11\_RECT *pDstRect,
_In_ D3D11\_VIDEO\_PROCESS\_CONTEXT *pContext
);
The pSrcResource parameter specifies the source Direct3D 11 resource. The pDstResource parameter specifies the destination Direct3D 11 resource. The pSrcRect and pDstRect parameters specify the source and destination rectangles. The pContext parameter specifies the Video Process Blt D3D11 DDI context.
Adding PIX\_FMT\_D3D11 Support to Media Foundation Encoder (mfenc)
To add PIX\_FMT\_D3D11 support to the Media Foundation Encoder (mfenc), we need to register the PIX\_FMT\_D3D11 media type. The PIX\_FMT\_D3D11 media type specifies the Direct3D 11 format.
To register the PIX\_FMT\_D3D11 media type, we need to call the MFInitiateMediaTypeWriterSinkCallback function. The MFInitiateMediaTypeWriterSinkCallback function allows us to register a callback function that will be called when the Media Foundation Encoder (mfenc) initializes the media type writer.
HRESULT MFInitiateMediaTypeWriterSinkCallback(
_In_ IMFTransform *pTransform,
_In_ IMFMediaType *pMediaType,
_Out_ IMFAsyncCallback **ppAsyncCallback
);
The pTransform parameter specifies the Media Foundation Transform. The pMediaType parameter specifies the media type. The ppAsyncCallback parameter specifies the pointer to the async callback.
Inside the callback function, we need to add the PIX\_FMT\_D3D11 media type to the media type writer.
HRESULT AddD3D11MediaType(
_In_ IMFTransform *pTransform,
_In_ IMFMediaType *pMediaType
)
{
IMFMediaType *pMediaTypeWriter = NULL;
HRESULT hr = MF_E_INVALIDREQUEST;
hr = pTransform->GetOutputStreamMediaType(0, &pMediaTypeWriter);
if (SUCCEEDED(hr))
{
hr = pMediaTypeWriter->CopyAllItems(pMediaType);
if (SUCCEEDED(hr))
{
hr = pMediaTypeWriter->SetGUID(MF\_MT\_SUBTYPE, MFVideoFormat\_ARGB32);
hr = pMediaTypeWriter->SetUINT32(MF\_MT\_VIDEO\_CHROMATICITY,
MFVideoChromaticity\_601);
hr = pMediaTypeWriter->SetUINT32(MF\_MT\_VIDEO\_PRIMARIES,
MFVideoPrimaries\_BT601);
hr = pMediaTypeWriter->SetUINT32(MF\_MT\_VIDEO\_TRANSFER\_FUNCTION,
MFVideoTransferFunction\_709);
hr = pMediaTypeWriter->SetUINT32(MF\_MT\_VIDEO\_GAMMA, MFVideoGamma\_1_0);
hr = pMediaTypeWriter->SetUINT32(MF\_MT\_VIDEO\_LIGHTING,
MFVideoLighting\_ITU\_R\_624_1986);
hr = pMediaTypeWriter->SetUINT32(MF\_MT\_VIDEO\_COLOR\_SPACE,
MFVideoColorSpace\_ITU\_R\_709\_2);
if (SUCCEEDED(hr))
{
hr = MFTranscodeSetD3D11RenderTarget(&pMediaTypeWriter);
}
}
}
return hr;
}
- In this article, we discussed implementing a hardware scaling filter using the Video Process Blt D3D11 DDI. We also discussed adding PIX\_FMT\_D3D11 support to the Media Foundation Encoder (mfenc).
- To implement a hardware scaling filter using Video Process Blt D3D11 DDI, we need to create a custom video processor, set up the input and output media types, create the Direct3D 11 device and device context, set up the Video Process Blt D3D11 DDI, and perform the scaling operation.
- To add PIX\_FMT\_D3D11 support to the Media Foundation Encoder (mfenc), we need to register the PIX\_FMT\_D3D11 media type using the MFInitiateMediaTypeWriterSinkCallback function.
References
- Microsoft Documentation: Direct3D 11
- Microsoft Documentation: Media Foundation
- Microsoft Documentation: IMFVideoProcessor
- Microsoft Documentation: ID3D11VideoProcessorContext
- Microsoft Documentation: Media Foundation Transforms