Intel Media And Audio For Linux

Mentors: Student: xu jun

<p>FFmpeg DNN (deep neural network) module supports the dnn-based filters, it has two backends, one backend is TensorFlow which invokes TensorFlow C library for model loading and inference, the other backend is native. The native backend is a CPU fallback option when the system does not support TensorFlow, and so we can’t introduce 3rd party library for native mode. The native mode is still in early development stage and the performance has not been tuned yet.</p> <p>This project focuses on the native conv2d layer optimization with c/asm on Intel CPUs. Firstly, we will do some research on how to implement conv2d layer in a way which is convenient to operate parallelly. Secondly, write the corresponding C code for preliminary optimization. Thirdly, add some x86 SIMD optimization for conv2d layer to get better performance.</p>

Mentors: Student: ymdatta

<p>Fuzzing the SOF IPC input to make the SOF FW more resilient. There are few open-source fuzzing tools available such as AFL/Peach/oss-fuzz. The scope of this project involves leveraging the input generation in the open-source fuzzers and integrating it with the SOF fuzzer test harness and QEMU to fuzz the code that parses the topology file and sends the IPCs to the DSP.</p>

Mentors: Student: achie27

<p>Often a low resolution image is needed to be up-scaled to a higher resolution. Even though there are already a lot of interpolation methods which accomplish that, none of them actually make the image look better and natural to the human eye.</p> <p>Super resolution methods output an up-scaled image with improved details. The essentially unknown details in the high resolution output are filled in using different SR techniques.</p> <p>For this project, I will focus on deep learning based SR using, mainly, EDSR and ProGANSR. I will train and model these architectures with PyTorch and convert them to ONNX representations which the DNN module (OpenVINO) of libXcam can infer from.</p>

Mentors: Student: Sergey Zvezdakov

<p>VMAF is a new full-reference perceptual video quality metric developed by Netflix, which has a high correlation with subjective quality scores. This metric is widely used in industry, and I want to add VMAF plugin to GStreamer. Also, I have some additional interesting ideas related to VMAF accelerating and tuning.</p>

Mentors: Student: Georg Ottinger

<p>With the proposed GSOC 2018 project I'd like to contributing to an up-to-date libva-utils, by adding VP8 and HEVC encoding as well as VP8 temporal scaling support. With the help of QR Codes I'd also like to include automated testing for basic image reproduction of the encoders. If time allows optionally temporal scaling support for VP9 is targeted.</p>