INOS VMOS Assessment Tool: Redefining Video Quality Assessment for OTT Video

The INOS Video Mean Opinion Score (VMOS) Assessment Tool represents a groundbreaking advancement in evaluating both User Quality-of-Experience (QoE) and Network Quality of Service (QoS) for adaptive video streaming on Facebook. By seamlessly merging these critical aspects, the tool delivers unparalleled benchmarking and optimization capabilities. Built upon an innovative architecture, it integrates high-performance analysis with a user-centric design, ensuring top-notch video quality evaluation across various platforms. Specifically designed for mobile phone testing, the VMOS Assessment Tool integrates seamlessly from the client side, making it ideal for efficient evaluation of mobile video performance.

Features:

Real-Time Analysis at Unprecedented Speed: Experience instantaneous, precise assessments with our tool’s advanced algorithms, ensuring rapid feedback and swift resolution of performance issues.

Enhanced QoE with ITU-T P.1204.3 Compliance: Aligned with the latest ITU-T P.1204.3 standards, the VMOS Assessment Tool offers refined evaluations that adhere to the most current benchmarks for perceptual video quality.

High-Quality Database Integration: Support for up to 8K resolution and 60 frames per second ensures comprehensive analysis of high-definition video content, enabling optimal performance and clarity.

Network QoS Optimization: Improve video playback with our tool’s focus on optimizing start-delay and buffering frequency, leading to smoother viewing experiences.

Integrated QoE and QoS Evaluation: The VMOS Assessment Tool seamlessly combines QoE and QoS metrics, providing a holistic analysis that ensures both user experience and network performance are optimized for superior video quality.

Flexible Device Compatibility and Viewing Distance: The VMOS Assessment Tool is designed to adapt to different streaming device dimensions, including PC, laptop, and mobile phone, and various viewing distances, ensuring optimal video quality regardless of the device or viewing conditions.

Seamless Platform Integration: Designed for effortless compatibility, the VMOS Assessment Tool integrates smoothly with existing video platforms, ensuring a hassle-free transition and minimal operational disruption.

Zero Client-Side Integration Required: The VMOS Assessment Tool manages the entire process, from video playback and network statistics recording to the final MOS score assessment, eliminating the need for any client-side integration.

Architecture Overview:

The INOS VMOS Assessment Tool encompasses multiple stages. Initially, it interacts with the video platform to obtain various encoded files, which are transmitted to the user network based on bandwidth availability. Subsequently, in the packet capturing phase, network packets are recorded into a PCAP file, along with the corresponding SSL decryption log key. During the packets processing phase, network packets are filtered to isolate only those related to video playback and player events. The final stage involves predicting the VMOS score by integrating video playback quality fluctuations, which reflect user QoS, with player events, which indicate network QoS.

INOS Facebook VQA Output Sample:

These output samples are derived from our Facebook quality testing on a mobile network operator in the United Kingdom. The results display a range of evaluation metrics utilized for the final VMOS assessment. Each performance metric is accompanied by geospatial testing locations on the map, time-domain values, and histogram values. The performance metrics will be discussed in the following points:

Facebook Streaming Success:

This metric measures the success rate of logging into Facebook and streaming the video.

Facebook Streaming Start Delay:

This metric measures the time interval between the initiation of video loading and the commencement of video playback.

Facebook Streaming Buffer VMOS: 

This metric assesses the Network QoS VMOS, estimated from platform player events such as start delay, rebuffering event frequency, and rebuffering event duration relative to the original video duration.

Facebook Streaming Resolution per Second: 

This metric indicates the video playback resolutions per second, highlighting that Facebook frequently reduces the resolution to 540 pixels for mobile users.

Facebook Streaming Quality VMOS per Second:

This metric reflects the quality VMOS of video playback per second as a result of video quality fluctuations.

Facebook Streaming Quality VMOS:

This metric assesses the User QoE VMOS, indicating the Quality VMOS for the entire playback sequence, calculated from the Quality VMOS per second.

Facebook Streaming Final VMOS:

This metric represents the final VMOS score by merging both Network QoS and User QoE into a single score that encapsulates the overall experience.

INOS Tool Summary:

The INOS VMOS Assessment Tool is a Comprehensive Video Quality Evaluation tool for adaptive video streaming on Facebook, ensuring optimized user experience and network performance.

The Tool Features Innovative System Architecture by processing stages from obtaining encoded files, capturing and filtering network packets, to predicting the VMOS score.

The Tool Offers Advanced Real-Time Analysis with instantaneous, precise assessments and support for high-definition video content up to 8K resolution and 60 frames per second.

The Tool Provides Seamless Client-Side Integration for Mobile Testing, requiring no client-side integration and adapting to various device dimensions and viewing distances for efficient evaluation of mobile video performance.

The Tool Produces Detailed Output Samples for Comprehensive Evaluation.

The Tool Ensures Compatibility with Other Video Platforms, including YouTube, Shahid, TikTok, and Instagram.

We would like to extend our sincere thanks to Obeidallah Ali, our R&D Director at Digis Squared, for his invaluable contribution to this white paper. His expertise and insights have been instrumental in shaping this content and ensuring its relevance!