MSU Video Quality Measurement Tool — Complete Guide & How to Use It

How to Run Objective Video Quality Tests Using the MSU ToolObjective video quality testing is essential when you need reliable, repeatable measurements of how compression, transmission, or processing affect visual quality. The MSU Video Quality Measurement Tool (commonly referred to as MSU VQMT) is a widely used desktop application designed to compare reference and test video sequences using many objective metrics. This article walks you through installing MSU VQMT, preparing test materials, selecting metrics, running tests, interpreting results, and best practices to ensure accurate, meaningful evaluations.

What is MSU Video Quality Measurement Tool?

MSU VQMT is an application developed by the Graphics & Media Lab at Moscow State University for objective video and image quality assessment. It supports a variety of metrics (PSNR, SSIM, MS-SSIM, VMAF in some builds, etc.), frame-alignment tools, and batch processing modes. MSU VQMT compares a reference (original) video to a distorted/test video and computes frame- and sequence-level scores, along with difference visualizations to help locate quality issues.

Installation and System Requirements

• Supported platforms: Windows (main builds), some legacy builds for Linux.
• Typical requirements: modern CPU, sufficient RAM (4+ GB), enough disk space for test sequences, and optionally a GPU if you intend to run GPU-accelerated encoders alongside tests (MSU itself is CPU-based).
• Download: Obtain the latest MSU VQMT release from the official Graphics & Media Lab MSU page or the project repository. Verify checksums if provided.

Installation steps (Windows):

1. Download the installer or ZIP package.
2. Run the installer or extract the ZIP to a folder.
3. If required, install any runtime dependencies (e.g., Visual C++ Redistributable).
4. Launch the application (VQMT.exe).

Preparing Reference and Test Videos

1. Use lossless or high-quality reference: The reference should be the best-available representation (preferably uncompressed or visually lossless) to avoid bias. Common choices: raw YUV, high-bitrate H.264/HEVC, or lossless codecs.
2. Match resolution, frame rate, and color format: Ideally test and reference should match exactly. If not, pre-process them so they align (same width, height, framerate, chroma subsampling, and color space).
3. Avoid re-encoding references: Each additional encode introduces artifacts; keep the reference pristine.
4. Naming and organization: Keep a clear folder structure and descriptive filenames to simplify batch tests.

Preprocessing tips:

• Use FFmpeg to convert and crop videos. Example converting to 4:2:0 YUV 8-bit:

  
  ffmpeg -i input.mp4 -pix_fmt yuv420p -vsync 0 reference.yuv 

• Use exact frame counts; trim start/end using frametime or frame indexes if necessary.

Common Metrics — What They Tell You

• PSNR (Peak Signal-to-Noise Ratio): Simple pixel-wise fidelity metric; higher is better. Good for coarse comparisons but not aligned with human perception.
• SSIM (Structural Similarity): Perceptual metric focusing on luminance, contrast, structure; better correlated with visual quality than PSNR.
• MS-SSIM (Multi-Scale SSIM): Multi-scale variant of SSIM — more robust across resolutions and viewing distances.
• VMAF (Video Multi-method Assessment Fusion): State-of-the-art perceptual metric developed by Netflix; combines several features via machine learning to predict perceived quality. Note: VMAF support in MSU builds varies; you may need an external implementation.
• Other metrics: VIF, IF, ADM — useful for research contexts.

In MSU VQMT you’ll typically choose multiple metrics so you can compare objective values and ensure consistent trends.

Setting Up a Test in MSU VQMT

1. Launch MSU VQMT.
2. Create a new project/session: set project name and output folder.
3. Add pair(s) of videos: select the reference file and the test file. For batch tests, add multiple test files paired with a single reference.
4. Configure format parameters: resolution, frame rate, color space, bit depth — make sure these match your files.
5. Select metrics: check PSNR, SSIM, MS-SSIM, and VMAF (if available). Also enable per-frame results and difference visualizations if desired.
6. Alignment settings: enable temporal and spatial alignment if your test files might have small shifts or delays. MSU provides automatic frame alignment options.
7. Choose processing options: whether to ignore border pixels (useful to avoid codec-typical border artifacts), set chroma weighting, and set averaging method (mean, median).

Running the Test and Monitoring Progress

• Start the measurement run. MSU displays progress with current frame index and interim metric values.
• For batch runs, MSU processes each pair sequentially and writes a CSV or XML summary for later analysis.
• If tests are slow: ensure files are local (not on network shares), close other CPU-heavy apps, and consider splitting large batches into parallel runs on multiple machines.

Interpreting Results

• Numeric outputs: MSU provides per-frame and aggregate values (mean, median, min, max) for each metric. Compare mean values across codecs/encoders.
• Visualizations: difference frames, error heatmaps, and temporal plots help identify where artifacts concentrate (e.g., blockiness, blurring, flicker).
• Cross-metric comparisons: PSNR may disagree with SSIM/VMAF; prioritize perceptual metrics (SSIM, MS-SSIM, VMAF) when subjective quality matters.
• Pay attention to outliers: a low mean may hide occasional severe frames; inspect min/percentile values.

Best Practices for Accurate, Reproducible Tests

• Use high-quality reference files and avoid multiple encodes of references.
• Keep test conditions consistent (same player, display settings, container formats).
• Control viewing conditions for any subjective follow-ups (same monitor, distance, lighting).
• Report metrics with context: metric names, versions, averaging method, preprocessing steps (cropping, color conversion).
• Run multiple passes and seeds for codecs with non-determinism (randomized motion search, parallelization).
• Use statistical tests (paired t-test, Wilcoxon signed-rank) when comparing many samples to ensure differences are significant.

Example Workflow (Practical)

1. Prepare reference: convert raw source to yuv420p, 8-bit, exact framerate.
2. Encode test files with different codecs/settings.
3. Open MSU VQMT, add reference and test pairs.
4. Select PSNR, SSIM, MS-SSIM, enable per-frame CSV output.
5. Run batch, then export CSV.
6. Analyze CSV in Excel or Python: compute mean, median, 95th percentile, and plot curves.
7. Inspect difference frames for samples with worst scores.

Limitations and When to Use Subjective Testing

Objective metrics approximate human perception but can be fooled by certain distortions (e.g., color shifts, certain temporal artifacts). For final quality evaluations—especially for small differences or viewer-critical content—perform subjective user studies (MOS) alongside objective tests.

Conclusion

MSU VQMT is a powerful tool for objective video quality assessment when used with careful preparation, appropriate metrics, and consistent methodology. By combining perceptual metrics, detailed per-frame analysis, and good experimental practices, you can obtain reliable, actionable measurements to guide codec tuning, streaming optimization, and research.