Xiaomi’s MiLM Plus Releases PROVE: Perception-Aligned Object Removal Metrics RC-S and RC-T With a Real-World Video Benchmark
Video object removal has matured considerably in recent years, with diffusion-based models now capable of plausibly reconstructing shadows, reflections, and structures that were partially hidden behind removed objects. The problem is that the metrics used to score these models have not kept pace. Widely used measures such as PSNR, SSIM, LPIPS, ReMOVE, and CFD frequently produce rankings that conflict with how humans actually perceive the quality of an edited video.
The core difficulty is structural. Object removal is an ill-posed, one-to-many problem - there is rarely a single correct way to fill the space left behind, meaning no definitive ground-truth frame exists to compare against. Pixel-level distance metrics assume such a ground truth, so they are working against the nature of the task from the start. This leads to situations where a visually convincing result scores worse than a blurry or artifact-laden one, simply because the convincing result chose a different but equally valid reconstruction.
To address this, Xiaomi's MiLM Plus team has released PROVE, which stands for Perception-aligned Object Removal Video Evaluation. The framework introduces two new metrics: RC-S, which measures spatial coherence - how well the reconstructed region fits the surrounding scene in a single frame - and RC-T, which measures temporal coherence, assessing whether the filled region remains consistent and stable across frames over time. Both metrics are designed to align more closely with human perceptual judgments rather than pixel-level ground-truth comparisons. The team also releases a real-world benchmark dataset to support standardized testing, which has been a secondary gap in the field since most prior evaluations relied on synthetic or semi-synthetic data.
The practical significance here is that better metrics can change which models get developed and prioritized. If researchers optimize for PSNR, they build different systems than if they optimize for perceptual coherence. PROVE represents an attempt to realign those incentives by giving the field evaluation tools that reward what actually matters visually. Whether RC-S and RC-T gain broad adoption will depend on how well they hold up across diverse removal scenarios, but the benchmark dataset at least provides a shared testing ground for that conversation to happen.
