AdaptiveAE: An Adaptive Exposure Strategy for HDR Capturing in Dynamic Scenes


ICCV 2025


Tianyi Xu1,3,4*, Fan Zhang1, Boxin Shi3,4†, Tianfan Xue2,1†, Yujin Wang1†,
1Shanghai AI Laboratory, 2The Chinese University of Hong Kong, 3State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University, 4National Engineering Research Center of Visual Technology, School of Computer Science, Peking University

arXiv

Abstract

Mainstream high dynamic range imaging techniques typically rely on fusing multiple images captured with different exposure setups (shutter speed and ISO). A good balance between shutter speed and ISO is crucial for achieving high-quality HDR, as high ISO values introduce significant noise, while long shutter speeds can lead to noticeable motion blur. However, existing methods often overlook the complex interaction between shutter speed and ISO and fail to account for motion blur effects in dynamic scenes.

In this work, we propose AdaptiveAE, a reinforcement learning-based method that optimizes the selection of shutter speed and ISO combinations to maximize HDR reconstruction quality in dynamic environments. AdaptiveAE integrates an image synthesis pipeline that incorporates motion blur and noise simulation into our training procedure, leveraging semantic information and exposure histograms. It can adaptively select optimal ISO and shutter speed sequences based on a user-defined exposure time budget, and find a better exposure schedule than traditional solutions. Experimental results across multiple datasets demonstrate that it achieves the state-of-the-art performance.

Main idea

AdaptiveAE takes camera preview images as input and automatically predicts the ISO and shutter speed for each LDR captures for exposure fusion through a 3-stage sequential refinement procedure to achieve an optimal balance between noise level and motion-related problems for high quality HDR capturing in dynamic scenes with deep reinforcement learning. AdaptiveAE achieves PSNR 39.7 on HDRV dataset, while baseline methods that either only predicts shutter speed or do not consider motion can only achieve PSNR below 37.6 and has evident motion blur and ghosting artifacts in HDR results.

AdaptiveAE Teaser

Video Presentation

Method Overview

Training pipeline of our method. The ISO and shutter speed prediction process is conceptualized as a Markov Decision Process, where a CNN-based policy network predicts the ISO and shutter speed of the next exposure sets. Concurrently, a CNN-based value network estimates the state value. We leverage our blur-aware image synthesis pipeline to synthesize the predicted LDRs and employ DeepHDR to fuse the predicted LDR images, generating our HDR result and calculating the reward for the current policy. The entire system is optimized using the A3C (Asynchronous Advantage Actor-Critic) method.

AdaptiveAE Main Pipeline

Detailed Sequential Decision Process

The training scheme of AdaptiveAE. States are defined as the three LDRs synthesized using predicted ISOs and shutter speeds. Starting from s0 where the three LDRs has EV {−2, 0, +2} with arbitrary EV 0 baseline, ISOs and shutter speeds, the agent sequentially predicts, customizes or inherits capturing parameters (i.e. ISO and shutter speed) for the next stage and synthesize the corresponding LDR using our image synthesis pipeline. Unlike training, the LDRs will be captured rather than synthesized during inference.

AdaptiveAE Side Pipeline

BibTeX

@inproceedings{xu2025adaptiveae,
  title={AdaptiveAE: An Adaptive Exposure Strategy for HDR Capturing in Dynamic Scenes},
  author={Xu, Tianyi and Zhang, Fan and Shi, Boxin and Xue, Tianfan and Wang, Yujin},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  year={2025}
}