News

Abstract

We present a single-image data-driven method to automatically relight images with full-body humans in them. Our framework is based on a realistic scene leveraging precomputed radiance transfer (PRT) and spherical harmonics (SH) lighting. In contrast to previous work, we lift the assumptions on Lambertian materials and explicitly model diffuse and specular reflectance in our data. Moreover, we introduce an additional light-dependent residual term that accounts for errors in the PRTbased image reconstruction. We propose a new deep learning architecture, tailored to the decomposition performed in PRT, that is trained using a of L1, logarithmic, and rendering losses. Our model outperforms the state of the art for full-body human relighting both with synthetic images and photographs.

Downloads

Bibtex

@inproceedings{Lagunas2021humanrelighting, title={Single-image Full-body Human Relighting}, booktitle={Eurographics Symposium on Rendering (EGSR)}, publisher={The Eurographics Association}, author={Lagunas, Manuel and Sun, Xin and Yang, Jimei and Villegas, Ruben and Zhang, Jianming and Shu, Zhixin and Masia, Belen and Gutierrez, Diego}, year={2021}, DOI = {10.2312/sr.20211301} }

Acknowledgements

We want to thank the anonymous reviewers for their feedback on the manuscript; also, thanks to Ibon Guillen, and Adrian Jarabo for the occasional discussions about the paper. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (CHAMELEON project, grant agreement No 682080), from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreements No 765121 and 956585, from the Spanish Ministry of Economy and Competitiveness (project PID2019-105004GB-I00), and generous gifts from Adobe Systems.