{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,9,5]],"date-time":"2024-09-05T01:13:32Z","timestamp":1725498812588},"reference-count":45,"publisher":"Wiley","issue":"4","license":[{"start":{"date-parts":[[2019,7,30]],"date-time":"2019-07-30T00:00:00Z","timestamp":1564444800000},"content-version":"vor","delay-in-days":29,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Computer Graphics Forum"],"published-print":{"date-parts":[[2019,7]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We suggest a method to directly deep\u2010learn light transport, i. e., the mapping from a 3D geometry\u2010illumination\u2010material configuration to a shaded 2D image. While many previous learning methods have employed 2D convolutional neural networks applied to images, we show for the first time that light transport can be learned directly in 3D. The benefit of 3D over 2D is, that the former can also correctly capture illumination effects related to occluded and\/or semi\u2010transparent geometry. To learn 3D light transport, we represent the 3D scene as an unstructured 3D point cloud, which is later, during rendering, projected to the 2D output image. Thus, we suggest a two\u2010stage operator comprising a 3D network that first transforms the point cloud into a latent representation, which is later on projected to the 2D output image using a dedicated 3D\u20102D network in a second step. We will show that our approach results in improved quality in terms of temporal coherence while retaining most of the computational efficiency of common 2D methods. As a consequence, the proposed two stage\u2010operator serves as a valuable extension to modern deferred shading approaches.<\/jats:p>","DOI":"10.1111\/cgf.13783","type":"journal-article","created":{"date-parts":[[2019,7,30]],"date-time":"2019-07-30T12:12:02Z","timestamp":1564488722000},"page":"207-217","update-policy":"http:\/\/dx.doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Deep\u2010learning the Latent Space of Light Transport"],"prefix":"10.1111","volume":"38","author":[{"given":"P.","family":"Hermosilla","sequence":"first","affiliation":[{"name":"Ulm University  Germany"}]},{"given":"S.","family":"Maisch","sequence":"additional","affiliation":[{"name":"Ulm University  Germany"}]},{"given":"T.","family":"Ritschel","sequence":"additional","affiliation":[{"name":"University College London  United Kingdom"}]},{"given":"T.","family":"Ropinski","sequence":"additional","affiliation":[{"name":"Ulm University  Germany"},{"name":"Link\u00f6ping University  Sweden"}]}],"member":"311","published-online":{"date-parts":[[2019,7,30]]},"reference":[{"key":"e_1_2_8_2_2","doi-asserted-by":"publisher","DOI":"10.1201\/b22086"},{"key":"e_1_2_8_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/1356682.1356686"},{"key":"e_1_2_8_4_2","article-title":"Dynamic ambient occlusion and indirect lighting","volume":"2","author":"Bunnell M.","year":"2005","journal-title":"GPU Gems"},{"key":"e_1_2_8_5_2","doi-asserted-by":"crossref","unstructured":"BakoS. VogelsT. McWilliamsB. MeyerM. Nov\u00e1kI. HarvillA. SenP. DeRoseT. RousselleF.: Kernel\u2010predicting convolutional networks for denoising monte carlo renderings.ACM Trans. Graph. (Proc. SIGGRAPH) 36 4 (2017).","DOI":"10.1145\/3072959.3073708"},{"key":"e_1_2_8_6_2","unstructured":"ChangA. X. FunkhouserT. GuibasL. HanrahanP. HuangQ. LiZ. SavareseS. SavvaM. SongS. SuH. et al.: Shapenet: An information\u2010rich 3D model repository.arXiv:1512.03012(2015)."},{"key":"e_1_2_8_7_2","unstructured":"ChristensenP. H.:Course : Global Illumination Across Industries \u2010 Point\u2010based Global Illumination for Movie Production. Tech. rep. Pixar 2010."},{"key":"e_1_2_8_8_2","doi-asserted-by":"crossref","unstructured":"ChaitanyaC. R. A. KaplanyanA. S. SchiedC. SalviM. LefohnA. NowrouzezahraiD. AilaT.: Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder.ACM Trans. Graph. (Proc. SIGGRAPH) 36 4 (2017).","DOI":"10.1145\/3072959.3073601"},{"key":"e_1_2_8_9_2","doi-asserted-by":"crossref","unstructured":"DahmK. KellerA.: Learning light transport the reinforced way.arXiv:\/1701.07403(2017).","DOI":"10.1145\/3084363.3085032"},{"key":"e_1_2_8_10_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.aar6170"},{"key":"e_1_2_8_11_2","doi-asserted-by":"crossref","unstructured":"GautronP. K\u0159iv\u00e1nekJ. BouatouchK. PattanaikS.: Radiance cache splatting: A GPU\u2010friendly global illumination algorithm. InACM SIGGRAPH 2005 Sketches(2005).","DOI":"10.1145\/1187112.1187154"},{"key":"e_1_2_8_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/1141911.1141998"},{"key":"e_1_2_8_13_2","doi-asserted-by":"crossref","unstructured":"HermosillaP. RitschelT. VazquezP.\u2010P. VinacuaA. Ropins kiT.: Monte Carlo convolution for learning on non\u2010uniformly sampled point clouds.ACM Trans. Graph. (Proc. SIGGRAPH Asia) 37 6 (2018).","DOI":"10.1145\/3272127.3275110"},{"key":"e_1_2_8_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/1609967.1609970"},{"key":"e_1_2_8_15_2","unstructured":"KainzF. BogardR.:Technical introduction to OpenEXR 2009."},{"key":"e_1_2_8_16_2","doi-asserted-by":"crossref","unstructured":"KellerA.: Instant radiosity. InProc. SIGGRAPH(1997) pp.49\u201356.","DOI":"10.1145\/258734.258769"},{"key":"e_1_2_8_17_2","unstructured":"KarA. H\u00e4neC. MalikJ.: Learning a multi\u2010view stereo machine.arxiv abs\/1708.05375(2017)."},{"key":"e_1_2_8_18_2","doi-asserted-by":"crossref","unstructured":"KuznetsovA. KalantariN. K. RamamoorthiR.: Deep adaptive sampling for low sample count rendering.Comp. Graph. Forum (Proc. EGSR) 37 4 (2018) 35\u201344.","DOI":"10.1111\/cgf.13473"},{"key":"e_1_2_8_19_2","doi-asserted-by":"crossref","unstructured":"KallweitS. M\u00fcllerT. McwilliamsB. GrossM. Nov\u00e1kJ.: Deep scattering: Rendering atmospheric clouds with radiance\u2010predicting neural networks.ACM Trans. Graph. (Proc. SIGGRAPH Asia) 36 6 (2017).","DOI":"10.1145\/3130800.3130880"},{"key":"e_1_2_8_20_2","unstructured":"Leimk\u00fchlerT. SinghG. MyszkowskiK. SeidelH.\u2010P. RitschelT.: End\u2010to\u2010end sampling patterns.arXiv:1806.06710(2018)."},{"key":"e_1_2_8_21_2","unstructured":"LiuH.\u2010T. D. TaoM. JacobsonA.: Paparazzi: Surface editing by way of multi\u2010view image processing.ACM Trans. Graph. (Proc. SIGGRAPH Asia)(2018)."},{"key":"e_1_2_8_22_2","doi-asserted-by":"crossref","unstructured":"LehtinenJ. ZwickerM. TurquinE. KontkanenJ. DurandF. SillionF. X. AilaT.: A meshless hierarchical representation for light transport.ACM Trans. Graph. (Proc. SIGGRAPH) 27 3 (2008).","DOI":"10.1145\/1360612.1360636"},{"key":"e_1_2_8_23_2","doi-asserted-by":"crossref","unstructured":"MittringM.:Course : Advanced real\u2010time rendering in 3D graphics and games \u2010 Finding Next Gen \u2010 CryEngine 2 2007.","DOI":"10.1145\/1281500.1281671"},{"key":"e_1_2_8_24_2","unstructured":"MaraM. McGuireM. NowrouzezahraiD. LuebkeD.: Deep G\u2010buffers for stable global illumination approximation. InProc. HPG(2016)."},{"key":"e_1_2_8_25_2","unstructured":"M\u00fcllerT. McWilliamsB. RousselleF. GrossM. Nov\u00e1kJ.: Neural importance sampling.arXiv:1808.03856(2018)."},{"key":"e_1_2_8_26_2","doi-asserted-by":"crossref","unstructured":"NalbachO. ArabadzhiyskaE. MehtaD. SeidelH.\u2010P. RitschelT.: Deep shading: Convolutional neural networks for screen\u2010space shading.Comp. Graph. Forum (Proc. EGSR) 36 4 (2017).","DOI":"10.1111\/cgf.13225"},{"key":"e_1_2_8_27_2","unstructured":"Nguyen\u2010PhuocT. LiC. BalabanS. YangY.: Rendernet: A deep convolutional network for differentiable rendering from 3d shapes. InNIPS(2018) pp.7902\u201312."},{"key":"e_1_2_8_28_2","doi-asserted-by":"crossref","unstructured":"NalbachO. RitschelT. SeidelH.\u2010P.: Deep screen space. InProc.i3D(2014).","DOI":"10.1145\/2556700.2556708"},{"key":"e_1_2_8_29_2","unstructured":"NehabD. SanderP. V. LawrenceJ. TatarchukN. IsidoroJ. R.: Accelerating real\u2010time shading with reverse reprojection caching. InGraphics Hardware(2007) pp.61\u201362."},{"key":"e_1_2_8_30_2","unstructured":"PickupD.e. a.: SHREC'14 track: Shape retrieval of non\u2010rigid 3d human models. InProc. EG 3DOR(2014)."},{"key":"e_1_2_8_31_2","volume-title":"Physically Based Rendering: From Theory to Implementation (3rd ed.)","author":"Pharr M.","year":"2016"},{"key":"e_1_2_8_32_2","doi-asserted-by":"crossref","unstructured":"PfisterH. ZwickerM. Van BaarJ. GrossM.: Surfels: Surface elements as rendering primitives. InProc. SIGGRAPH(2000) pp.335\u201342.","DOI":"10.1145\/344779.344936"},{"key":"e_1_2_8_33_2","unstructured":"QiC. R. SuH. MoK. GuibasL. J.: Pointnet: Deep learning on point sets for 3D classification and segmentation.CVPR(2017)."},{"key":"e_1_2_8_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/2766899"},{"key":"e_1_2_8_35_2","doi-asserted-by":"crossref","unstructured":"RitschelT. EngelhardtT. GroschT. SeidelH.\u2010P. KautzJ. DachsbacherC.: Micro\u2010rendering for scalable parallel final gathering.ACM Trans. Graph. (Proc. SIGGRAPH Asia) 28 5 (2009) 132.","DOI":"10.1145\/1618452.1618478"},{"key":"e_1_2_8_36_2","unstructured":"RonnebergerO. FischerP. BroxT.: U\u2010net: Convolutional networks for biomedical image segmentation.CoRR abs\/1505.04597(2015)."},{"key":"e_1_2_8_37_2","doi-asserted-by":"crossref","unstructured":"RitschelT. GroschT. SeidelH.\u2010P.: Approximating dynamic global illumination in image space. InProc. i3D(2009) pp.75\u201382.","DOI":"10.1145\/1507149.1507161"},{"key":"e_1_2_8_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2462009"},{"key":"e_1_2_8_39_2","doi-asserted-by":"crossref","unstructured":"ShadeJ. GortlerS. HeL.\u2010w. SzeliskiR.: Layered depth images. InProc. SIGGRAPH(1998) pp.231\u201342.","DOI":"10.1145\/280814.280882"},{"key":"e_1_2_8_40_2","doi-asserted-by":"crossref","unstructured":"SloanP.\u2010P. GovindarajuN. K. NowrouzezahraiD. SnyderJ.: Image\u2010based proxy accumulation for real\u2010time soft global illumination. InProc. Pacific Graphics(2007) pp.97\u2013105.","DOI":"10.1109\/PG.2007.28"},{"key":"e_1_2_8_41_2","doi-asserted-by":"crossref","unstructured":"ScherzerD. H.NguyenC. RitschelT. SeidelH.\u2010P.: Pre\u2010convolved radiance caching.Comp. Graph. Forum (Proc. EGSR) 31 4 (2012).","DOI":"10.1111\/j.1467-8659.2012.03134.x"},{"key":"e_1_2_8_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/566654.566612"},{"key":"e_1_2_8_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073335"},{"key":"e_1_2_8_44_2","doi-asserted-by":"crossref","unstructured":"ViciniD. AdlerD. Nov\u00e1kJ. RousselleF. BurleyB.: Denoising deep Monte Carlo renderings.Comp. Graph. Forum(2018).","DOI":"10.1111\/cgf.13533"},{"key":"e_1_2_8_45_2","unstructured":"WuJ. ZhangC. XueT. FreemanB. TenenbaumJ.: Learning a probabilistic latent space of object shapes via 3D generative\u2010adversarial modeling. InNIPS(2016) pp.82\u201390."},{"key":"e_1_2_8_46_2","unstructured":"ZhengQ. ZwickerM.: Learning to importance sample in primary sample space.arXiv:1808.07840(2018)."}],"container-title":["Computer Graphics Forum"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1111%2Fcgf.13783","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/cgf.13783","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1111\/cgf.13783","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/cgf.13783","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,8]],"date-time":"2023-09-08T20:19:58Z","timestamp":1694204398000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/cgf.13783"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,7]]},"references-count":45,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2019,7]]}},"alternative-id":["10.1111\/cgf.13783"],"URL":"https:\/\/doi.org\/10.1111\/cgf.13783","archive":["Portico"],"relation":{},"ISSN":["0167-7055","1467-8659"],"issn-type":[{"value":"0167-7055","type":"print"},{"value":"1467-8659","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,7]]},"assertion":[{"value":"2019-07-30","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}