Beyond point-to-point triangulation: dynamic 3D imaging under global illumination

3D imaging under global illumination

As the “eyes of industry”, 3D measurement technology provides comprehensive, accurate, and reliable 3D information for high-end precision manufacturing. Optical measurement methods, such as structured light projection, have been widely adopted in various fields, including industrial inspection, intelligent assembly, advanced manufacturing, and material sorting. These methods, based on triangulation principle, rely on the traditional geometric optics concept of “point-to-point” imaging. Consequently, they are typically limited to scenarios with direct illumination condition.

As manufacturing processes evolve and smart application scenarios multiply, the demands on measurement tasks are shifting from simple static 3D measurements in straightforward environments to high-precision dynamic measurements in complex mixed environments. In industrial inspection and intelligent assembly, for example, measurements often involve various reflective surfaces. Challenges include multi-path indirect illumination caused by inter-reflections on metallic or mirror-like surfaces, and subsurface scattering in mixed surfaces such as jade, ice, skin, and wax. Similarly, advanced manufacturing and material sorting require dynamic 3D measurements through complex transmission media. For instance, real-time monitoring of the forming state in metal additive 3D printing must contend with volumetric scattering indirect illumination, while rapid identification and classification of objects within translucent surfaces in material sorting involves multi-path direct/indirect illumination, as illustrated in Fig. 2. All these challenging illumination conditions are collectively referred to as global illumination.

Under global illumination, light propagation paths disrupt the “point-to-point” single reflection conditions required by traditional imaging model. The direct application of conventional 3D measurement methods may fail due to the mixing and/or loss of depth-encoded information, significantly limiting the applicable scenarios for existing measurement techniques.

Multi-scale parallel single-pixel imaging

To this end, a Fourier dual-slice projection with depth-constrained localization is presented to separate and utilize different illumination and reflection components efficiently, which can significantly decrease the number of projection patterns in each sequence from thousands to fifteen, as shown in Fig. 3(a). Subsequently, multi-scale parallel single-pixel imaging (MS-PSI) is proposed based on the proposed and proven position-invariant theorem, which breaks the local regional assumption and enables dynamic 3D reconstruction, as shown in Fig. 3(b).

Results

Our methodology successfully unveils unseen-before capabilities such as (1) accurate depth measurement under interreflection and subsurface scattering conditions, (2) dynamic measurement of the time-varying high-dynamic-range scene and through thin volumetric scattering media at a rate of 333 frames per second; (3) two-layer 3D imaging of the semitransparent surface and the object hidden behind it, as shown in Fig. 4. The experimental results confirm that the proposed method paves the way for dynamic 3D reconstruction under complex optical field reflection and transmission conditions, benefiting imaging and sensing applications in advanced manufacturing, autonomous driving, and biomedical imaging.

Dynamic 3D imaging results are shown in Fig. 5-Fig. 7.

Outlook

Unlike the traditional structured-light method, the proposed MS-PSI method is no longer limited by the point-to-point triangulation rule and breaks existing PSI methods’ regional light reception assumption. And it provides an efficient and general solution for separating complex illumination information, including subsurface scattering, inter-reflection, multi-layer superpositions, and thin volumetric scattering media, by recording the four-dimensional light transport coefficient. This technology paves the way for achieving dynamic 3D reconstruction under unusual reflection conditions, which has potential in challenging measurement tasks, such as crash test analysis with interreflection, biomechanical testing with subsurface scattering, and icing tests with mixed reflection conditions. In addition, the rapid illumination separation ability of MS-PSI offers new opportunities for 3D imaging under complex optical field transmission conditions, such as 3D imaging under strong light interference, 3D imaging through scattering media, and multi-layer 3D imaging.

###

References

DOI

10.37188/lam.2024.034

Original Source URL

https://www.light-am.com/en/article/doi/10.37188/lam.2024.034

Funding information

This research was supported by the National Natural Science Foundation of China (62205226, 62075143), the National Postdoctoral Program for Innovative Talents of China (BX2021199), the General Financial Grant from the China Postdoctoral Science Foundation (2022M722290), the Key Science and Technology Research and Development Program of Jiangxi Province (20224AAC01011), and the Fundamental Research Funds for Central Universities (2022SCU12010).

About Light: Advanced Manufacturing (LAM)

Light: Advanced Manufacturing (LAM) is a new, highly selective, open-access, and free of charge international sister journal of the Nature Journal Light: Science & Applications. The journal aims to publish innovative research in all modern areas of preferred light-based manufacturing, including fundamental and applied research as well as industrial innovations.