Deep Shape-from-Template: Single-image quasi-isometric deformable registration and reconstruction
Autores
Fuentes Jiménez, David; Pizarro Pérez, Daniel; Casillas Pérez, David; Collins, Toby; Bartoli, AdrienIdentificadores
Enlace permanente (URI): http://hdl.handle.net/10017/53974DOI: 10.1016/j.imavis.2022.104531
ISSN: 0262-8856
Editor
Elsevier
Fecha de publicación
2022-11-01Patrocinadores
Agencia Estatal de Investigación
Ministerio de Educación
Cita bibliográfica
Fuentes Jiménez, D., Pizarro Pérez, D., Casillas Pérez, D., Collins, T. & Bartoli, A. 2022, "Deep Shape-from-Template: Single-image quasi-isometric deformable registration and reconstruction", Image and Vision Computing, vol. 127, art. no. 104531.
Palabras clave
Monocular
3D Model
Registration
Reconstruction
Wide-baseline
Dense
Deformable
Shape-from-Template
Proyectos
info:eu-repo/grantAgreement/AEI/Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020/PID2020-115995RB-I00/ES/TECNICAS DE APRENDIZAJE PARA RESOLVER LA RECONSTRUCCION Y REGISTRO DEFORMABLES APLICADOS A IMAGENES DE LAPAROSCOPIA/
CAS21/00182
Tipo de documento
info:eu-repo/semantics/article
Versión
info:eu-repo/semantics/publishedVersion
Versión del editor
https://doi.org/10.1016/j.imavis.2022.104531Derechos
Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)
Derechos de acceso
info:eu-repo/semantics/openAccess
Resumen
Shape-from-Template (SfT) solves 3D vision from a single image and a deformable 3D object model, called a template. Concretely, SfT computes registration (the correspondence between the template and the image) and reconstruction (the depth in camera frame). It constrains the object deformation to quasi-isometry. Real-time and automatic SfT represents an open problem for complex objects and imaging conditions. We present four contributions to address core unmet challenges to realise SfT with a Deep Neural Network (DNN). First, we propose a novel DNN called DeepSfT, which encodes the template in its weights and hence copes with highly complex templates. Second, we propose a semi-supervised training procedure to exploit real data. This is a practical solution to overcome the render gap that occurs when training only with simulated data. Third, we propose a geometry adaptation module to deal with different cameras at training and inference. Fourth, we combine statistical learning with physics-based reasoning. DeepSfT runs automatically and in real-time and we show with numerous experiments and an ablation study that it consistently achieves a lower 3D error than previous work. It outperforms in generalisation and achieves great performance in terms of reconstruction and registration error with wide-baseline, occlusions, illumination changes, weak texture and blur.
Ficheros en el ítem
Ficheros | Tamaño | Formato |
|
---|---|---|---|
Deep_Fuentes_Image_Vis_Comput_ ... | 1.374Mb |
|
Ficheros | Tamaño | Formato |
|
---|---|---|---|
Deep_Fuentes_Image_Vis_Comput_ ... | 1.374Mb |
|
Colecciones
- ELECTRON - Artículos [243]