Trimačio objekto erdvinio modelio sukūrimas iš dvimačių vaizdų naudojant mašininį mokymąsi /

This work explores approaches to the structure-from-motion problem [1] by employing well-established methods to reconstruct 3D objects from various selected yet limited 2D image sets. We utilize synthetically generated views of textured meshes rendered with PyTorch3D [2] with variable image quality, as well as photographs of objects from the natural world. Reconstructing 3D meshes from any given image set requires knowledge of the camera positions from which the shots were taken. Neither of the selected approaches to create datasets (synthetic or natural) did not involve a way of logging camera positions. To locate those positions, COLMAP package [3] was used to select intrinsic features of images, mark them as points within the pictures, cross-match them, and find a global solution. However, achieving a successful reconstruction of any given scene is not straightforward. Therefore, this work investigates the minimal quantity and quality of images required in various controlled synthetically generated datasets to ensure a robust reconstruction. Using the extracted camera view positions, we employ the Neural Radiance Field method (NeRF) to synthesize novel views and generate a point cloud much more detailed than that obtained by COLMAP. The model used in this study is same to the one presented in the original paper [4] introducing the concept of NeRFs. There were also other models discussed and used in this work. Such models utilizes a volumetric method to project rays out of the picture, interacting with pixel data, to reconstruct object's density in 3D space. Synthetically generated models with recovered camera positions were employed to recreate 3D meshes and explore the results achievable with limited quality and quantity image sets. Natural world images were used to create acceptable quality meshes and 3D scene reconstructions. One of the selected objects of interest is the asteroid Ryugu, which was photographed by the Hayabusa2 asteroid sample-return mission [5]. Selected photographs were utilized to reconstruct a 3D mesh of Ryugu. This reconstruction incorporated both traditional photogrammetry techniques and chosen machine learning methods. The results were then compared with prior reconstructions [5]. The findings were discussed, providing explanations for the varying degrees of success achieved in these reconstructions. Literature: [1] Hartley, R. I., Zisserman, A. (2004). A Brief Overview of Structure from Motion. International Journal of Computer Vision. [2] Ravi, N. et al. (2020). Accelerating 3D Deep Learning with PyTorch3D. arXiv:2007.08501. [3] Schönberger, J. L., Frahm, J.-M. (2016). Structure-from-Motion Revisited. In Conference on Computer Vision and Pattern Recognition (CVPR). [4] Mildenhall, B. et al. (2020). NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In ECCV. [5] Japan Aerospace Exploration Agency, Hayabusa2 project, 2020. Accessed: 2021-03-05. [6] S. Watanabe, M. Hirabayashi, N. Hirata, N. Hirata, et al., Hayabusa2 arrives at the carbona- ceous asteroid 162173 ryugu-a spinning top-shaped rubble pile, Science, 2019, 364, 268–272.

Grantor dc:publisher

Institutional Repository of Vilnius University

Year dc:date

2024

Author dc:creator

• Jedik, Andrius,

Statement dc:rights

• info:eu-repo/semantics/openAccess

Language dc:language

lit

Repository record dc:identifier

https://repository.vu.lt/VU:ELABAETD210579503&prefLang=en_US

OAI identifier oai:identifier

oai:vu.lt:elaba:210579503