Perched atop a hill in the Kathmandu Valley, the ancient Swayambhu Temple, a jewel of Buddhist heritage, proudly reflects Nepal’s rich cultural tapestry. Recognized as one of the ten UNESCO World Heritage sites in Nepal, its history spans more than 1,500 years, marking it as one of the country’s most venerable religious shrines. In a groundbreaking initiative this past October, a team from UNC made up of Lauren Leve, an associate professor of religious studies, and Jim Mahaney, a research scientist in the department of computer science, traveled to Swayambhu to start the long process of capturing the data needed to create an accurate 3D model of the entire temple complex.
Blending academia and tech
The project itself leverages the distinct skill sets of Leve and Mahaney. Leve received a doctorate in anthropology from Princeton University and now specializes her research in religion and culture in South and Southeast Asian Buddhism. This gives her keen insight into the religious importance of various aspects of the temple, which was essential not only for planning what to capture, but how to do so in a manner which respects local Nepalese customs. With regular trips to Nepal over the past 32 years, Leve maintains deep ties to the community—without which access to such a holy site would not have been possible. Mahaney brings more than 25 years of engineering experience to the project, including extensive work with the Deltasphere 3000 Scene Digitizer, a 3D scanner developed by UNC Computer Science researchers Anselmo Lastra and Lars Nyland in the early 2000s. Drawing from extensive experience with field campaigns, including work with Duke University and NASA, Mahaney came prepared to handle the technical aspects of this project. Funding was provided by a Lenovo Instructional Innovation Grant, of which Leve and Mahaney are co-principal investigators, awarded in the spring of 2023.
Precision laser scanning, ensuring for accuracy
In order to create a photorealistic model of the temple complex, three distinct types of data were collected. First, highly accurate measurements were taken using a 3D laser scanner. Today’s laser scanners, just like the original Deltasphere, all work on the same basic principle: a beam of laser light is emitted from the device, and it strikes a point on an object and is then reflected back into the device. A simple time calculation is used to determine the exact distance between the device and the point struck by the laser. In order to construct an entire scene, the laser is pulsed thousands of times per second while being reflected off of a rotating mirror. This sends the laser pulses in a 300-degree arc from the device. The device itself then slowly rotates 360 degrees, so that point data is captured from all sides of the device. For this project, the team used a Faro S150 laser scanner, which has a range of 150 meters and accuracy to within one millimeter. Even with the long range provided by this device, hundreds of scans will be needed to map the entire temple, as the site is very complex, and every structure will need to be scanned from multiple angles to capture all the necessary data.
Photorealistic imagery through DSLR cameras
The second type of data consists of thousands of images taken using digital single-lens reflex (DSLR) cameras, which are standard modern cameras used by amateurs and professionals alike. When combined using a technique called photogrammetry, these images create highly detailed photorealistic 3D models. The key is that the images need to overlap so that the software can find identical points in multiple images and then triangulate the position of each point in 3D space. This technique does not provide the same accuracy as laser scanning, but it does provide much better imagery, so combining the two techniques makes a better model than either technique can provide alone. With standard DSLR cameras used for capturing photogrammetry data, some locations that could not be reached by the tripod-mounted scanner could still be photographed, which lessens the potential for gaps in the data.
A drone’s eye view
The final type of data was aerial images captured by a drone. Since 3D scanners must be securely mounted to a fixed tripod and the DSLR cameras were tethered to students on the ground, the drone data will be used to fill in imagery of rooftops and the central spire on the temple. The drone was programmed to fly a grid over the entire temple complex, taking high resolution images from its onboard camera. These images will then be combined using the same photogrammetry techniques used with the images captured from the ground-based DSLR cameras.
Navigating Challenges: crowds, monkeys, and lots of data
One of the main challenges of capturing Swayambhu was working around crowds of people and animals. In typical historical preservation scenarios, the site would be closed to allow for scanning and photogrammetry, but as an active temple, Swayambhu could not be closed. Both worshippers and tourists visit nearly 24 hours a day, along with a local population of more than 400 rhesus macaque monkeys, which have free range over the entire site. While the humans could be reasoned with, the monkeys refused to take direction from the team, often flaunting their independence by climbing about in the background of the scans. The unfortunate result of this interference will be hours of additional post-processing to remove monkeys from the 3D model.
Because of the immense scale of this task, Leve and Mahaney partnered with a local heritage preservation group, Baakhan Nyane waa (which translates from the Newar or Nepal Bhasa language as “Come, listen to stories”), and the Kathmandu Engineering College. Raj Maharjan from Baakhan Nyane waa partnered with Mahaney to teach a seven-day course on laser scanning and photogrammetry to graduate students from the college, which included many hours of hands-on fieldwork at Swayambhu. During the course, the team
completed 92 scans and took thousands of photos from the ground and hundreds from the sky. The work of combining these into a 3D model will take many months, and UNC Computer Science students will be given the opportunity to work with this dataset to explore various advanced techniques in virtual reality, such as redirected walking. Students in Nepal will also continue to work with the data, making this an international collaboration which not only works to create an outstanding model of the site, but takes into account the subtle nuances of how the site is viewed by the Nepalese.
While the amount of data collected on this first trip is vast, additional visits to Nepal are necessary to capture the entire temple. Plans are already being finalized for a trip in May 2024 to begin expanding the model to include more of the structures located adjacent to the main temple. The team plans to build on the recent successes and to continue to train the local team members, so that some aspects of the research can continue even when Leve and Mahaney are back at UNC.
By Jim Mahaney, Department of Computer Science