Geospatial and photogrammetry: When the mapping world and the gaming world collide
The gaming and the mapping world might look like two very distinct industries but when we give a closer look, we can discern some similarities. Both industries are now more than ever benefiting from each other’s unique features. In one hand, interactive 3D GIS are now using improved graphics, atmospheric and particles effects to visualise real-world data as well as interactive navigation capabilities similar to a video game. In the other hand, maps are the core component of many video games and can be based on either fantasy or real-world information derived from geospatial data. A growing number of games developers have also started to use 3D assets generated using 3D photogrammetry techniques.
Let us have a quick look at what’s happening on these different topics.
Geospatial data in games
Maps and by extension the spatial aspect of it are a core component of virtual worlds used in video games. These maps are used as a base for the creation of fantasy worlds or designed to simulate real-world environments. Geographic data are used as input information during games development phases to givedevelopers real-world information to build their maps on and emulate real environments to improve player immersion and familiarity.
For example, this is the case for Call of Duty based in Iraq and Grand Theft Auto in California. Beside simulation for training and military applications, the most popular game using real-world geographic data has been Microsoft Flight simulator with a huge selection of scenery packs and add-ons based on aerial orthophotography. Another game based on aerial imagery is Real World Racing
What is Photogrammmetry?
The “Photogrammetry” word has been popping up more and more in the headlines recently but the technique has been around since the mid-nineteenth century. An increase in processing power of computers, more powerful digital camera, the democratisation of photogrammetry processing software and drones can explains this new popularity.
But what is photogrammetry? Photogrammetry is the art and science of extracting accurate geometric properties of objects from overlapping photographs. The theory behind the photogrammetric process is as old as the invention of photography itself however the widespread development of computers has drastically improved the photogrammetric workflow through automation.
Triangulation is the fundamental principle behind the photogrammetric process where overlapping photographs taken at different angles intersect. By mathematically modelling intersecting lines associated with features visible on the photographs, accurate 3D points located in space are extracted. Through a bundle-adjustment process, which includes processing the camera calibration and orientation as well as triangulation of the photographs, the final 3D point coordinates are extracted. To generate the final 3D objects, the 3D-point cloud goes through a meshing and texturing process.
Applications of photogrammetry includes survey and mapping products such as orthophotography, digital elevation models and more recently 3D city models.
Example of 3D city models generated from aerial photogrammetry
It is also used in fields as varied as archaeology, architecture, engineering, quality control, movies and as this article shows in video game developments.
Photogrammetry in games
3D models produced with photogrammetry techniques are usually human size objects, very realistic and contain details which would be very hard and time consuming for a 3D artist to recreate. As the guys from skull Theater said “Photogrammetry … lets you fall back on classical skills, working in a medium where realism is a natural byproduct rather than a conscious effort.” Check out there FAQ for more information
The downside of producing this kind of models is the large number of polygons which constitute captured 3D objects.
A big challenge during game development process is the geometric optimization of the multiple assets which constitute the game and it is why only a few games are currently using these types of assets.
Fortunately processing power and computer graphics are become more powerful every day which will make the ingestion of this complex data more mainstream.
While still relatively underused in the gaming industry, some games have made the headlines with their innovative used of 3D photogrammetry to create assets from rocks, buildings, character faces. One of the most popular is the Vanishing of Ethan Carter from the Astronauts.
Facial and body scanning is also a big deal on the sports game market. Using structured light similar to the ASUS Xtion or Microsoft Kinect technology (Primesense now owned by Apple provides the core chip for both), Sony Computer Entertainment and 2K ports captured more than 1400 players in the American Major League Baseball in 2006. 3D scanning uses direct measurement to generate a 3D model as opposed to photogrammetry which needs to be post-processed. Check this interesting article about 3D scanning vs photogrammetry for 3D printed figurines.
Smaller but nonetheless innovative indie game companies such as Skull Theater have also embraced this new technology to develop Rustclad.
Get Even includes 3D environments generated with photogrammetry
The company behind ReRoll is also planning to use photogrammetry to reproduce the planet at scale using drones and aerial photogrammetry techniques. A very ambitious project …. As a mapping company, we know very the challenges in such an endeavour.
The guys at Inxile Entertainment have successfully funded through their kickstarter campaign the development of the fourth installment of The Bards Tale game. Built in Unreal Engine 4, they will make use of 3D photogrammetry to capture locations and bring to life the 5000 year old village of Skara Brae on the West coast of Mainland, the largest island in Orkney archipelago of Scotland.
While there are little doubts that we will see more and more of 3D assets created with 3D photogrammetry, the amount of data will still be limited by our computer and graphic power. The good news is improvements don’t stop.
The following video is an experiment where an aero3Dpro 3D city model generated with 3D photogrammetry techniques has been successfully integrated into Unreal Engine 4.9 along with height-map and aerial imagery as background data. We are hoping to see much more of it in the near future.