He had a hand in the creation of the fantastical rocky formations of the Red Planet and oversaw the setting for the ancient Direwolves. Calgary-based location pro Mike Johansen/LMGI shares the ups and downs of doing the job in the digital age…
by Mike Johansen
@open_myke
Photos courtesy of Mike Johansen/LMGI, except where noted.
Augmented Elbow Falls, as seen in The Bourne Legacy
There’s been a peak of interest this year in new technology to help us do our jobs as scouts and LMs. Especially as those jobs became more out of focus for some of us as we emerge from the COVID-19 lockdown, or as I call it, “Lock-up!”
I had worked with the VFX team on Lost in Space S2 to create the Red Planet in the Badlands of Alberta, Canada. It was that and really several projects between 2018 and 2019 that allowed me to understand how the technologies of photogrammetry* and LiDAR* impact the department in regards to production schedules and scope of work.
The pace of technological advancement is staggering at this moment in near-history. By the time you read this, some of it may already be out of date! But these few projects offer such interesting & unique applications, I’d be remiss not to mention them here.
The Water Is Cold…
Heartland LM Brian Dunne reminded me that I didn’t need to look far for a direct comparison to just how fast digital filmmaking has come. Scenic and oft-filmed Elbow Falls is actually close to where we both live. He had worked on The Bourne Legacy, which featured Jeremy Renner as the Bourne character, emerging from a very cold river pool below the falls after doing a little Wim Hof Method** (see sidebar following this post) in the first two minutes of the film.
The waterfall has the appearance of being much taller than it actually is due to another frozen tier added in the background, along with some mountains and other pretty things. This was Brian’s first experience with LiDAR. The LiDAR head was suspended from a helicopter by cable. There may have been some early application of photogrammetry to help map the environment as well, but that’s unclear as per his recollection.
Fast-forward to spring 2019 when we shot the same location for Jumanji: The Next Level. Believe me, for their scene beneath the falls, the Jumanji cast did not have to take the plunge into those frigid waters as Mr. Renner bravely did nine years earlier. The VFX crew took care of that!
**Did you know…
Wim Hoff Method
A discipline of rigorous breathing exercises, cold therapy and training of one’s mindset to improve health and strength.
Recce
Reconnaissance, military scouting; a production term used in the UK, Europe, India, Australia, New Zealand, South Africa, Canada and Malaysia which refers to a pre-filming visit to a location to determine its suitability for shooting. In the U.S., the term “tech scout” is commonly used.
Direwolves
In Game of Thrones, they are larger than a pony and can rip a man’s arm from its socket. These fiercely intelligent creatures play a more prominent and mystical role in the books because the VFX needed to create them limited their storyline in the TV series.
The Mandalorian
A Disney+ TV series set in the Stars Wars universe that uses virtual sets and a 360-degree video wall to create the series environments.
Goat Creek, Alberta, Canada
Beware: Going Guerrilla
On a practical note, although we were too busy with the main unit crew above tree line in the alpine to be with the VFX unit at Elbow Falls that shoot day, I did take the team to the location during prep where we walked the area to determine camera positions and drone flight paths. We also had the mountain safety team along who would have to get people down a steep slope to the river bottom below the waterfall on the day.
Because of my experience on Lost in Space the summer before, I noticed pretty quickly when the cameras came out of the bags and it escalated from “a few reference shots” to something that more closely resembled an exercise in data collection/photogrammetry.
Photogrammetry is a reality capture workflow which involves taking hundreds or even thousands of photographs using handheld DSLR cameras* to create a point cloud*, which turns into a mesh*, that is then refined and textured using specialized software. It’s a commercial activity.
We were in a provincial park for which our permit was stamped with a different date for the filming. We were supposed to be there for a “look-around.”
The VFX crew would only have a day to actually capture the area, so they wanted to take advantage of the fine weather on that scout day. Kudos to Mark Breakspear, leading the VFX team that day, who pushed for usable assets* in a discreet way, but shut it down when the public started showing up.
A complaint would have jeopardized our permit with Alberta Parks. Because photogrammetry is done using pretty much any good quality DSLR, you may see a small team of three or four VFX people joining a recce** without too much other gear along. If you plan to have a location liaison present for the survey, it could possibly get weird and even jeopardize the deal, something VFX doesn’t necessarily think about.
I understand the temptation to go guerrilla-mode, but I feel it’s important for a scheduled, permitted, paid-for and insured dedicated time period for VFX to do their thing.
It all worked out well on the shoot day. But when you watch the scene, it isn’t clear looking at the final edit on the big screen how much of the physical environment was used and how much was constructed and enhanced using CG. The waterfall looked very unaltered from its current natural state. The nature of discovery presented angles that couldn’t be filmed easily or safely. Also, by the time you’ve removed the tourist path, added CG people and a CG horse, changed the water to CG and added them jumping in, there is not much left that was original plate.
We would normally try to shoot plates, but this section of the story was still being flushed out, so we needed to shoot more to protect the “can you do this angle…” requests. VFX is relied on to “do anything” and they didn’t want to come up short! Some would call it “hosing it down.”
Largest proposed build site, photogrammetry via drone, Stoney Nakoda First Nation, Canada
No Time to Die and the Big Build
I began scouting for UM/LM Mark Voyce on No Time to Die in early 2018. There was a set described in the original version of the script which was beyond anything I had seen before—and it was my job to find a build site for it. It was comprised of five separate large builds within several square kilometres.
There was to be a practical train moving on track surrounded by industrial buildings and machinery, all enclosed by a massive security fence made using 300-350 telephone poles. A large area filled with set dressing would stretch beyond the fence. A set this large was going to need six to eight months to build so we were reaching hard for final answers on a few good options in locations near Calgary.
The director at the time was still shooting another film in Europe and couldn’t attend the early scouts. The designer, Mark Tildesley, and his art department, anxious to begin the build, wanted to create a presentation based on actual location options on which to mount 3D drawings. This would help move along the approval process.
It was decided to spend the money for 3D scanning* of the areas I had identified. Because we were looking at this so far ahead of time and MGM’s strict legal liability rules for contracting service providers, especially drone operators, it was a clunky process. Luckily, we found local contractors who normally specialize in civil engineering type work and managed to squeak onto the locations as part of the scouting process which didn’t require production insurance.
My first day was spent supervising the contractors moving a set of sticks with a LiDAR head around the two potential gravel pit locations within the mountains. The greater the detail needed determines the amount of setup points within the area. The more points of capture, the slower the scan at each point. LiDAR worked well for the smaller sites with more “noise” such as rocks, trees, and hills.
Largest proposed build site, photogrammetry via drone, Stoney Nakoda First Nation, Canada
Smaller Potential Build Site: Terrestrial LiDAR
The next day, I met the drone company at the larger location option which was much more open and bare. Photogrammetry by drone for this site was advised vs. LiDAR. They flew over the private land in a pre-programmed grid pattern which took several hours, batteries, and numerous memory cards. I sat with the contractors as the robot did all the work flying the grid to snap HD photos at programmed, high-frequency intervals. It would even fly itself home for battery and memory card replacement. This took several hours longer than I had expected. Pack a lunch!
Once they received the large data files from the sites that were LiDAR scanned, and the thousands of photos shot by the drone, the art department could anchor their set concept drawings to each of the physical location options to show not only the dimensions of the area but topographic detail down to every undulation, rock, existing vegetation, and body of water.
When a set-build site is the purpose of getting photogrammetry and/or LiDAR work done, it is helpful to know how to send a contractor what area you want scanned by sourcing a .kml file*. This shows the precise size of the general area to be scanned.
To get a .kml, you will need to use Google Earth. There’s YouTube videos on how to do that. It’s very quick and easy and it will help speed up the quoting process substantially.
After construction started building the huge industrial set, there were changes to the personnel above the line. The locations I had scouted so hard for all went away—as did the movie—to Europe! Dozens of telephone poles had to be removed and the site reclaimed. What a crazy industry we work in!
*VFX Tech Terms
Photogrammetry is the art and science of extracting 3D information from photographs using a DSLR camera. The process involves taking overlapping photographs of an object, structure or space, and converting them into 2D or 3D digital models.
LiDAR
Standing for “Light Detection and Ranging” or “Laser Imaging, Detection, and Ranging,” a popular remote sensing method used for measuring the exact distance of an object on the earth’s surface. A LiDAR instrument principally consists of a laser, a scanner, and a specialized GPS receiver
LiDAR is sometimes called 3D laser scanning, a special combination of a 3D scanning and laser scanning.
3D Scanning
Using a laser or white light to capture a physical object’s exact size and shape to create a digital three-dimensional model to quickly generate highly accurate point clouds.
Point Cloud is a set of data points in space with coordinates representing a 3D shape or object. Generally produced by 3D or photogrammetry software, which measures many points on the external surfaces of objects around them.
Mesh
In computer graphics, a polygon mesh is the collection of vertices (the point where two lines meet to form an angle), edges, and faces that defines the shape and contour of every 3D character & object, whether it be used for 3D-animated film, advertising, or video games.
.KML File (Keyhole Markup Language) is a file format created for storing geographic data and associated content with Google Earth.
DSLR Camera
A digital single-lens reflex camera is a digital camera that combines the optics and the mechanisms of a single-lens reflex camera with a digital imaging sensor.
VR (Virtual Reality)
Computer-generated simulation in which a person can interact within an artificial three-dimensional environment using electronic devices, such as special goggles with a screen or gloves fitted with sensors.
Reality Capture
Photogrammetry software which creates 3D models out of unordered photographs or laser scans without seams. The most common fields of its current use are full-body scanning, gaming, surveying, mapping, visual effects and virtual reality in general.
360-Degree Camera uses ‘flat’ sensors, but back-to-back fisheye lenses, each capturing a 180-degree angle of view. The camera merges these two hemispherical images to produce a 360-degree spherical view of everything around you.
Small, lightweight and affordable, they create video and stills used as virtual reality content in suitable browsers or apps, but can also be edited into regular ‘flat’ videos where you can point the camera where you like or follow any subject—AFTER you’ve captured the video.
Asset is a 3D model created by digitally capturing the geometry, surface texture and color of a real physical object through laser scanning or photogrammetry that can then be manipulated in a virtual environment.
The Direwolves** Are in Alberta!
If you’ve never shot with wolves before, it’s not as much fun as it sounds! Several years ago, while working on a dog food commercial with wolves, I asked the animal trainer to reposition his trailer so the crew had room to get by on a tight mountain ridge. He was so furious, he dented the door of my rental truck with his fist. Didn’t I know that the wolves were the stars of the shoot? The crew peed in the bush that day.
I learned that it’s best to film certain animals away from the shooting crew whenever possible!
So, I was only too happy to head to a private “Wolf Ranch” for a photogrammetry-only shoot of the superstar Direwolves for Game of Thrones S8.
This was something that had been done in Alberta since the start of the series but I hadn’t been a part of until then. The shooting crew included myself, a local genny op, crafty and a group of three or four young techs from Clear Angle Studios out of Georgia. They set up their 180+ DSLR camera array in a near 360-degrees grid inside a tent in the interior of a quonset somewhere in rural Alberta.
Quite a setup technically, but it was done in a fairly small, simple space. I had never seen that before, until I realized that I had been aware of the approach from the video game industry, specifically sports games which use the actual likeness, movement & mannerisms of professional athletes in the game.
I really didn’t do much on that week-long project compared to most jobs. It wasn’t the new shooting technology that was the reason for that. As an LM, I was essential to the process, but just utilized more as a location liaison.
Does this make me worry about film crew job security in an age of high-tech production techniques? Well, if the tech didn’t exist, the wolves and trainers would have had to travel to the shooting crew in Europe. The crew of GOT in Europe didn’t have to worry about shooting a live animal, as well as all the other crazy logistics on location.
Not every job will stretch our skillset to the max, and I welcomed the respite. I got to see their setup, grab a couple pics of the GOT Direwolves, hand out some cheques and hang out organizing my overflowing scouting files.
Lost in Space, Alberta Badlands, Canada
Landing on the Red Planet Before Perseverance
Area quoted (blue) vs. area to be scanned
I realized early on that I was scouting for something really special with Lost in Space. I was tasked with finding epic landscapes devoid of any vegetation to play for the Red Planet, which would appear in four episodes in Season 2.
What a fun assignment! I’ve scouted and shot in the Alberta Badlands in the past but mostly for mountain biking, car commercials or as a double for New Mexico or Utah in a Western. I’d always hoped for a “space” show because the unusual features always looked otherworldly to me!
After viewing my scouting images, the producer and director were excited to see more. I couldn’t have asked for a better team to work with: seasoned director/producer Alex Graves, “been there, done that” designer Frank Walsh, young gun VFX savant Jabbar Raisani, and the progressive even-keeled producer/PM team of Brad Van Arragon and Dennis Penney/LMGI.
Jabbar’s team would be taking care of photogrammetry in-house. They started shooting the different location options early using their DSLR cameras to create digital environments for not only their own discussions but to aid in pre-production planning for every department. My stills and video using the drone helped to create boards and facilitate discussions toward final decisions on the hero locations.
I asked Jabbar if there was anything we, as scouts, could do to facilitate the search for these types of proposed environments. “Using drone stills and video would be a minimum, but going further and using photogrammetry/LiDAR would be even better so we can virtually scout in 3D programs and VR (Virtual Reality)* if necessary,” he advised.
Lessons Learned
But there was more. I was caught off-guard when I was told that we needed to have all of the locations LiDAR scanned sometime after all of the sets and dressing were in place and before striking it all but not during actual production. A few people on site were fine but they couldn’t have 200-500 cast, crew & extras on or near set. It would mess up the scan.
What I was facing at the time was an 80-person unit shoot which had grown to 270 crew plus extras, helicopter load in/out for set dressing, multiple access road improvements and constant maintenance using heavy machinery. I had to build four access bridges (two permanent and two temporary), circumvent seasonal tourism to ensure safety & privacy, and protect fossil beds and rare slow-growing lichen, to mention a few of the other unseen elements that were my department’s responsibility.
There was a point where I was the middleman for way too many questions going back-and-forth between the art department, VFX and the LiDAR crew that I had quote the job.
I was being asked for .kml files of the sites to be 3D scanned before I knew exactly where we wanted to shoot what. The areas I was asked to get quoted grew by 300 percent to 400 percent in size, with my budget expanding proportionally.
Added to the mayhem because the scope of the shoot grew so large so quickly, hotel rooms nearby were in short supply, which added another layer of complexity to schedule the LiDAR scans which the ADs also handed off to me.
In retrospect, knowing what I now know, if I was to do another show like this, I would make sure I was staffed appropriately and have dedicated assistants just for VFX.
There was a point where the weather had turned unseasonably foul and the “completely dry planet” I had scouted and everyone was excited about was being drenched by rain and even snow! The crew started splintering to try and work around the weather and I was stretched between four locations. One of which, a UNESCO World Heritage Site, was a 3.5-hour drive away!
At that point, the VFX lead for the LiDAR scanning process for all sets mentioned they would like to just stop at various interesting features wherever they saw them off the side of the road to scan them. You know, hop a fence or two—no big deal.
Creating virtual scouts for Production
As I watched the LiDAR crew scan the locations to the nearest mm, I wondered about the future of film and shooting on location. Is it possible that the studios direct the VFX or art department team to carefully archive and tag the photogrammetry and LiDAR data files for easy recovery from a library? I mean, the detail is insane.
I reached out to Brent Walmsley, whom I sourced for the VFX department to do the LiDAR scans on Lost in Space. Brent, now with GeoVerra in Ontario, refreshed my memory on the technical details. “We collected 1 million 3D points/second with professional-grade, highest quality 3D scanners. Now, it’s possible to measure up to 1 km with that equipment. Lower end scanners are around 30,000 pts/sec with 50 m range and offer less detail accuracy. With final, processed and merged together scan files, we’re looking at a 0.1mm resolution. Regardless of resolution though, we generally process these files into very high poly-count mesh geometries that can be refined by VFX teams.”
Mike Johansen on Lake Louise, Alberta
Where We Go from Here…
So, is the future of our profession in jeopardy? When studios spend millions to travel hundreds of people to remote areas only to be faced with crazy weather outside all historical norms which threatens the tight schedule, will it make more sense to keep it indoors on volume stages as with a show such as The Mandalorian **?
The younger generations that may prefer to use tech over risking being on a physical location still have a decade or two before they start to steer the ship. Every director and DP I know wants to shoot their own establishing shot or sunset on the location that fits the emotional qualities of the film.
I see The Mandalorian as an interesting case study, but as actual VFX supervisors like Jabbar Raisani would tell us, “The reality is that solutions like The Mandalorian are incredibly expensive and technically difficult to achieve. Mass adoption of these techniques is unlikely in the near future.“
Yes, COVID is creating a lot of buzz around the subject and people will undoubtedly use the tech to help solve some of our current travel restrictions. But most productions can’t afford this type of post-production technology, at least not in the near term.
Virtual production is similar to green screen. Currently, every production has the technology to film everything on a stage in front of green screen. But when we can, we typically prefer to film real people in real locations.
I posed the question of whether or not LMs will be looked at to shoot photogrammetry/LiDAR scans for production to Brent, an industry outsider/LiDAR contractor.
“With the use of reality capture technology, there is a track record of blending automation with creativity; powerful results with growth and innovation,” he says. “Within the TV/film industry, I think those ideas could be first manifested by location managers who become early adopters of the technology.
“Location managers have a unique opportunity to revolutionize the work they do, and create new opportunities for themselves, like building their own resource of digital location assets generated from 3D scanning LiDAR data that they have embraced.”
Based on those thoughts, location pros may want to think about a “Shutterstock for Photogrammetry”—there’s already 360 stock footage for sale online. Jabbar adds, “If a company went out and started scanning environments for the express purpose of licensing them to productions, we may see a shift to more of a stock photography scenario, but thus far, there hasn’t been a demand for such assets.”
Talking Tech and Scouting On
At the end of the day, maybe there’s the potential for new revenue-generating opportunities using LiDAR and photogrammetry in conjunction with location scouting, but I’m personally not interested in getting into the business of selling digital locations as a type of stock footage asset. The hardware & software costs are a legitimate barrier to entry.
I looked into the newest, lightest LiDAR scanners and Leica has the BLK360 for a cool $25K. Then you’re looking at the software to create a digital mesh. The BLK360 which Jabbar’s team carries in the kit actually spits out a full-colour scan to a Wi-Fi-synched iPad as the scan is being done. So, is photogrammetry on its way out?
“Photogrammetry is relatively simple, and doesn’t require expensive equipment, so I think it’s here to stay. A 5D gives you more resolution and better color depth—for now,” says Jabbar.
I believe that I couldn’t have sold the locations for the Red Planet, as well without using the drone to convey the size and scope of the epic landscapes that Alex Graves was insistent on having. But would I have been approved for the extra time & budget to go the extra mile for Reality Capture* efforts for that budget level at that early stage? Probably not.
Will I go out and invest time and money into specific equipment to 3D scan areas? Not yet. There are vendors who’ve already made this their business either specifically for film or not. Yes, getting the quotes, contracts and NDAs all done is a pain especially in early pre-pro when there’s no POs yet, but we’ve got enough to do already and everyone wants options yesterday.
I don’t think there’s enough patience to wait for reality capture assets at that stage unless we’re at blockbuster budget levels. I think my addition of the 360-Degree Camera* to my kit this year is a logical and affordable step forward. Creating INT & EXT virtual walkthroughs like Google Streetview combined with drone and terrestrial stills and video will satisfy early scouting requirements.
I was thinking back last night about how “check the gate” actually required a physical task and how I used to have to stamp “call pushed,” write in “1/2 hour,” and hand-deliver paper copies of the call sheet to the actors at their hotel rooms every night when I started in the AD department.
Unless you constantly work on sci-fi or big action films, the latest & greatest tech is going to keep taking leaps without you. I spoke to a friend who had some large projects restricted from coming to Alberta to film our amazingly picturesque highways due to COVID recently. So, he drove the roads with a 360-degree camera on the roof of a vehicle. Those massive digital files will be used in some of the biggest film & commercial projects in 2021.
In some ways, for the sake of nostalgia, I hope that those shots don’t work and it’s decided that using a camera car is best after all. But, because I think film technology is inevitably advancing (and cool), I hope I get a call to drive around with a camera on my roof soon!
So until global economic shutdowns and mass quarantines start to become a cyclical thing and Epic Games, ILM and all the other studios start to build volume stages in every city, let’s keep calm and scout on!
www.filmlocationscout.ca