Animatronics, Avatars, Afghani And The Marines: Immersive Environment Training

A young Marine on foot patrol stands in the middle of a village road.   Mud walled single story structures face the street, their doors and windows shuttered against the dust.   He holds his M16 across his body just below his chest. 

Around the corner on the upper left, a woman walks toward him, then passes through a door into the second building mid way down the street.   On the other side of the street two turbaned men stand together talking.   One holds a camera, and begins taking pictures of the Marine.    A few other villagers cross the street and pass by in front of the Marine to his right.    

The Marine appears alert, but unsuspicious.  Two or three minutes pass. 

One of the two turbaned men standing nearby abruptly turns and walks away from the Marine.  The man with the camera disappears inside the door of the schoolhouse building.    Moments later gunshot shatters the dusty silence.     The Marine is dead.                                  

The Marine, Afghans, guns and filmed scenes are real.   However,  the bullets are simulated, the Afghans are paid actors, and the village is just a few miles from the Pacific Ocean.   The dead Marine is evacuated from the scenario, but rejoins his squad upon completion and the entire squad is a bit wiser for his experience.   

We are in the debriefing room watching video footage shot during an earlier Infantry Immersion training session.    

The debriefing room is part of the Immersion Training Center at Camp Pendleton, a multi faceted compound initially built in 2007 and expanded in 2010, which serves as a prototype for modern “immersive environment” training.    Here Marines come for 10- hour training days which include two or three scenarios like the one above. 

The facility utilizes a mix of animatronics, computer generated avatars, scent generators and  multi-speaker systems injecting recorded sounds,  along with contracted Afghan actors, to create a virtual reality combat environment  for the Marines. 

Buildings, made of stucco and sand are decorated and styled to be accurate replications of common structures found in a typical Afghan town.    There is a town center, complete with government offices and a jail; as well as a school,  and marketplace.   

Some of the buildings are clustered in an outdoor setting, lit by daylight.    Roads and meandering walkways are bordered with private homes, a mosque, café and merchant shops.   

Other buildings are constructed inside a much larger warehouse type structure which gives a darker closed in effect.   Here there is a warren of narrow passageways, through which only a single file line of Marines can pass.   Closed doors and boarded windows present possibility of ominous threats within. 

 Snake through the passageway safely and you emerge into a town square, where you see Afghan men standing in their merchant stalls, offering vegetables and other wares.   They speak in Pashtu  and gesture with their hands to come near.   These are Animatrons, eerily lifelike in appearance and movement.   They are used in place of actors for background effect. 

Walk through one of the doors into a building, and an “angry grandmother” avatar pops up on the wall,   begins shouting in Pashtu and waving her hands at you to get out. 

There are more than 100 unique Avatar (scenarios) which can be utilized, all computer generated and programmed to deliver a variety of decision making challenges for the Marines.   These range in age from the elderly lady all the way down to a rifle toting twelve year old boy.

“We don’t want the Marines to experience a child threatening them for the first time in real life,” says Robert Thielen, Site Manager of the facility and our tour guide today.   “Avatars are programmed to deliver a large number of confrontational situations, which require Marines to decipher and correctly react within a matter of seconds.”

Look up and you see wires overhead, which  guide simulated but realistic rocket propelled grenade  ("RPG") explosions, cameras recording action as it unfolds, and tracking devices, which monitor the Marine’s movements through radio frequency identification  devices attached to the Marine’s vests.  

Around and behind the rabbit warren of walkway walls, is a command control center, filled with computers and monitoring devices.  In another room are several large screen TVs that beam live video feed during the training sessions. 

Marines are fitted with RFID devices attached to their vests which transmit their actions and movements onto the screens in the control center.   Those avatars which are programmed to appear only when activated come to life only when RFID indicate a Marine is entering the room.  In experimentation,  other avatars have been controlled by live puppeteers.  Staff wear urethane suits fitted with wired sensors which transmit motion to the avatars, enabling the avatars to interact in depth with the action of the Marines.    

Here, Marines come for 10-hour training days, which include two or three scenarios like the ones above.

Each training day involves an elaborate staging of mock conflicts, involving the squad of Marines, a host of contracted  Afghani actors, and the IIT staff behind the scenes who are filming, directing and influencing the action as it unfolds.   Staged “incidents” are designed but necessarily carry an element of spontaneous reaction as the Marines’ responses,  decisions and actions are real and unrehearsed.   

Site Manager Thielen explains that following each Immersion Training episode, the squads assemble in this room with staff to watch, dissect and learn from the incident and their own reactions to it.

Meanwhile, back in the debriefing room  video of a second incident begins rolling on the screen.  Without preface or explanation,  Thielen  has us watch the scene unfold.    In this manner we can almost feel the multitude of observations and rapid fire decisions  Marines must make.         

A Marine squad approaches the town.  Two Marines move to the right side of the screen and two move off to the left side.   Several Afghan villagers mill about, some in pairs, some alone.   A young boy pushes a wheelbarrow up the road.   A dog barks.   There are women among the villagers with their faces mostly hidden behind traditional head scarves. 

The Marine to the front left positions himself at a corner near a high mud-plastered wall.    He holds the stock of his M16 so that the barrel is perpendicular to his torso.      He shifts from foot to foot, and moves his head side to side, scanning the road.    Two turbaned, tall Afghans walk toward the Marine, their full length white robes flowing behind them in the wind.   They speak rapidly in Pashtu , and gesture with their right hands at the Marine.

He quickly moves the barrel of his M16 toward the pair and keeps his eyes on both men.   The two Afghans continue approaching and gesturing, until, when the Marine does not flinch or change posture, the men turn away from the Marine, and direct their conversation to one another, ignoring the Marine they had tried to engage.   The video flickers and then ends. 

There was no violence.  But was there progress?   Inexperienced eyes would see something quite different than would practiced eyes.   Robert  Thielen is a retired Master Sergeant  with thirty years experience in the Corps.    He has the heart and mind of a seasoned leader in war, and in life.   Ideally cast in his new position,  he helps us see what he sees and feel what Marines feel. 

We come away from this experience with our own immersion training.  What we witnessed gives us tremendous understanding and exposure to the myriad of seemingly inconsequential but, ultimately, life saving or threatening, behaviors  Marines will confront overseas. 

NEXT: What Immersion Training tests, teaches and tightens.