Students use AR, VR to plug in to helping those with neurodiversities

Junior Jinny Nguyen dives into the Lopes Mart Simulator during the College of Engineering and Technology’s Augmented Reality/Virtual Reality Showcase on Tuesday at the Cyber Center of Excellence.

Photos by Ralph Freso / Slideshow

On most days, Santa is hanging out with his elf buddies.

They’re talkin’ toys.

They’re talkin’ magical reindeer.

They’re eating cookies.

But today, Santa’s talkin’ nourishment at his Santa’s Fast Food Restaurant, where he’s behind the counter asking a customer, “What can I get for you today?”

“Pizza! Hot dogs! Chips!” one hungry customer blurts out.

Luckily, the restaurant offers those items. If that customer would have ordered, say, a chicken kabob, he might have been met with the dreaded, “Our elves don’t recognize that order.”

Junior Gerum Halle-Lincoln (center) explores Kahler Litynski (left) and Kyler Harden’s VR project, Santa’s Fast Food, during the AR/VR Showcase.

“We have Santa taking orders so it will put people at ease,” said Grand Canyon University software engineering sophomore Kyler Harden, who spent the spring semester creating the Santa’s Fast Food virtual reality project with classmate Kahler Litynski for Dr. Isac Artzi’s CST 320 Human-Computer Interaction and Communication course.

When the two were building assets for the project, Harden said that he and Litynski thought, “All right, we need to have someone take the orders. Do we use a serious person? What do we do?”

When they stumbled on a Santa asset online, they knew.

A student places a virtual reality order in the Santa’s Fast Food app.

“Let's do that. No one’s going to forget that – and everyone feels comfortable talking to Santa Claus.”

Santa’s Fast Food was just one of the end-of-semester projects students presented at the Augmented Reality/Virtual Reality Showcase Tuesday afternoon at the Cyber Center of Excellence in GCU’s Technology Building.

Students in the past have tackled everything from bringing Bible stories to life to teaching physics using VR. This time, they were tasked with building an environment to help learners with neurodiversities succeed.

“It could be helping someone who might have crowd anxiety or who might be on the autism spectrum or might be too nervous on a job interview,” said Artzi, associate professor of computer science who recently completed his master’s degree in clinical health mental counseling.

Harden knew what kind of project he wanted to build from the get-go.

“I’ve been around quite a bit of people on the (autism) spectrum,” he said. “I used to be the leader on the robotics team for my high school, and there were several people on the spectrum.”

They would go to a restaurant after a practice session and would notice that the robotics team members who were on the spectrum wouldn’t look at the menu and would just order something.

“We wanted to build a way for them to practice,” Harden said, and wanted to make them comfortable while doing so, so Santa seemed like the perfect fit.

The biggest challenge in building the app?

In 13 weeks, students are going from knowing nothing to ... we're asking you to complete a project as complex as writing a book."

Dr. Isac Artzi, associate professor of computer science

“The biggest challenge was to get the project to actually run in the headset. We ran into a lot of issues of incompatible versions and failure to build the application. But once we got past that, it was a lot of fun to build the assets,” Harden said.

He already knows what he and Litynski want to change about their prototype restaurant simulator. They want to make the menu more dynamic so users aren’t just memorizing the menu. They also are looking to tweak the speech-to-text model (they used a free speech-to-text option), which Harden said is slow and sometimes inaccurate.

Artzi said at the beginning of the semester, students weren't familiar with the programs or computer languages they might be using, so the learning curve is steep in the course.

“In 13 weeks, students are going from knowing nothing to ... we’re asking you to complete a project as complex as writing a book," Artzi said.

Sophomore Joey Clauss gets his headset adjusted by Matthew Edelson (left) and Sam Hardeman, who created the Lopes Mart Simulator.

Over the semester, computer science/big data sophomore Matthew Edelson created the “Lopes Mart Simulator,” a virtual game tribute to the campus convenience store.

The team, which also included fellow big data analytics students Alex Fried and Sam Hardeman, wanted to build a game to help employees with neurodiversities learn how to complete job tasks. They consulted with faculty at the LOPES Academy at the Cardon Center, a two-year, job-focused, nondegree college program at GCU for neurodiverse adults.

Several LOPES Academy participants have interned at Lopes Mart.

At Tuesday's showcase, visitors could pop on a Meta Quest VR headset, grab a couple of controls and see the interior of a campus Lopes Mart lovingly re-created virtually.

Users are prompted to place cereal boxes on shelves in the Lopes Mart Simulator.

They're directed to a list of tasks on a clipboard. One of the tasks: Place soda cans in the refrigerator display case.

Users turn around and see soda cans in front of a refrigerator. They press a button on the controller to grab a soda can, do the same to open the door of the refrigerator, then let go of the button to place the can on a red circle in the fridge. Once they do so, the red circle turns green.

Success! Mission complete.

In the game, players also can stock apples, soup cans, cereal dubbed "Mysterios" and the like.

“We’re trying to re-create this task so they can learn the technical aspect of what they have to do in their job and not have the pressure of customers or other people watching them,” Fried said. “It’s a stress-free environment where they can practice doing the tasks they have to do.”

Creating the game “was difficult at first because we had to learn how to use the program,” said Edelson.

Added Fried, “We were brand new to Unity (a cross-platform game engine), so we had to learn everything about the software that we were going to be on.”

Across the way at the Cyber Center of Excellence, software engineering junior Samson Becenti and his team were showing a fellow student what their Virtual Reality Job Interview Simulator could do.

Dr. Isac Artzi (right) listens as a student explains his VR project.

With the VR headset on, the student started talking about how she loved to play the guitar.

What she saw in the simulator: a job interviewer at a desk who asked her to introduce herself and what position she was applying for.

Depending on the answer, the virtual interviewer will change the questions, just like a real-life employer might do during a real-life job interview, Becenti said.

The team wanted to create the simulator, Becenti said, “to tackle autism, ADHD and social anxiety so we can help them feel more confident in their job interview.”

The idea is to provide a place for users to practice answering interview question so they’ll be more prepared, and less anxious, when they do go on that big job interview.

The idea, like for many experiences at GCU, is also to do good and help people.

Said Sam Hardeman, who was excited to explain his team’s Lopes Mart Simulator: “It’s an experience, not just a game, not just an app. It’s an experience."

Manager of Internal Communications Lana Sweeten-Shults can be reached at [email protected] or at 602-639-7901.

Related content:

GCU News: Virtual reality gym, pirate ship projects a treasure

GCU News: Technology a hit at end-of-program event

GCU News: Cyber program earns prestigious National Security Agency designation


Calendar of Events

M Mon

T Tue

W Wed

T Thu

F Fri

S Sat

S Sun

1 event,

0 events,

3 events,

3 events,

6 events,

3 events,

1 event,

0 events,

1 event,

0 events,

0 events,

1 event,

1 event,

1 event,

0 events,

0 events,

0 events,

0 events,

0 events,

3 events,

1 event,

0 events,

0 events,

2 events,

1 event,

0 events,

0 events,

0 events,

0 events,

2 events,

0 events,

GCU Magazine

Bible Verse

But the fruit of the Spirit is love, joy, peace, patience, kindness, goodness, faithfulness, gentleness and self-control. Against such things there is no law. (Galatians 5:22-23)

To Read More: