Getting into XR as designer
This article was originally published on my personal website.
Quite often, folks reach out to me asking how they can get into the VR/AR field. Usually, they’re design students or designers. I don’t believe that there is one short path to getting the title of XR designer. As I can see, everyone ended up here for different reasons and in different ways. Here I write what skills might be useful for you in pursuing an XR career.
This article should be helpful not only for designers but for coders, game designers, architects, philosophers, and just anyone else.
I should remind you that this guide is only my point of view, and doesn’t necessarily match with job descriptions or someone else’s recommendations.
1. UX skills
It’s not the most important skill for the majority, but it’s the one that brought me to VR in the first place. UX for VR (AR, MR, whatever XR) is a bit of a wild west in terms of interactions. There is so much to explore and invent.
In this section, I cover all the skills that help you understand why and how things work. I used the term UX, as it’s the closest to User eXperience for a flat UI.
1.1. Flow
XR apps, the same as any other apps, have flows and user journeys. Being able to draw flowcharts or even better to visualize them is an excellent skill. Yeah, quite often the navigation will be different from that on a website, but the idea of flow is still the same.
To practice, try to draw flowcharts of experiences that you tried. Also, before working on any of your own experiences, draw the user journey diagram before jumping to development or design. It can be a diagram with sketches of environments, specific UI elements, or just names of scenes and states.
1.2 Interactions
When mobile came out, it took some time to figure out all the advantages of gesture-based input and nail down all interactions. Spatial interactions are even more different. Yeah, there is a lot to explore, but some norms and expected behaviours are critical to know.
To start with, try paying attention to the mechanics that different experiences use. I remember when Serious Sam for VR came out, I bought it right away because it had most of the locomotion options. I was changing settings and checking how each of them feels. Obviously, I got quite dizzy after it :)
Next, try to come up with new interactions. I’m sure that you’ve thought “I could do better” while playing VR. It might be easier than you think. Check #3 😉
2. Visual skills
This one is about what you would call UI design for flat interfaces. Everything that relates to what experience looks like.
But for VR and AR, it might be even more important than for flat UI. Look, you can just go into Figma, Sketch or whatever, draw a few rectangles, add a few lines of text and get the design of a website. For VR it’s not so easy, as instead of rectangles you should use cubes 😉
2.2 3D
You should know how 3D works. 3D modeling software is getting simpler and more approachable every day, but it’s still a long way from being as easy to get into as Figma. You could also use assets from Sketchfab for all your designs, but anyway sooner or later, you will need some custom assets.
I would advise starting with Blender, as it’s free. You don’t need to be a pro in shaders, but a basic understanding of 3D modeling and texturing will open the door to creating immersive experiences.
Also, you could model directly in VR. It might be even easier to get into, but eventually, you would need to have some desktop app to convert or texture stuff. For modeling in VR, I would recommend Gravity Sketch, which has recently become free.
A few other tools popular in the field: Cinema 4D, Maya, Houdini.
2.3. Video
The same as any other point in this list, it’s not a must-have, but a great contribution to your toolbox. I often use videos to quickly show ideas in high quality, without building a prototype, and when static pictures aren’t enough. Like this or this.
I use After Effects, which is a great, but kind of complicated tool. I wouldn’t advise you to learn it when you’re only starting out. For showing how UI components would work, even the Sketch + Principle would be enough.
2.4 Regular UI design
Having experience in traditional UI design is quite useful. First, because you get the idea of how different components can have different properties and affordances. Also, XR UI quite often has a similar set of components to NUI. I don’t think that just copying flat gesture-based interfaces into 3d space is a good thing in general, but definitely knowing how it works is crucial.
There is no need to go crazy here; any 2d design tool is good to go, prepare flat assets, design parts of the mockups, or just draft your ideas to share with coworkers or clients. You can use Figma, Sketch, Photoshop, Illustrator or even Keynote or Google Slides.
2.5. Shaders and textures UV
This is 3D-specific. As you might guess, 3d objects don’t just have colours, but a wider set of properties to describe their surface look. It’s a good idea to do research about how shaders and PBR materials work, and what UV mapping is for 3d models.
Just a few videos on YouTube is good enough to start with. Then you can practice creating and tuning materials and shades in your platform of choice. For texturing, I would advise trying Substance Painter. Just check out their video for inspiration:
3. Prototyping
And if you want to experience it, you should be able to build it. “What?”, I can hear you saying, “I’m a designer. I tell other people how to develop stuff”.
For me, this is the main difference between regular UI and spatial design. There is no way to know if something would work or not until you experience it. And the easiest way is to build it yourself. Yeah, large companies like Facebook pair designers and prototypers that work as a single unit, each bringing their own part of the expertise.
Of course, you can hire or just ask your Unity developer friend to help you build your ideas. But when starting (and I believe all the time), iterating fast is the key to growth. You should make your workflow loops as short as possible, as with each iteration you will keep growing and gaining more and more experience for the next one.
3.1 Coding
Let’s put it this way: without knowing how HTML/CSS works you wouldn’t be able to design a website that works fine on all platforms. It’s the same story here. You should have at least some basic ideas how 3D rendering, game engines, and surface recognition for AR work. It’s always better to be able to build things with your hands. And coding, in general, is a great skill that can be applicable to many other fields, for example after learning C# for Unity, you can get into the development of Snapchat lenses, or even web apps using JavaScript.
Before selecting a programming language, I would advise you to select a game engine. Using game engines is the easiest way to get things built, as they and all additional packages handle most of the basic stuff for you, leaving you to enjoy the building.
3.2 Unity
This is the most popular game engine. It means that there are countless paid and free course tutorials and how-to videos. It makes it quite easy to pick up. It’s the first to get all the integrations with new tech.
I would advise you to get a taste of building your own stuff as quickly as possible. After this you will be unstoppable ;). At first I would go through some basic tutorials on the official Unity website to get an idea how it works in general and the principles of 3D. Then jump into YouTube and try to repeat some simple tutorials on building anything for VR or AR.
Then you can go back and learn advanced coding techniques for Unity, but with a clear understanding of how to apply them. One more trick is to avoid building something new from scratch. Try to do a tutorial, and think about how you can modify it. Or how to combine a few tutorials or principles.
And don’t forget that it’s not about the destination, but about the journey. I believe my Unity skills are still poor, but I never aimed to become a Unity developer in the first place. I want to know Unity just enough for building prototypes that I want. There will always be some things that are confusing, or just don’t work, but the trick is just to figure them out and keep moving forward.
3.3 Unreal
This is the second most popular game engine. It doesn’t have such a huge community as Unity and is more difficult to pick up. But unsurprisingly it has something to offer that Unity does not. It has way better quality. So, if your goal is to make everything photorealistic, and just top-notch, you can have a look at Unreal.
I have to admit I don’t have much experience with Unreal. I used it a few years ago for building VR experiences of my wife’s interior designs, and believe me they looked awesome in Unreal. But I somehow didn’t have a reason to work in it more, as my focus was on quick interactions.
The same principle of getting your hands dirty first and figuring out the rest later applies here as well.
3.4 AR platforms
Augmented reality lenses in Instagram and Snapchat are becoming more and more advanced. You can build quite cool stuff using Lens Studio or Spark AR. It’s not the most flexible or scalable way of entering into XR as it’s locked into a single platform. But I’m sure that you can have plenty of fun working on AR lenses.
I would advise starting from official documentation and tutorials first (Lens studio, Spark AR)
3.5 Prototyping in VR
There are quite a few tools for 3D modeling directly in VR. They can be successfully used to create “mockups” of 3D environments, to get an idea of how it feels and looks. Of course, they are quite limited, as you can’t test any interactions, but sometimes it’s just enough to get the job done.
You can try Gravity sketch, Maquette, tvori, Minsar Studio etc.
4.Soft skills
Didn’t see it coming?
Obviously, they are important for any profession, but for XR it’s especially the case. Therefore I decided to add them as a bonus skills set. Here are the key ones:
4.1 Empathy
Each successful designer should have the empathy to understand users’ needs and goals, but for XR it’s even more important. VR and MR immerse users into their environments and provoke emotions. We should always keep in mind users’ feelings and intentions that make sure that the app is delivering exactly the experience that we planned.
Try to step into the user’s shoes every time you develop something and think about how the user feels. And then think about if it’s a good thing or not. For example, Facebook is good at connecting people, but making people scroll through a series of ads is a good thing for the company, not for the user. There is a saying that VR is the ultimate empathy machine. Couldn’t agree more. We should do our best to use it for good.
4.2 Grit
You will have days when Unity just won’t listen to you. And you will have weeks when each of your interaction ideas gets worse and worse. Also, you might have months when your projects won’t take off, whatever you do. And your excitement about how VR will change how people interact, live, or whatever won’t help you. In these times you need to have grit. It’ll help you to move forward and keep doing your stuff until you have a good day.
4.3 Communication
“I can talk,” you say. Sometimes, and quite often, it’s just not enough. Having clear communication is crucial for the XR field. A lot of concepts are new or still fuzzy. Therefore the ability to deliver your idea clearly is so important.
Here are the main areas that you need to focus on. Again there is no need to perfect each of them, but having a basic understanding of all of them definitely helps, especially in understanding what you want to do next.
There are quite a few other directions that are gaining popularity. And as the field grows, they will grow as well, for example, you can look into AR and VR gaming mechanics, spatial audio, advanced shaders, etc.
Hope this article didn’t confuse you more than help. You can also check out my work process as an MR designer contractor.
Let me know your feedback, and what else you would want to learn.
