AR Quick look allows people to quickly place 3d models in the real world to see what they look like. Developers / designers provide a 3D model and AR Quick look does the rest. This uses quick look technologies throughout the system. On non-ARKit compatible devices the user can only explore the model, without seeing it in the real world. AR models can contain animations. Transform and skeletal animations are supported.
AR Quick look also works with accessibility and voice over. User get queues when items go off screen or when they come back on screen.
Files, mail messages, notes, news and safari have adopted AR Quick look. People can now use a special html tag to add 3d models to websites.
USDZ is a new file format for distributing 3D models in iOS. A model and textures are bundled in a single file. It’s based on Pixar’s USD format. This file format is supported on iOS and macOS and a converted is available.
You can also integrate AR Quick look in your own apps to allow people to easily view 3d models in their environment. You have control over transitions and presentation modes.
Create an instance of QLPreviewController and set a dataSource and a delegate. When you present the preview controller, you are asked for the number of items that need to be previewed. Then you are asked for a url for a preview item. Finally, you are asked for a view that should be the parent for the presentation of the preview controller.
You decide how the controller is presented, you can for instance push it on a navigation stack or show it modally.
The data source protocol is QLPreviewControllerDataSource. You can cast a URL to a QLPreviewItem. AR Quicklook is intended to be shown fullscreen, that’s how it works best. The transition works best if your source view has square frame because that animates very naturally.
Safari on iOS now supports USDZ contents natively. If you use Apple’s suggested html markup, you automatically get a badge that indicates that the content is AR compatible.
<a rel="ar" href="model.usdz">
<img src="preview-image.jpg">
</a>
This also works with the <picture> html element. Only rule is that you must provide a single <picture> child.
Your USDZ file must have the correct Mime-Type.
Six things must be done when creating the model:
- Placement
- Size
- Animation
- Contact shadow
- Appearance
- Transparency
To get placement right, the object should face towards the camera. The base of the object must be on the floor and the pivot point must be configured correctly.
When rotating the option, position it as if you are placing the object in a display in the real world. When in doubt ask a child to draw the object. Children often pick the most natural orientation when drawing objects.
If your object is too small or too large when you export it, it will look wrong in the real world. You must make sure that you understand and know your units in the 3D software you use.
Imaginary objects, like a funny robot should be made a size that makes them easy to position and scale.
When you make a fun model of a nice character that is alive, add an idle animation. For instance a small head bop can make a model look alive instead of boring and static. Don’t animate objects away from where they are to make sure people can easily pick it up and move it.
Animations should make sense at a static location. For instance making an object run on the spot is weird. But making a dinosaur stamp it’s feet makes sense.
A fish is a floating creature so keeping it in one place is strange. You can add a surrounding scene, like a bit of floor, to ensure that there is a nice bounding box that remains similar.
Do not bake a shadow into the model. AR Quick Look will generate one based on. The environment. By default, the first frame in the model is used by AR Quick Look to generate the shadow. Make sure to pick a first frame that makes as much sense as possible.
AR Quick Look uses a Physically Based Rendering (PBR) shader. The following is analysed by this shader:
- Albedo (base color) RGB / RGBA
- Metallic areas Grayscale
- Roughness Grayscale
- Normal (surface details) RGB
- Internat shadows on the model Grayscale
- Light emitters RGB
More info about this shader and using it is in the Creating Great AR Experiences talk.
- Transparency should be communicates through a separate material so don’t mix transparent and non-transparent in one material.
- Transparency is communicated through a texture that has transparency in the image
- Any image format supported by iOS can be used
- Textures should be square, dimension should be a power of 2
Since iOS 12 is supported on many devices, iOS will take care of optimisations. Make sure to optimise for a high-power device like the iPhone X. iOS will downsample textures for slower devices. Animations and meshes will remain the same, it’s only textures that are downsized.
- Freeze transforms and merge adjacent vertices.
- Less materials / textures is better
- You don’t have to provide all textures so don’t include ones you don’t need
- Focus on areas in your texture that adds most realism
- Be aware that pixels have a physical size in AR
As part of Xcode 10, a command line tool to convert to USDZ is provided. It maps PBR textures textures to meshes and submeshes and accepts OBJ, Alembic and existing USD files.
A USDZ file is basically an uncompressed zip file. It contains a USDC file and all textures that are used by the model.
Basic use case:
xcrun usdz_converter source_file target_file.usdz
There are possibilities to take more control about grouping of meshes and textures.