A rough guide on how to embed a Unity view in a native iOS or React Native-based app

CustomVuforiaNativeRendererController.mm

// NOTE: Copied and cleaned-up from Unity/Libraries/VuforiaNativeRendererController.mm
// REASON: The .mm file did not expose an interface and categorically overrode the Unity app delegate subclass
// using `IMPL_APP_CONTROLLER_SUBCLASS(VuforiaNativeRendererController)`, preventing us from providing our own
// subclass of `VuforiaNativeRendererController` to serve as the final app delegate. Diff this file with the
// source file (specified above) to view the applied changes.

#import "CustomVuforiaNativeRendererController.h"
#import "VuforiaRenderDelegate.h"

// Unity native rendering callback plugin mechanism is only supported
// from version 4.5 onwards
#if UNITY_VERSION>434

// Exported methods for native rendering callback
extern "C" void VuforiaSetGraphicsDevice(void* device, int deviceType, int eventType);
extern "C" void VuforiaRenderEvent(int marker);

#endif

@implementation CustomVuforiaNativeRendererController

- (void)shouldAttachRenderDelegate
{
  self.renderDelegate = [[VuforiaRenderDelegate alloc] init];
  
  // Unity native rendering callback plugin mechanism is only supported
  // from version 4.5 onwards
#if UNITY_VERSION>434
  UnityRegisterRenderingPlugin(&VuforiaSetGraphicsDevice, &VuforiaRenderEvent);
#endif
}
@end

unity-embed-ios-native.md

Notes for Unity integration into a native iOS / React-Native app

WARNING: As of Unity 2019.3, there is an official way to integrate Unity into an existing app, although it's not without its limitations. You should consider doing that intead of this procedure if possible.

• Works as of January 2018 (Xcode 9, Unity 2017.0.2f3)

• Note that we don't use the Unity-generated project at all here; instead, we add some of the Unity-generated files to our own Xcode project (either a native Xcode-generated project or the one generated by React-Native).

• Also note that this is not a complete guide, however this contains the most hard-to-resolve issues that we encountered.

Relevant Resources

A combination of information found in those articles was used to perform our integration:

• https://the-nerd.be/2015/11/13/integrate-unity-5-in-a-native-ios-app-with-xcode-7/ (outdated but still a good overview)
• https://forum.unity.com/threads/unity-appcontroller-subclassing.191971/page-1
• http://www.makethegame.net/unity/add-unity3d-to-native-ios-app-with-unity-5-and-vuforia-4-x/

Integration Hints

• Roughly follow this guide: https://the-nerd.be/2015/11/13/integrate-unity-5-in-a-native-ios-app-with-xcode-7/

  • Settings inside Unity itself are less important as they mostly affect the Unity-generated Xcode project, which we don't use
  • Add frameworks as described (IMPORTANT!)
  • Add groups and folder references as described (IMPORTANT!)
  • But skip the build settings part; instead, transcribe all "Customized" build settings from the Unity-iPhone target in the Unity-generated Xcode project into the custom app (there is a "Customized" filter at the top of Xcode's build settings that makes customized settings easy to spot). This will ensure that the latest requirements of the Unity project are mirrored in our custom app.
  • Set Dead Code Stripping to YES to avoid some more linker errors (https://forum.unity.com/threads/unity-arkit-plugin-faq.495191/)

• Build the app for a real device (or Generic iOS Device); Simulator is not supported!

  • Else some libraries will fail to link due to missing x86_64 architecture in the .a files
  • Specifically, this is an issue with libil2cpp (required for Unity) and Vuforia static libraries (if using Vuforia)

• Since ~ 2013, Unity unofficially supports taking control of the app delegate to simplify customizing the Unity-generated app.

  • Use IMPL_APP_CONTROLLER_SUBCLASS(AppDelegate) to "subclass" the UnityAppController class.
  • Caution: Vuforia already uses this method to initialize its own stuff, so we must customize the Vuforia setup a bit in order to be able to provide our own subclass. See CustomVuforiaNativeRendererController.h/.mm for details.
  • Don't forget to call superclass methods in the custom AppDelegate!
  • The "officially" supported way to take ownership of the root app UI is to override -willStartWithViewController: in our custom AppDelegate

• Unity MUST be initialized with the app (managed by its global UnityAppController) for it to work properly.

  • This implies that the Unity stuff is always loaded
  • However, it can be paused and resumed at will using UnityPause(1) and UnityPause(0), preventing app slowness and battery drain.

• When using new features in the Unity C# code, some files might be added in the "Native" directory of the Unity-built Xcode project. Make sure to update the "real" Xcode project to reference those files as needed!

How's the performance? Looking to use React Native for the UI layer, and dive into Unity for AR ...

Hi, where is the CustomVuforiaNativeRendererController file?
My unity view shows the camera and vuforia marker, initiate the camera in log, but can't recognize any marker.

Thanks!

Performance is as good as it can be, as the setup is exactly the same as the app Unity outputs (Objective-C code initializing the Unity view in a native app). React Native only serves as a layer on top of that to manage when views are shown. So, no performance hit. Something to keep in mind, though: There can only be one Unity view in the app AFAIK. You might cheat by moving that view around your app's view hierarchy, but we didn't try this.

Hi, have you tried to do this using ARkit? If so any special considerations?

You can put [self.unityController applicationDidBecomeActive:[UIApplication sharedApplication]]; in showUnityWindow method, then unity part won't be loaded with native stuff.

Have you tried this on newer versions of unity and react native at all? We are having issues with reactnative unity view 1.2.2 and any unity version above 2018.2.17, it simply doesn't show the unity app at all, or rather is transparent (as I can still communicate with it), wondered if you had any suggestions at all?

@zarackritter Sorry for the delay. The CustomVuforialNativeRendererController.mm file is specific to our implementation and was something I forgot to add to my original gist. I've added it now. You might have to tweak it for your project.

@cauri Sorry for the delay.

Hi, have you tried to do this using ARkit? If so any special considerations?

I did not, but once the Unity part is working I don't see why any AR stuff within the Unity view would not work, as long as it's stuff supported by Unity or a Unity package for ARKit that normally works on iOS.

@pixelknight I am not familiar with react-native-unity-view, but the latest versions of Unity we've used this with is 2018.1.0f2. We've often had "transparent" unity view issues while trying to make the whole thing work initially, but unfortunately I can't remember what we did to fix them.

@fortinmike hello. Will you be doing the integration of unity into ios when it is used a library? When you have is as a framework. I am quite unsure how to code it up natively to load Unity

@Danik711 Not sure what you're asking about, but as you can see from the integration notes it's quite a hassle (and it's somewhat prone to breaking when Unity and Xcode are updated).

I'd recommend avoiding integrating Unity in a native iOS application this way if at all possible. We certainly won't do it again ourselves and we have no intention of updating this procedure unless a Unity / Xcode version bump is necessary to push an update of our client's app. AFAIK there is still no simple (official) way to add a Unity UIView into a native iOS app.

@fortinmike, as for Unity 2019.3 and higher developers may export Unity as a xcode project that has a Unity.framework that you can add to your xcode project. Developers create a workspace out of two xcode projects and just do some native code to run unity.

Anyway, if unity is not a good decision, then may you please let me know how to add Augmented Reality to native ios apps. It was no problem to do that for Android, but IOS is a beast, if you know what I am talking about.

@Danik711 Interesting! I didn't know that was now officially supported. If you can live with the limitations (described here) it seems like this is a much better way to integrate Unity inside a native iOS app, rendering most of my procedure obsolete.

I'd be much more comfortable recommending integrating Unity in an app this way. Of course our original goal of using Unity was to implement the 3D part of our app once and have it work on both iOS and Android directly, instead of having to implement things separately as ARKit and ARCore, so Unity remains relevant.

Anyway, if you want to go for native iOS (say you already have an Android implementation of AR with Android-specific code and you don't mind reimplementing it in native iOS) I suppose the way to do it is using RealityKit and Reality Composer but honestly I haven't looked much into that.

@fortinmike, I can definately live with these limitations. My issue here is that I do not know how to properly code it up in objective-c. If you are a pro at ios developement I will be greatful if you find some time to make a new tutroial on that. I will subscribe to all of you social media :). I am more into Android dev and if you are asking why I need IOS it is because I want to do it in react-native. I was able to do it for Android and render AR Unity via native modules now I need IOS. If you also redner Unity via react-native ios native modules I will fall in love with you 🤣

Sorry I cannot help you with that, I'm on something else entirely at the moment. However, there is a lot of information on the subject online. You might take a look at what raywenderlich.com offers on that front, if you're looking for tutorials.