Shadowed submodule error

error.md

Shadowed submodule error

Problem

I am adding Swift Package Manager support to the camera_avfoundation package. See: flutter/packages#6710

However, the Mac_x64 ios_build_all_packages master check fails:

Warning: Building for device with codesigning disabled. You will have to manually codesign before deploying to device.
Building com.example.allPackages for device (ios)...
Running pod install...                                              3.6s
Running Xcode build...                                          

Xcode build done.                                           16.7s
Failed to build iOS app
Parse Issue (Xcode): Build a shadowed submodule 'camera_avfoundation.messages_g'

Encountered error while building for device.

Building all_packages using Cocoapods failed.

Reproduction

Simple reproduction

You can reproduce this locally using Xcode 15.2 or older; Xcode 15.3 and newer do not reproduce this error:

1. Checkout my branch: https://github.com/loic-sharma/flutter-packages/tree/spm_camera

   git checkout -b loic-sharma-spm_camera main
   git pull [email protected]:loic-sharma/flutter-packages.git spm_camera
   

2. Add use_frameworks! to packages/camera/camera_avfoundation/example/ios/Podfile:

   diff --git a/packages/camera/camera_avfoundation/example/ios/Podfile b/packages/camera/camera_avfoundation/example/ios/Podfile
   index bcdae34190..61fe86ef41 100644
   --- a/packages/camera/camera_avfoundation/example/ios/Podfile
   +++ b/packages/camera/camera_avfoundation/example/ios/Podfile
   @@ -28,6 +28,8 @@ require File.expand_path(File.join('packages', 'flutter_tools', 'bin', 'podhelpe
    flutter_ios_podfile_setup
   
    target 'Runner' do
   +  use_frameworks!
   +
      flutter_install_all_ios_pods File.dirname(File.realpath(__FILE__))
   
      target 'RunnerTests' do

3. Now build the camera_avfoundation example app using Cocoapods:

   sudo xcode-select --switch /path/to/xcode/15.2/or/older
   flutter config --no-enable-swift-package-manager
   cd packages/camera/camera_avfoundation/example
   flutter clean
   flutter build ios
   

   This will error:

   Running Xcode build...
   Xcode build done.                                           16.1s
   Failed to build iOS app
   Parse Issue (Xcode): Build a shadowed submodule 'camera_avfoundation.messages_g'
   
   
   Encountered error while building for device.
   

Minimal reproduction

1. Follow the simple reproduction steps above

2. Open the example app in Xcode 15.2 or older

3. In Project navigator, open Runner > Flutter > Debug. Append VERBOSE_SCRIPT_LOGGING=true.

4. Clean your build folder & do a build

5. Open the Report navigator and expand the failing step:

   

6. Copy the clang command.

7. In your terminal, and run the clang command. This should reproduce the build error:

   fatal error: build a shadowed submodule 'camera_avfoundation.FLTCamMediaSettingsAVWrapper'
   /Users/loicsharma/Library/Developer/Xcode/DerivedData/Runner-ajljfhewikpbjigrasflzcnbhont/Build/Intermediates.noindex/Pods.build/Debug-iphonesimulator/camera_avfoundation.build/module.modulemap:1:18: note: previous definition is here
   framework module camera_avfoundation {
                    ^
   1 error generated.
   

The build error is caused a clang invocation that compiles a .m file. The intermediate build folder included in this build invocation contains only the correct Cocoapods module map (it does not contain the Swift Package Manager module map).

Known workarounds

Use Xcode 15.3 or newer

The app builds fine using Cocoapods if you use Xcode 15.3.

If you edit the clang compile command from the minimal repro above to use Xcode 15.3's clang executable, the file compiles successfully.

Use Swift Package Manager

This build error only reproduces if you use Flutter's Cocoapods integration. This build error does not reproduce if you enable Swift Package Manager.

Remove use_frameworks! from your Podfile

This build error only reproduces if your app's Podfile has use_frameworks!.

The camera_avfoundation example app builds fine if this line is removed.

Rename the Swift Package Manager module map

Flutter plugins that support Swift Package Manager and have custom module maps will have two .modulemap files.

For camera_avfoundation:

1. Cocoapods module map: packages/camera/camera_avfoundation/ios/camera_avfoundation/Sources/camera_avfoundation/include/cocoapods_camera_avfoundation.modulemap
2. Swift package manager module map: packages/camera/camera_avfoundation/ios/camera_avfoundation/Sources/camera_avfoundation/include/module.modulemap

If you rename or delete the Swift package manager module map, the build error no longer reproduces.

This indicates that clang is unexpectedly using the Swift Package Manager module map when Swift Package Manager is disabled.

Potential solution

It appears that the Swift package manager module map is used unexpectedly when Flutter's Swift package manager feature is disabled. This hypothesis would also explain bug flutter#148307.

I've created an experiment that separates the public header file directories for Swift package manager and Cocoapods: flutter/packages#6732

The camera_avfoundation plugin now has two directories containing public header files:

1. packages/camera/camera_avfoundation/Sources/camera_avfoundation/include - This is the Cocoapods public header directory
2. packages/camera/camera_avfoundation/Sources/camera_avfoundation/public - This is the SPM public header directory

This experiment appears to solve this issue.

However, this experiment has a drawback: it requires significant code duplication. I plan to do some follow-up experiments:

1. Reduce the number of public header files by converting them to private header files. This would be a breaking change but it does reduce the code duplication significantly
2. Use symlinks instead of duplicating header files. This would make it easier to keep header files synced across SPM and Cocoapods.

Pub doesn't support symlinks. Example error: https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8747737440527014529/+/u/Run_package_tests/publishability/stdout

ERROR: Pub does not support publishing packages with directory symlinks: `/b/s/w/ir/x/w/packages/packages/camera/camera_avfoundation/ios/camera_avfoundation/Sources/camera_avfoundation/cocoapods_headers/camera_avfoundation`.

Tracking issue: dart-lang/pub#3143
Pull request that attempted to add symlink support: dart-lang/pub#3298