Xcode new build system debugging tricks

xcbuild-debugging-tricks.md

New Build System Tricks

Command Line

alias xcbuild=$(xcode-select -p)/../SharedFrameworks/XCBuild.framework/Versions/A/Support/xcbuild
# THIS DOESNT WORK YET: xcbuild openIDEConsole  # … then switch to Xcode ➡️
xcbuild showSpecs
xcbuild build <foo.pif> [—target <target>]

# Some minimal additional logging (this is safe to leave on).
defaults write com.apple.dt.XCBuild EnableDebugActivityLogs -bool YES

# Enable build debugging mode (safe to leave on, but slows down the build system & litters DerivedData/<project>/Build/Intermediates.noindex), generally should only be enabled when trying to capture a trace for incremental build debugging purposes.
defaults write com.apple.dt.XCBuild EnableBuildDebugging -bool YES
# You can also use:
env EnableBuildDebugging=YES xcodebuild -UseNewBuildSystem=1 ...

# Use `xcbuild` to dump a headermap.
xcbuild headermap --dump <path>

IDE Console

open with Internal > XCBuild Console

writePIF <workspace name> <path>
        # … you can use this with the `xcbuild build …` command (above) to build via the service directly
showStatistics
clearAllCaches
setConfig EnableBuildDebugging true
	# … then save DerivedData & build log; same as above dwrite, but not persisted

@ddunbar, is there a good way to detect why a specific file needed to be compiled even though Intermediates.noindex folder exists in the build folder? Like visualizing the dependency graph for the build?

Our incremental builds were working in the legacy build system if I keep Intermediates.noindex folder and guarantee that the mtime for source files are earlier than the mtime of corresponding object file under Intermediates.noindex.

With new build system I can't guarantee that anymore. Would be great if I can somehow debug this.

I guarantee mtime of the following file extensions to be older than object file generation: c,cc,h,pch,cpp,hpp,m,mm,xcconfig

@manuyavuz not yet, but we are actively working on code in the llbuild project (https://github.com/apple/swift-llbuild) to help with analysis.

Were you doing something special with modifying mtime? llbuild enforces exact match of mtime as well as some other file attributes.

@ddunbar, to be able to get incremental builds on our CI builds, I cache Intermediates.noindex folder, and mtimes of all related source files and .xcconfig files (Xcode understandably seems to rebuild target if corresponding .xcconfig file changes.

During next build, after a fresh checkout (where source files will surely have a later mtime than pre-built object files because git does not keep mtime as information), I modify mtime values of not changed source files to time when cached built was performed.

Actually, I'm using https://github.com/iboB/mtime_cache to perform all these steps, and it works pretty good with legacy build system. Once I reset mtime values for c,cc,h,pch,cpp,hpp,m,mm,xcconfig extensions, I get a very fast build. It still builds some parts of the files from scratch (e.g. source files under test targets) for some reason that I don't know yet, but in general it does a pretty good job.

However, I couldn't make the exact setting work under new build system. I assume new build system might be using more information to detect if a file should be rebuilt, and I'm not caching that info. Currently what I restore is basically just mtime values for not changed files, and it seems to be working good in general for legacy build system.

If you could guide me what should be added additionally, or how to debug an incremental build, it would be really appreciated! I tried to read a generated trace file after EnableBuildDebugging=1, but it's toooo much verbose, and probably being used by another internal script to provide a visualization of debug summary. It includes nodes in the graph and their attributes, along with tons of other information :)

I guess there is no easy way to directly use llbuild to build an iOS project, right?

No, you have to build through XCBuild, via Xcode.

@ddunbar thanks for sharing this.
I use xcspec files to look for execpaths so that I can override them for my customisations.
For example,
ExecPath = "$(SWIFT_EXEC)" in Swift.xcspec

I'm trying to figure out how to prevent the build system from trying to use discovered dependency file (.d) next.
To prevent the following error.

error: unable to open dependencies file

Any clue how can I make sure of xcspec files above to find what I need?