Clone the Swift compiler and all the related subprojects and then just create the build directories without compiling:
mkdir swiftlang
cd swiftlang
git clone https://github.com/apple/swift.git
./swift/utils/update-checkout --clone
cd swift
utils/build-script --release-debuginfo --xcode --skip-build
That's it.