To remove the GB language pack:
Get-AppxPackage -AllUsers Microsoft.LanguageExperiencePacken-GB | Remove-AppxPackage
Then in the registry, remove the GB entry from Computer\HKEY_CURRENT_USER\Keyboard Layout\Preload
.
According to MSDN docs
the value should be 0x809
.
If it is not the last value, make sure to rename the others!