Figuring out neovim windows escaping issues

The problem

I thought this was fixed earlier, but the following fails to execute if shell=cmd.exe

:echo system('cd /d "c:\Windows"')

Take the following script

let &sxq='('
let &sxe='"&|<>()@^'

echo system('cd "c:\Windows"')
echo system('echo "c:\Windows"')
echo system('echo ""')

let &sxq=''
let &sxe=''

echo system('cd "c:\Windows"')
echo system('echo "c:\Windows"')
echo system('echo ""')

In Neovim

The filename or directory name, or volume label syntax is incorrect 
\"c:\Windows^\"
\"^\"
The filename or directory name, or volume label syntax is incorrect 
\"c:\Windows\"
\"\"

In Gvim

"c:\Windows"
""
"c:\Windows"
""

So backslash escaping seems independent of the sxq/sex?

About system(string:cmd)

system() executes the shell (set shell, set shellcmdflag) with the given string as argument i.e. argv would be [shell, shellcmdflag, cmd]. In practice this is not exactly accurate because shell, shellcmdflag can contain multiple arguments, but as far as this issue is concerned the problem lies with cmd.

The string cmd is converted according to options like sxq and sxe. The current defaults for windows seem wrong, but this does not explain the current issue.

After this conversion argv is passed on to do_os_system() that calls libuv to spawn the process. Libuv does its own escaping at this point.

Diagnosis

The actual argv array is build by shell_build_argv(), here is a print out of the array when running the previous example (sxe and sxq are empty)

0: cmd.exe
1: /c
2: cd "c:\Windows"

At this point this is passed on to do_os_system(). We already have tests for the underlying behaviour of system([...]) in tests/functional/eval/system_spec.lua. And given the previous output this should also happen with system(['cmd.exe', '/c', '"c:\Windows"']) ... and in fact it does, with the exact same error.

In Vim 8.0 :echo system('build\bin\printargs-test.exe "c:\Windows"') prints arg1=c:\Windows;.

Note: nvim includes a test binary printargs-test.exe that will print outs its arguments, you can use it to help debug this.

References

Of course we can try to use the regular vim setup for powershell

set shell=powershell
set shellcmdflag=-Command

http://stackoverflow.com/questions/94382/vim-with-powershell

Solutions

Libuv quoting function - https://github.com/libuv/libuv/blob/87df1448a48fb64c2b9ebe37e3344f7e5b81dd88/src/win/process.c#L480 does indeed escape inner quotes with a backslash.

" this fails, prints -\"\"-
set shell=cmd.exe
set shellcmdflag=/c
echom &shell &shellcmdflag
echom system('echo -""-')

" but it works with powershell, prints --
" (in powershell echo "" does not print the quotes)
set shell=powershell.exe
set shellcmdflag=-Command
echom &shell &shellcmdflag
echom system('echo -""-')

" FIXME: echom prints trailing ^@

So cmd.exe is a special snowflake when it comes to argument escaping.

This post does an excelent job of explaining why the inner quotes are handled this way in cmd.exe and how to fix it. The problem from our point of view is that the fix proposed there would need to happen after the quoting done by libuv i.e. it would need to be done upstream, possibly with an additional quoting flag - but it seems hard to justify without a more general case. The alternative would be to always set UV_PROCESS_WINDOWS_VERBATIM_ARGUMENTS and handle this on our own.

A couple easier ways to go forward

We know from previous experience that simply setting UV_PROCESS_WINDOWS_VERBATIM_ARGUMENTS breaks system([...]). I assume most windows programs do follow this convention (CommandLineToArgvW), and that we want to keep libuv's argument quoting when using system([...]), the problem then is if someone calls system(['cmd.exe', ...]) a quick google revelead at least one other program that does its own weird command line handling besides cmd.exe.
We could disable UV_PROCESS_WINDOWS_VERBATIM_ARGUMENTS for system('...') and hopefully shellxquote/shellxescape and shellescape() would handle this case. Presumably this would be the most compatible with Vim.

I don't have a solution for 1. Putting together a PR for 2, there are however some corner cases like &shell or &shellcmdflag having spaces and needing quoting.

equalsraf/neovim-escaping.md

The problem

About system(string:cmd)

Diagnosis

References

Solutions