Bash echo handles newlines based on shebang lines and POSIX compliance

shebang.md

How echo handles newlines based on shebang lines and Bash settings

The problem

Why would sh and bash perform differently on a system where they map to the same binary executable?

Consider such an environment:

> which sh && which bash
/bin/sh
/bin/bash

> /bin/sh --version && /bin/bash --version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin19)
Copyright (C) 2007 Free Software Foundation, Inc.
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin19)
Copyright (C) 2007 Free Software Foundation, Inc.

Running a simple echo "Hello\nWorld" statement through each shell demonstrates the problem:

> sh
sh-3.2$ echo "Hello\nWorld"
Hello
World

> bash
bash-3.2$ echo "Hello\nWorld"
Hello\nWorld

This can also be demonstrated within scripts. Note the only difference in the below scripts is the shebang line and the difference of each script's output.

> cat sh.sh && ./main.sh
#!/usr/bin/env sh
echo $SHELL
echo "Hello\nWorld"
/bin/bash
Hello
World

> cat bash.sh && ./main.sh
#!/usr/bin/env bash
echo $SHELL
echo "Hello\nWorld"
/bin/bash
Hello\nWorld

What's going on?

While #!/usr/bin/env sh is regarded as a highly portable line, it also changes how Bash works. One's first inclination would be to assume that the behavior of sh and bash would be identical. However, in direct defiance of my expectations, Bash changes how it handles newline characters in echo output based on its invocation as sh.

When invoked as sh, Bash enters POSIX mode after reading the startup files. https://www.gnu.org/software/bash/manual/html_node/Bash-POSIX-Mode.html#Bash-POSIX-Mode

A downstream effect of this the default setting of xpg_echo, which echo uses to address what to do with escaped characters,

If the -e option is given, interpretation of the following backslash-escaped characters is enabled... The xpg_echo shell option may be used to dynamically determine whether or not echo expands these escape characters by default. https://www.gnu.org/software/bash/manual/html_node/Bash-Builtins.html#Bash-Builtins

And indeed, we can see the difference clearly when working directly in Bash and using the echo flags:

> bash
bash-3.2$ echo "Hello\nWorld"
Hello\nWorld
bash-3.2$ echo -e "Hello\nWorld"
Hello
World

Conclusions

If the functions and commands in your shell scripts are behaving differently in your script than in your terminal, it probably has to do with the level of POSIX compliance. The best way to test shell scripts using a portable sh-referencing shebang line is to invoke your your shell with the same command.