Dockerfile alternatives for heredoc

heredoc-dockerfile.snip

#printf
RUN printf '#!/bin/bash\n\
echo hello world from line 1\n\
echo hello world from line 2'\
>> /tmp/hello

#echo
RUN echo -e '#!/bin/bash\n\
echo hello world from line 1\n\
echo hello world from line 2'\
>> /tmp/hello

RUN printf '%s\n' \
  '#!/bin/bash' \
  '' \
  'set -xe' \
  '' \
  'INTERFACES="$(ip -o link | sed -ne "s/[[:digit:]]*: \\(eth[[:digit:]]*\\)[^$]*/\\1/p")"' \
  '' \
  'echo "${INTERFACES}"' \
  \
> /entrypoint.sh && chmod a+x /entrypoint.sh

A more readable version imo.

👍

You can base64 encode the heredoc contents and then have the docker build process decode them.

A python example:

from textwrap import dedent
import base64
heredoc = dedent(
    """
    echo 'your
    script
    goes
    here'
    """
)
heredoc_b64 = base64.b64encode(bytes(heredoc, "utf-8")).decode()
dockerfile = dedent(
    f"""
    FROM whatever
    RUN echo '{heredoc_b64}' | base64 -d  > myscript.sh
    RUN bash myscript.sh
    """
)

@MatrixManAtYrService while this is nice, it completely obfuscates what is going on. I think the other approaches also had the intention to show the content of the script (e. g. maybe because some paths or variables are being used or even substituted).
Otherwise yes, this would good for longer scripts and to just use a single Dockerfile without anything else. But then maybe also add compression (if gzip is installed) for base64 encoding to somewhat decrease the size of the string. Not sure if base64 is a default package/tool for most images?

Finally buildkit added support for heredocs : https://www.docker.com/blog/introduction-to-heredocs-in-dockerfiles/

@Querela agreed, it's not a good solution for persistent Dockerfiles. Later on in the script I pipe the string into docker build - so there's never a Dockerfile for anyone to read and be confused about. heredoc support in buildkit seems even better though.

Yes, heredoc support is great.
I did some decrypting of data etc in a dockerfile entrypoint and embedded the script so no external files were required. But developing this took longer than expected because of different levels of quote escaping, subshells and parameter substitution or escaping ... It was fun but nothing you would want to revisit later. And a nightmare to debug.
An excerpt below: (DECRYPT_* are docker run environment variables)

RUN printf '%s\n' \
    '#!/bin/bash' \
    '' \
    'DATA_DIR=/mnt/data' \
    'echo "Data dir: ${DATA_DIR} ... $(du -sh ${DATA_DIR} | cut -f1)"' \
    '' \
    'echo "* Decrypt data ..."' \
    ' ( \' \
    '   KEY="${DECRYPT_KEY}" IV="${DECRYPT_IV}" \' \
    '   bash -c " \' \
    '       find ${DATA_DIR} -iname '"'"'*.jsonl.bz2.enc'"'"' -type f -exec \' \
    '       bash -c '"'"' \' \
    '           FILE="\${0%.enc}"; \' \
    '           openssl aes-256-cbc -K \${KEY} -iv \${IV} -d -in \"\${FILE}.enc\" -out \"\${FILE}\"' \
    "       '"' {} \; \' \
    '   " \' \
    ' )' \
    '' \
    'echo "* Decompress data ..."' \
    'find ${DATA_DIR} -iname '"'*.jsonl.bz2'"' -type f -exec bzip2 -d {} \;' \
    '' \
    'echo "* Cleanup data ..."' \
    '# remove encrypted and compressed data' \
    'find ${DATA_DIR} -iname '"'*.jsonl.bz2.enc'"' -type f -exec rm {} \;' \
    'find ${DATA_DIR} -iname '"'*.jsonl.bz2'"' -type f -exec rm {} \;' \
    '' \
    'echo "Data dir (ready): $(du -sh ${DATA_DIR} | cut -f1)"' \
    '' \
    'echo -e "\nDataset list:"' \
    'ls -1shR ${DATA_DIR}/*' \
    '' \
    > prepare_data.sh && \
    chmod +x prepare_data.sh

Also, there is official way to do it:

# syntax=docker/dockerfile:1.3-labs

RUN <<EOF
echo "Hello" >> /hello
echo "World!" >> /hello
EOF

I use it as the following:

# syntax=docker/dockerfile:1.4
RUN cat <<EOF >> /usr/local/bin/startup.sh
if [ \$# -gt 0 ]; then
    exec "\$@"
else
    /usr/bin/tini -- /usr/local/bin/jenkins.sh
fi
EOF

kudos @maxx27! I was banging my head why in 2024 it still has to be so difficult to use heredocs in Dockerfiles in a reasonable way. That last snippet of yours is the way to go! Thanks!

FROM debian
RUN <<EOT bash
  set -eux
  to_install=(
    vim
  )
  apt-get update
  failing_command # set -e exit with non-zero status 
  apt-get install -y "\${to_install[@]}"
EOT

# execute list shell
RUN <<EOF
apt-get update
apt-get upgrade -y
apt-get install -y ...
EOF

# run python
RUN <<EOF
#!/usr/bin/env python3
with open("/hello", "w") as f:
    print("Hello", file=f)
    print("World", file=f)
EOF

# create index.html
COPY <<EOF /usr/share/nginx/html/index.html
<html>hello, index</html>
EOF

# create /etc/mysql/my.cnf
RUN <<-EOF > /etc/mysql/my.cnf
[mysqld]
user=root
datadir=/app/mysql
port=3306
log-bin=/app/mysql/mysql-bin
EOF