So, I can replicate this ballooning:
$ docker -v
Docker version 17.06.2-ce, build cec0b72
Here's the base image size after doing "Reset to factory defaults"
du -h /Users/curtis/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2
1.3G /Users/curtis/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2