You can limit docker-windows memory usage by adjusting some parameters in wsl2 configuration.
First, close any instances of Docker Windows currently running and create a new file named .wslconfig in your user home directory, like C:\Users<username>:
[wsl2]
memory=4GB
processors=5