As of 12/11/2020, VMware no longer signs Tools for Windows Vista with SHA-1, so you need Windows 7 SP1 or later to use the latest version.
However, you can use an older version of VMware Tools, which is here: https://media.githubusercontent.com/media/thepwrtank18/thepwrtank18/master/VMware%20Tools%20for%20Windows%207%20RTM%20and%20Vista.iso
Link is dead. Instead, here's a full on guide.
This is an unsupported operating system, with unsupported drivers. Use this at your own risk.
You will need:
- Either a:
- x64 Windows Vista ISO
- x64 Windows Vista Disc + Reader
- VMware Workstation (Pro or Player) 15.x, or Pro 16.x and higher.
- An old version of the VMware Tools ISO (10.3.5 to be precise), from here
Note: Do not let VMware scan your ISO or disc, Easy Install may use the wrong version of Windows, whether a wrong edition or architecture.
Make a VM. If you have Pro or Player 15.x, do what you normally do. If you have Pro 16.x, make sure compatibility is set to Workstation 15.x (in Custom settings). After that, customize the settings you want. (Player does not support custom settings, so you either need to get 15.x, which is what you need anyway, or Pro 16.x and higher.)
Make sure you have your ISO or disc selected in the VM options. If there's a floppy disc or extra DVD drive with "autoinst" or the like on it, remove that drive.
Install as usual. When it tells you to create a user profile, don't do that for now. Go to the VM settings, then put in the VMware Tools ISO you got earlier. Don't install VMware Tools via VM>Install VMware Tools. After that, press Shift+F10 in the VM. Put in D:\setup64.exe
.
Do the VMware Tools installation as usual. When it says if it wants to restart the machine, say "Yes", then go to VM>Power>Reset. When it reboots, do the setup as normal. Let it monitor your computer's performance, as drivers are installed.
Hello, unsure about Windows 7 RTM, but for Windows Vista SP2 you can apply a KB4474419 update for Server 2008 which adds SHA2 support. VMware Tools installer also checks for Windows 7 and certain components depend on Windows 7 API. Some time ago I modified an early version of VMware Tools 11.x and added own 7+ stubs to make it fully working on Vista. D3D11 and OpenGL 4.1 acceleration was working fine in Vista guest.

If anyone is interested I might try that again with latest VMware Tools at some point.