I'm documenting upgrade issues and solutions to them in case someone else runs into the same issue.
This is because of angular/angular-cli#23866, the migration somehow didn't fix it for me.
To fix the issue go into your main.server.ts file and remove the line that exports from '@angular/platform-server'.
You can also search for '@angular/platform-server' in files and remove the line that looks like this: