"Whenever I start thinking about CORS, my intuition about which site hosts the headers is incorrect, just as you described in your question. For me, it helps to think about the purpose of the same origin policy.
The purpose of the same origin policy is to protect you from malicious JavaScript on siteA.com accessing private information you've chosen to share only with siteB.com. Without the same origin policy, JavaScript written by the authors of siteA.com could make your browser make requests to siteB.com, using your authentication cookies for siteB.com. In this way, siteA.com could steal the secret information you share with siteB.com.
Sometimes you need to work cross domain, which is where CORS comes in. CORS relaxes the same origin policy for
domainA.com, using the Access-Control-Allow-Origin
header to list other domains (domainB.com) that are trusted