Skip to content

Instantly share code, notes, and snippets.

@mucaho
Created July 14, 2022 10:10
Show Gist options
  • Save mucaho/df29f96df5bb5ea781b1dfa899a2b450 to your computer and use it in GitHub Desktop.
Save mucaho/df29f96df5bb5ea781b1dfa899a2b450 to your computer and use it in GitHub Desktop.
Google Sitemap Test

Minimum working example for issue described in https://support.google.com/webmasters/thread/171071268?hl=en

  • Requires installation of Node.js

  • Put these files into a directory, run npm install to install dependencies

  • Use ssh -R 80:localhost:8080 localhost.run to setup a temporary tunnel to your local machine, leave that shell open

  • Copy & paste the publicly accessible tunnel url that will be established, e.g. https://XXX.lhrtunnel.link

  • Open Google Search Console and create a new URL prefix property with that tunnel link

  • Use the meta tag verification method and paste it into the project's index.html file's header

  • In a new command shell, run npm start to start the server at localhost:8080

  • Complete property verification

  • Try to submit the sitemap /sitemap.xml to Google. It should fail with Couldn't fetch sitemap

  • Comment out the X-Robots-Tag header in app.js and try to resubmit. It should succeed.

const express = require('express')
const app = express()
const port = 8080
const path = require('path')
const fs = require('fs')
const replace = require('buffer-replace')
app.get('/', function(req, res){
res.redirect('/index.html');
});
app.get('/index.html', (req, res) => {
res.contentType('html')
res.sendFile('index.html', { root: path.join(__dirname) })
})
app.get('/sitemap.xml', (req, res) => {
// TODO: submitting sitemap doesn't work with this header, comment it out to fix it!
res.setHeader('X-Robots-Tag', 'noindex')
var buffer = fs.readFileSync('sitemap.xml', 'utf-8')
buffer = replace(buffer, '$hostname$', req.headers.host)
res.contentType('xml')
res.send(buffer)
})
app.listen(port, () => {
console.log(`Example app listening on port ${port}`)
})
<!DOCTYPE html>
<html lang='en'>
<head>
<title>Basic HTML5 document</title>
<meta charset='utf-8'>
<meta name="google-site-verification" content="REPLACE_ME" />
</head>
<body>
Test html!
</body>
</html>
{
"name": "google-sitemap-test",
"version": "1.0.0",
"description": "",
"main": "app.js",
"scripts": {
"start": "node app.js"
},
"author": "mucaho",
"dependencies": {
"buffer-replace": "^1.0.0",
"express": "^4.18.1"
}
}
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>https://$hostname$/index.html</loc>
<lastmod>2022-07-14</lastmod>
</url>
</urlset>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment