Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save thetafferboy/9553c1df2ea347ab9b01812912470f54 to your computer and use it in GitHub Desktop.
Save thetafferboy/9553c1df2ea347ab9b01812912470f54 to your computer and use it in GitHub Desktop.
{
"version": 1,
"snippets": [
{
"version": 1,
"javascript": "const settings \u003d {\\r\\n chatGpt: {\\r\\n \\/\\/ replace with your ChatGPT API key created at https:\\/\\/platform.openai.com\\/api-keys\\r\\n apiKey: \\\u0027ENTER CHATGPT API KEY\\\u0027,\\r\\n\\r\\n \\/\\/ the OpenAI model to use\\r\\n model: \\\u0027gpt-4-turbo\\\u0027,\\r\\n },\\r\\n alsoAsked: {\\r\\n \\/\\/ replace with your AlsoAsked API key created at https:\\/\\/alsoasked.com\\/developer\\/keys\\r\\n apiKey:\\r\\n \\\u0027ENTER ALSOASKED API KEY\\\u0027,\\r\\n\\r\\n \\/\\/ the language to search in\\r\\n language: \\\u0027en\\\u0027,\\r\\n\\r\\n \\/\\/ the region to search in\\r\\n region: \\\u0027gb\\\u0027,\\r\\n\\r\\n \\/\\/ the depth of the search\\r\\n \\/\\/ 2 is the default and returns the smallest number of questions, and costs 1 credit\\r\\n \\/\\/ 3 is the maximum and returns the largest number of questions, but costs 4 credits\\r\\n depth: 2,\\r\\n\\r\\n \\/\\/ whether to perform a fresh search, or attempt to use cached results\\r\\n \\/\\/ if cached results exist, this will not count towards your credit usage\\r\\n fresh: true,\\r\\n },\\r\\n};\\r\\n\\r\\nconst pageH1 \u003d document.querySelector(\\\u0027h1\\\u0027) ? document.querySelector(\\\u0027h1\\\u0027).innerText : \\\u0027\\\u0027;\\r\\n\\r\\n\\/\\/ ensure the page h1 is not empty\\r\\nif (!pageH1) {\\r\\n return seoSpider.error(\\\u0027The page h1 is empty\\\u0027);\\r\\n}\\r\\n\\r\\n\\/\\/ get questions from the AlsoAsked API using the page h1\\r\\nconst getQuestions \u003d new Promise(async (resolve, reject) \u003d\u003e {\\r\\n try {\\r\\n \\/\\/ the options to send to the AlsoAsked API search endpoint\\r\\n const searchOptions \u003d {\\r\\n terms: [pageH1],\\r\\n language: settings.alsoAsked.language,\\r\\n region: settings.alsoAsked.region,\\r\\n depth: settings.alsoAsked.depth,\\r\\n fresh: settings.alsoAsked.fresh,\\r\\n async: false,\\r\\n notify_webhooks: false,\\r\\n };\\r\\n\\r\\n \\/\\/ fetch the People Also Ask questions from the AlsoAsked API\\r\\n const response \u003d await fetch(\\\u0027https:\\/\\/alsoaskedapi.com\\/v1\\/search\\\u0027, {\\r\\n method: \\\u0027POST\\\u0027,\\r\\n headers: {\\r\\n \\\u0027Content-Type\\\u0027: \\\u0027application\\/json\\\u0027,\\r\\n \\\u0027X-Api-Key\\\u0027: settings.alsoAsked.apiKey,\\r\\n },\\r\\n body: JSON.stringify(searchOptions),\\r\\n });\\r\\n\\r\\n \\/\\/ ensure the response was successful\\r\\n if (!response.ok) {\\r\\n reject(\\r\\n `Failed to fetch questions from the AlsoAsked API: ${response.statusText}`\\r\\n );\\r\\n return;\\r\\n }\\r\\n\\r\\n const data \u003d await response.json();\\r\\n\\r\\n \\/\\/ check the response status\\r\\n if (data.status \u003d\u003d\u003d \\\u0027error\\\u0027) {\\r\\n reject(\\\u0027Failed to fetch questions from the AlsoAsked API\\\u0027);\\r\\n return;\\r\\n }\\r\\n\\r\\n if (data.status \u003d\u003d\u003d \\\u0027no_results\\\u0027) {\\r\\n reject(`No questions were found for the term \\\"${pageH1}\\\"`);\\r\\n return;\\r\\n }\\r\\n\\r\\n if (data.status !\u003d\u003d \\\u0027success\\\u0027) {\\r\\n reject(\\r\\n `Failed to fetch questions from the AlsoAsked API: ${data.status}`\\r\\n );\\r\\n return;\\r\\n }\\r\\n\\r\\n \\/\\/ ensure we have the results for the term we sent\\r\\n \\/\\/ this should never not be the case, as we\\\u0027ve checked\\r\\n \\/\\/ for errors above\\r\\n const queries \u003d data.queries;\\r\\n\\r\\n if (queries.length \u003d\u003d\u003d 0) {\\r\\n reject(`No questions were found for the term \\\"${pageH1}\\\"`);\\r\\n return;\\r\\n }\\r\\n\\r\\n \\/\\/ we only sent one term, so we should only get one set of questions back\\r\\n const query \u003d queries[0];\\r\\n\\r\\n \\/\\/ flatten the questions into a single array\\r\\n const flattenResults \u003d (questions) \u003d\u003e {\\r\\n return questions.reduce((flattenedQuestions, result) \u003d\u003e {\\r\\n flattenedQuestions.push(result.question);\\r\\n\\r\\n \\/\\/ if the question has children, flatten them too\\r\\n if (result.results) {\\r\\n flattenedQuestions.push(...flattenResults(result.results));\\r\\n }\\r\\n return flattenedQuestions;\\r\\n }, []);\\r\\n };\\r\\n\\r\\n const questions \u003d flattenResults(query.results);\\r\\n resolve(questions);\\r\\n } catch (error) {\\r\\n reject(`Failed to fetch questions from the AlsoAsked API: \\\\n${error}`);\\r\\n return;\\r\\n }\\r\\n});\\r\\n\\r\\n\\/\\/ query the ChatGPT API to find unanswered questions\\r\\nconst queryChatGpt \u003d (questions) \u003d\u003e {\\r\\n return new Promise(async (resolve, reject) \u003d\u003e {\\r\\n try {\\r\\n \\/\\/ the options to send to the ChatGPT API completions endpoint\\r\\n const completionsOptions \u003d {\\r\\n model: settings.chatGpt.model,\\r\\n response_format: { type: \\\u0027json_object\\\u0027 },\\r\\n messages: [\\r\\n {\\r\\n role: \\\u0027system\\\u0027,\\r\\n content: `List the questions in this JSON array ${JSON.stringify(\\r\\n questions\\r\\n )} which are not answered in the text content of this page, but would make sense to answer in context to the rest of the content. Output the questions that are not answered in a JSON array of strings within an object called unanswered_questions.`,\\r\\n },\\r\\n {\\r\\n role: \\\u0027user\\\u0027,\\r\\n content: document.body.textContent,\\r\\n },\\r\\n ],\\r\\n };\\r\\n\\r\\n \\/\\/ fetch the unanswered questions from the ChatGPT API\\r\\n const response \u003d await fetch(\\r\\n \\\u0027https:\\/\\/api.openai.com\\/v1\\/chat\\/completions\\\u0027,\\r\\n {\\r\\n method: \\\u0027POST\\\u0027,\\r\\n headers: {\\r\\n \\\u0027Content-Type\\\u0027: \\\u0027application\\/json\\\u0027,\\r\\n Authorization: `Bearer ${settings.chatGpt.apiKey}`,\\r\\n },\\r\\n body: JSON.stringify(completionsOptions),\\r\\n }\\r\\n );\\r\\n\\r\\n \\/\\/ ensure the response was successful\\r\\n if (!response.ok) {\\r\\n reject(\\r\\n `Failed to fetch response from the ChatGPT API: ${response.statusText}`\\r\\n );\\r\\n return;\\r\\n }\\r\\n\\r\\n \\/\\/ ensure we have a response from the ChatGPT API to our question\\r\\n const data \u003d await response.json();\\r\\n const choices \u003d data.choices;\\r\\n\\r\\n if (!choices || choices.length \u003d\u003d\u003d 0) {\\r\\n reject(\\\u0027No response was returned from the ChatGPT API\\\u0027);\\r\\n return;\\r\\n }\\r\\n\\r\\n \\/\\/ ensure the response contains the unanswered questions\\r\\n const responseText \u003d choices[0].message.content;\\r\\n\\r\\n if (!responseText) {\\r\\n reject(\\\u0027No response was returned from the ChatGPT API\\\u0027);\\r\\n return;\\r\\n }\\r\\n\\r\\n \\/\\/ parse the response text as JSON\\r\\n \\/\\/ from the question we asked, we expect the format to be\\r\\n \\/\\/ { \\\"unanswered_questions\\\": [\\\"question 1\\\", \\\"question 2\\\", ...] }\\r\\n const result \u003d JSON.parse(responseText);\\r\\n\\r\\n if (!result.unanswered_questions) {\\r\\n reject(\\\u0027No unanswered questions were found in the response\\\u0027);\\r\\n return;\\r\\n }\\r\\n\\r\\n resolve(result.unanswered_questions);\\r\\n } catch (error) {\\r\\n reject(`Failed to fetch response from the ChatGPT API: \\\\n${error}`);\\r\\n return;\\r\\n }\\r\\n });\\r\\n};\\r\\n\\r\\n\\/\\/ initially get the questions from the AlsoAsked API based on the\\r\\n\\/\\/ page h1, then query the ChatGPT API to find unanswered questions\\r\\nreturn getQuestions\\r\\n .then(queryChatGpt)\\r\\n .then((unansweredQuestions) \u003d\u003e {\\r\\n return seoSpider.data(unansweredQuestions);\\r\\n })\\r\\n .catch((error) \u003d\u003e {\\r\\n return seoSpider.error(error);\\r\\n });",
"name": "(AlsoAsked+ChatGPT) Find unanswered questions",
"comments": "Get PAA questions from the h1 of your URL via AlsoAsked API and use ChatGPT to list those that aren\u0027t answered in content and might make sense to answer",
"type": "EXTRACTION",
"actionTimeoutSecs": 1,
"contentTypes": "text/html"
}
]
}
@jrobison
Copy link

jrobison commented May 8, 2024

Can you do a walkthrough video? Unclear exactly where to paste the API keys and what to remove.

@thetafferboy
Copy link
Author

thetafferboy commented May 8, 2024

Can you do a walkthrough video? Unclear exactly where to paste the API keys and what to remove.

This is JSON for Screaming Frog and designed to be imported using their Import funciton (details here: https://www.screamingfrog.co.uk/seo-spider-20/). If you import it, it will be formatted as JavaScript and will be very clear where to put API keys :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment