Skip to content

Instantly share code, notes, and snippets.

@bee-san
Last active August 27, 2018 13:16
Show Gist options
  • Select an option

  • Save bee-san/f05c3e648d70b8225138f669f76b5391 to your computer and use it in GitHub Desktop.

Select an option

Save bee-san/f05c3e648d70b8225138f669f76b5391 to your computer and use it in GitHub Desktop.
function inverseDocumentFrequency(documents){
// calculates the inverse document frequency of every sentence
const words_without_stopwords = prettify(documents);
const sentences = documents.split(".")
sentences[0] = sentences[0].substring(146);
const lengthOfDocuments = sentences.length;
const WordCountDocuments = countWords(words_without_stopwords);
// calculate TF values of all documents
const unique_words_set = uniqueWords(words_without_stopwords);
let IDFVals = {};
for (let i = 0; i <= unique_words_set.length - 1; i++){
IDFVals[unique_words_set[i]] = Math.log10(lengthOfDocuments / WordCountDocuments[unique_words_set[i]]);
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment