Skip to content

Instantly share code, notes, and snippets.

@bee-san
Last active August 31, 2018 10:07
Show Gist options
  • Select an option

  • Save bee-san/86d7bd01dd05f414cde1b889b8352754 to your computer and use it in GitHub Desktop.

Select an option

Save bee-san/86d7bd01dd05f414cde1b889b8352754 to your computer and use it in GitHub Desktop.
function inverseDocumentFrequency(document){
// calculates the inverse document frequency of every sentence
const words_without_stopwords = prettify(document);
const unique_words_set = uniqueWords(words_without_stopwords);
const sentences = document.split(".").map(item => item.trim());
sentences[0] = sentences[0].substring(146);
const lengthOfDocuments = sentences.length;
// prettifys each sentence so it doesn't have stopwords
const wordCountAll = countWords(words_without_stopwords);
// counts words of each sentence
// as each sentence is a document
wordCountSentences = [];
for (let i = 0; i <= lengthOfDocuments - 1; i ++){
wordCountSentences.push(countWords(prettify(sentences[i])));
}
// calculate TF values of all documents
let IDFVals = {};
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment