Scrape Moodle courses data from module list search

readme.md

Scrape Moodle Module List

On the Manage Activities admin page there's a count of how many courses use a particular activity and you can click the count to execute a search that retrieves all those courses. There's a button to show all the courses on the same page, but no useful way to do further filtering or extract data. The JS above can be copy-pasted into your browser's JavaScript console to create a CSV of the courses listed on the module list page.

We have parentheticals for semester in our course titles like "20th Century Fashion (2022SP)" so this attempts to extract those, though the regex is primitive and will mess up with courses with a second set of parentheses. This is usually easy to cleanup manually.

scrape-moodle-modulelist.js

let output = "url,title,term,instructors,category,category_url\n"
const termRegex = / \(([0-9]{4}[A-Z]{2})\)/
$('.coursebox').each((idx, cb) => {
    // <a class="aalink" href="https://moodle.cca.edu/course/view.php?id=3294">20th Century Fashion (2022SP)</a>
    let course = [$(cb).find('.coursename .aalink').attr('href')] // course url
    course.push($(cb).find('.coursename .aalink').text().replace(/"/g, '""')) // course title
    const term_matches = termRegex.exec($(cb).find('.coursename .aalink').text())
    term_matches ? course.push(term_matches[1]) : course.push('')
    course.push($(cb).find('.teachers a').map((i, t) => t.textContent).get().join(", ")) // instructors
    course.push($(cb).find('.coursecat a').text()) // category
    course.push($(cb).find('.coursecat a').attr('href')) // category_url
    output += '"' + course.join('","') + '"\n' // quote all fields
})
console.log(output)