Skip to content

Instantly share code, notes, and snippets.

@NikolaRusakov
Last active March 12, 2020 15:50
Show Gist options
  • Select an option

  • Save NikolaRusakov/02e192c82a490964e52ffc1b2df6c5a9 to your computer and use it in GitHub Desktop.

Select an option

Save NikolaRusakov/02e192c82a490964e52ffc1b2df6c5a9 to your computer and use it in GitHub Desktop.
Scrape of LinkedIn profile, the manual way
const cheerio = require('cheerio')
// TODO: feed me with selected dom element with class of .profile-detail
const detailsCs = { data: ''}
const ce = cheerio.load(detailsCs.data, {
withDomLvl1: true,
normalizeWhitespace: true,
decodeEntities: true
});
let jsonframe = require('jsonframe-cheerio');
jsonframe(ce); // initializes the plugin
const skills = {
topSkills: {
heading: '.pv-skill-categories-section .pv-profile-section__card-heading',
skills: {
_s: ".pv-skill-category-entity__top-skill",
_d: [{
"name": '.pv-skill-category-entity__name-text',
"count": '.pv-skill-category-entity__endorsement-count',
"highlight": '.custom-highlight'
}]
}
},
otherSkills: {
sections: [".pv-skill-category-list h3"],
skills: {
_s: '#skill-categories-expanded .pv-skill-category-list',
_d: [{
heading: '.pv-skill-categories-section__secondary-skill-heading',
skill: {
_s: '.pv-skill-category-entity--secondary',
_d: [{
"name": '.pv-skill-category-entity__name-text',
"count": '.pv-skill-category-entity__endorsement-count',
}]
}
}]
}
}
};
const positions = {
positions: {
heading: '#experience-section .pv-profile-section__card-heading',
experience: {
_s: '.pv-profile-section__card-item-v2',
_d: [{
employedDuration: '.pv-entity__date-range',
duration: '.pv-entity__bullet-item-v2',
company: '.pv-entity__secondary-title',
companyLogo: 'img @ src',
position: 'h3',
location: '.pv-entity__location',
details: '.pv-entity__extra-details'
}]
}
},
education: {
heading: '#education-section .pv-profile-section__card-header',
data: {
_s: '#education-section .pv-entity__summary-info',
_d: [{
schoolName: '.pv-entity__school-name',
degreeName: '.pv-entity__degree-name .pv-entity__comma-item',
fieldOfStudy: '.pv-entity__fos .pv-entity__comma-item',
degree: '.pv-entity__grade .pv-entity__comma-item'
}]
}
},
skills
};
const expList = ce('.profile-detail').scrape(positions, {string: true})
console.log(expList);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment