Last active
September 29, 2017 22:26
-
-
Save Granitosaurus/b9a1bdb97ee0d767066d121749fa671d to your computer and use it in GitHub Desktop.
converts a list of games into linked items with reviews.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import requests | |
from parsel import Selector | |
def scrape(): | |
data = """ | |
7 Grand Steps: What Ancients Begat (DRM Free + Steam) | |
2064: Read Only Memories (DRM Free + Steam) | |
A Virues Named TOM (DRM Free + Steam) | |
AI War: Fleet Command (DRM Free + Steam) | |
Day Of The Tentacle Remastered (DRM Free + Steam) | |
Ellipsis (Steam) | |
Girls Like Robots (DRM Free + Steam) | |
Guacamelee (Super Turbo Championship Edition) (DRM Free + Steam) | |
Hot Tin Roof: The Cat That Wore A Fedora (DRM Free + Steam) | |
Human Resource Machine (DRM Free + Steam) | |
Invisible Inc. (DRM Free + Steam) | |
Mini Metro (DRM Free + Steam) | |
Monster Loves You! (Steam) | |
Mushroom 11 (DRM Free + Steam) | |
Ninja Pizza Girl | |
No Time To Explain (Steam) | |
Nuclear Throne (Steam) | |
Octodad: Dadliest Catch (Steam) | |
Overgrowth (Early Access) (DRM Free + Steam) | |
Retro Game Crunch (DRM Free + Steam) | |
Robot Roller-Derby Disco Dodgeball | |
ROCKETSROCKETSROCKETS (DRM Free + Steam) | |
Secrets Of Rætikon (DRM Free + Steam) | |
Song Of The Deep (Steam) | |
Spirits (DRM Free + Steam) | |
Sproggiwood (DRM Free + Steam) | |
The Stanley Parable (DRM Free + Steam) | |
Stardew Valley (Steam) | |
Streamline (Early Access) (Steam) | |
Subnautica (Early Access) (Steam) | |
Super Hexagon (DRM Free + Steam) | |
Super Meat Boy (DRM Free + Steam) | |
Superbrothers: Sword & Sworcery EP (DRM Free + Steam) | |
The Swapper (DRM Free + Steam) | |
Thirty Flights Of Loving (Steam) | |
Tower Of Guns (DRM Free + Steam) | |
VVVVVV (DRM Free + Steam) | |
Waking Mars (DRM Free + Steam) | |
The Witness (DRM Free + Steam) | |
World Of Goo (DRMM Free + Steam) | |
""" | |
items = data.splitlines() | |
items = [i.split(' (')[0] for i in items if i] | |
for item in items: | |
# Search string on steam, get first result | |
resp = requests.get('http://store.steampowered.com/search/?term={}'.format(item)) | |
first = Selector(text=resp.text).css('a.search_result_row::attr(href)').extract_first().split('?')[0] | |
# Also get rating | |
resp = requests.get(first) | |
rating = Selector(text=resp.text).css("span.game_review_summary::attr(data-store-tooltip)").re('\d+%') | |
rating = rating[1] if rating else '~%' | |
# print markdown'ed string | |
print('[{}]({}) - {} '.format(item, first, rating)) | |
if __name__ == '__main__': | |
scrape() |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This strategy of taking the first result fails with "Spirits (DRM Free + Steam)".