Skip to content

Instantly share code, notes, and snippets.

@yishenggudou
Last active December 11, 2015 00:39
Show Gist options
  • Select an option

  • Save yishenggudou/4518041 to your computer and use it in GitHub Desktop.

Select an option

Save yishenggudou/4518041 to your computer and use it in GitHub Desktop.

###Spider提交任务说明

####匹配内容

支持jquery语法 Jquery变量为Q

网页html为变量context

支持python语法

#####正文 Q(context)('.content_single').html()

#####标题 Q(context)('title').text().split('-')[0]

#####cat [Q(i).text() for i in Q(context)('[rel=category tag]')]

或者直接`['it','python']

#####tag [Q(i).text() for i in Q(context)('[rel=tag]')]

或者直接`['it','python']

全站网址

如:luo.bo/sitemap.xml

批量网址

支持数据范围, 如 �http://luo.bo?p={id} id=range(10,100,2)

单个网址

一般不建议单个网页 这样成本有点大 如 直接填写网址就行

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment