Skip to content

Instantly share code, notes, and snippets.

@rafikahmed
Created September 29, 2018 11:37
Show Gist options
  • Save rafikahmed/01a138dc4a1e8baea263853008824079 to your computer and use it in GitHub Desktop.
Save rafikahmed/01a138dc4a1e8baea263853008824079 to your computer and use it in GitHub Desktop.
import scrapy
from scrapy.loader.processors import MapCompose, TakeFirst
from w3lib.html import remove_tags
def remove_whitespace(value):
return value.strip()
class JokeItem(scrapy.Item):
joke_text= scrapy.Field(
input_processor= MapCompose(remove_tags, remove_whitespace),
output_processor= TakeFirst()
)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment