Idea to get the spider works

README.md

Idea to get the spider works

Make the spiders work!

Info

This topics of the documentation are a must read to understand how everything works.

• http://doc.scrapy.org/en/latest/topics/request-response.html
• http://readthedocs.org/docs/scrapy/en/latest/topics/spiders.html

Idea

The is ideas is from BManojlovic http://pastie.org/3133918

Thanks BManojlovic whoever you are that help me a lot.

spider.py

 def parse(self,response):
        hxs = HtmlXPathSelector(response) 
        companies=hxs.select('//div[contains(@class,"rezultatPretrage")]/h2/a/@href').extract()
        for url in companies:
            yield Request(urljoin_rfc(get_base_url(response), url), callback=self.parseCompanyData)

        nexturl=hxs.select('//div[@class="stranicePretrage"]/a/img[@alt="Next"]/..').select('@href').extract()
        if nexturl:
            yield Request(urljoin_rfc(get_base_url(response), nexturl[0]), callback=self.parse)

    def parseCompanyData(self, response):

You are welcome :)