Elias Dorneles eliasdorneles

Scrapy Release Procedure

This page outlines the procedure used to release a new (major/minor) version of Scrapy. Micro/patch versions are published automatically.

You will need bumpversion installed.

Here is an example, assuming we are releasing 1.1.0rc1:

Focus on scraping use cases

Warn about /table/tbody/tr in the query, if no results (common mistake)
- Consider using HTML5 parser instead
Shorter method names
Guess encoding in the case of bytes input
Selector vs SelectorList <- could we have only one class, make it list-like? (like jQuery)
Consider adding a shortcut to parse JSON (demjson, to support liberal input JS-like)
Add a simple link extraction method

	#!/usr/bin/env python
	# -- coding: utf-8 --
	"""
	For those times when you just want a button.

	Run and notice the new icon in your notifications area.

	Inspired by: https://glyph.twistedmatrix.com/2015/07/just-a-button.html
	"""

	#!/usr/bin/env python
	# A toy Python BDD framework


	# Decorator for test classes
	def _context():
	def add_test_method(cls, method, name):
	def test(self, args, *kwargs):
	return method(self, self.subject(), args, *kwargs)
	test.func_name = 'test_' + name

	#!/usr/bin/env python
	# -- coding: utf-8 --
	"""Reservoir sampling for line-based input
	"""

	from __future__ import print_function
	import sys
	import random

	#!/usr/bin/env python
	# -- coding: utf-8 --
	"""Lisp parser (future interpreter)
	"""

	from __future__ import print_function
	import sys
	import re
	from collections import namedtuple

	#!/usr/bin/env python
	# -- coding: utf-8 --
	"""Cookiecutter PyPackage development tasks
	"""
	# To add a task, create a function with name starting with 'do_' that
	# receives one argument (the parsed cmdline arguments object).
	# Use a short docstring to serve as help.
	#
	# Example:
	#

	>>> js_with_html_comments = u'<!-- some... --> alert("oi"); <!-- ...comments -->'
	>>> import js2xml
	>>> from lxml import etree
	>>> def strip_comments(js):
	....: node = etree.fromstring(u'<node>%s</node>' % js)
	....: etree.strip_tags(node, etree.Comment)
	....: return node.text
	....:
	>>> strip_comments(js_with_html_comments)
	' alert("oi"); '

	#!/bin/bash
	# This script assumes you're using the directory layout described in the
	# documentation at: https://voc.readthedocs.io/en/latest/tutorials/tutorial-0.html

	set -e
	set -x

	abort() {
	echo "$*"; exit 1;
	}

	# How one could go about implementing a check for abstract Python classes
	# written in the old style (raise NotImplementedError),
	# by looking at the methods bytecode


	def is_oldstyle_abstract(cls):
	def has_raise_notimplemented_code(func_code):
	# this is too magic, I know.
	# you can see how it works by disassembling the bytes below:
	#