Support for functional pseudo-elements

[redapple]

Some modifications to the parser to explicitly handle pseudo-elements and functional pseudo-elements.

XPath translation happens in xpath_xxxx_functional_pseudo_element and xpath_xxxx_pseudo_element which are probably not the best method name patterns.

A new flag in GenericTranslator is used to handle these customizations or keep original behaviour.

This is somewhat related to scrapy/scrapy#176 (though I'm not sure that very PR will use pseudo-elements in the end)

[dangra]

this is great stuff, it will be used for scrapy/scrapy#176 for sure.

[redapple]

Travis CI build failed on job 17.4 which uses Python 3.1, which isn't available (anymore) in Travis CI (http://about.travis-ci.org/docs/user/ci-environment/#Python-VM-images)

[redapple]

Rebased to make things tidier.

[SimonSapin]

Hi. Thanks a lot for starting this. It turns out I have pretty strong opinions on how to do this, so rather than having you jump through hoops I went ahead and wrote the thing. I didn’t use code (other than tests) from this PR, though, sorry :)

Rather than having a boolean flag to enable this and have pseudo-elements be part of the "normal" tree, pseudo-elements are still in the Selector.pseudo_element attribute. In addition to None or a string for a "simple" pseudo-element, this attribute can be an instance of the new FunctionalPseudoElement class.

I don’t want to encourage too much implementing pseudo-elements with XPath (as I think this is the wrong approach), but in order to still enable what Scrapy wants to do, "translator" objects now have a xpath_pseudo_element method that just raises but can be overridden to transform the XPathExpr object. On other words, the name-based method dispatching is left up to the subclasses, as show in the tests.

[redapple]

That's ok Simon, I wasn't particularly proud of it anyway ;)
I'll have a look at your implementation.
Thanks.

[SimonSapin]

I just released cssselect 0.9.1, which is backward-incompatible with 0.9 if you use pseudo-elements with the selector_to_xpath method. Details in the changelog.

[redapple]

Thanks for the update @SimonSapin