If you've been writing Scrapy spiders, you've probably found yourself doing this:



def parse(self, response):
# Extract data from current page
yield {'title': 'something'}

# Find all links
for link in response.css('a::attr(href)'):
yield response.follow(link,...