New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add CRAWL_ONCE_RESET setting #5

Open

SteveSmirnoff wants to merge 1 commit into TeamHG-Memex:master from SteveSmirnoff:master

SteveSmirnoff commented Mar 17, 2021 •

edited

Loading

Aimed to fix #4
Added setting similar to DELTAFETCH_RESET

Expected usage:
in settings.py:
CRAWL_ONCE_RESET = True
or in terminal:
scrapy crawl spider_name -a crawl_once_reset=True

If True,
SqliteDict.clear() is called on the db


          CRAWL_ONCE_RESET setting added

8ee8004

SteveSmirnoff marked this pull request as ready for review

March 17, 2021 10:01

SteveSmirnoff changed the title ~~CRAWL_ONCE_RESET setting added~~ add CRAWL_ONCE_RESET setting

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet