mhdzumair 863247018d Add dramatiq-abort support and extend scrapy CloseSpider to abort the dramatiq worker on stale spiders ...

Integrate `dramatiq-abort` with Redis backend for task abortion and enhance the `CloseSpider` extension to abort jobs with no recent activity. These changes improve task management and error handling for idle spiders in the scraping process. Updated dependencies and project settings accordingly.

2024-12-15 23:01:24 +05:30

..

__init__.py

Add dramatiq-abort support and extend scrapy CloseSpider to abort the dramatiq worker on stale spiders

2024-12-15 23:01:24 +05:30

main.py

Fix health check endpoint due to httpx session

2024-12-15 10:38:24 +05:30

middleware.py

Refactor Dramatiq TaskManager Middleware

2024-12-15 22:58:40 +05:30

scheduler.py

Add Movies/TV TGX spider and scheduling support

2024-12-14 21:40:45 +05:30

task.py

Added Prowlarr feed scraping & Improve Advanced scraping capability for prowlarr, zilean, torrentio & more bugfixes & improvements (#286)

2024-09-16 08:26:57 +05:30