* fix: add secret_str handling in middleware when using encoded_user_data
* fix: fallback only
* fix: get directly
* fix: use request.scope
* fix: await
* style: format
This change introduces API call with 'encoded_user_data' Header value. This simplifies the API call from other scraper Addons instead of encrypting and storing the encrypted userdata
Introduced sorting and ordering functionality for MDBList, allowing users to customize the display order of items. Updated backend, frontend, and database schemas to support these options, including caching, API requests, and user interface enhancements. Adjusted styles and badges for better presentation of sorting information.
Reworked the caching logic to handle a missing cache key for "events" type gracefully. Integrated a new DLHD schedule service for improved event metadata and stream handling, leveraging fuzzy matching and IPTV-org data retrieval for enhanced TV channel mappings. Updated scraper configuration, enhanced genre handling, and added support for dynamic event descriptions.
Store UserData in redis if compress + encryption data length is more than 1000, or else directly setup in the url secret string. Replaces legacy encryption and decryption methods with an optimized `CryptoUtils` class supporting compression, improved encryption, and Redis-based storage.
Integrate `dramatiq-abort` with Redis backend for task abortion and enhance the `CloseSpider` extension to abort jobs with no recent activity. These changes improve task management and error handling for idle spiders in the scraping process. Updated dependencies and project settings accordingly.
Introduced `TaskInfo` dataclass and caching mechanism for task metadata to improve performance. Refactored Redis operations into a dedicated helper method with error handling. Enhanced code readability and reduced redundancy by modularizing task key generation and processing logic.
Introduced mechanisms to handle preconfigured sensitive fields with masking and reset options in the UI. Extended User functionality to reuse existing encrypted configuration for sensitive fields when editing. Improved styling and user experience for managing credentials and secure configurations.
Introduced `MoviesTVTgxSpider` for parsing TGx movie/TV torrents and added relevant pipelines for processing. Enhanced scheduler, constants, and configurations to support the new functionality. Made updates to improve torrent detail parsing and handle dynamic content loading using Playwright.
Replaced custom proxy implementation and `curl-cffi` with unified `httpx` and `aiohttp` proxy support using `requests_proxy_url` config from settings. Simplified and standardized HTTP client instantiations across various modules for improved maintainability and consistency.
Enhanced the stream sorting functionality to support sorting direction (ascending or descending) based on user preferences in both the backend and UI. Introduced 'SortingOption' in the schema to capture sorting criteria with direction, and updated the sorting logic to accommodate these changes. Additionally, modified the user interface to include buttons for toggling sort directions, providing a more intuitive sorting experience for users.
Introduce a system for toggling streaming providers, allowing specific providers to be disabled via configuration. Enhance error handling in user data decryption by adding validation checks and returning precise error messages in JSON responses. These changes improve flexibility for administrators and provide better feedback to users.
Moved Redis client setup to a new module `redis_database` and updated all references across the codebase. Also added a new configuration setting for redis_max_connections for better resource management.
Consolidate error handling logic across middleware components to improve maintainability. Adjust the order of middleware addition in the app for improved logging and user data access.
Removed git revision from the version string in the API response. Added a new field for branding description in settings and integrated it in the home template. Introduced VERSION argument and environment variable in the Dockerfile.
* Add configuration mode switch for user customization
Introduce a 'Configuration Mode' switch to toggle between 'Newbie' and 'Pro' modes, allowing users to select between simplified and advanced configurations. Updated CSS and JS to handle the display of specific settings and sections based on the selected mode, enhancing user experience and customization options.
* Update default language filters and sorting priority.
* Refactor stream filtering and catalog configuration.
Removed redundant catalog selection logic and refactored stream filtering to streamline the code. Updated configuration scripts and UI to align with new functionality, leading to cleaner and more maintainable code.
Enhanced the /health endpoint to perform an HTTP HEAD request to Google, returning the status code and response time. This change provides a more accurate health status and logs errors if the check fails.
* cleanup lifespan vars
* Show API process time in logs
* store the cache validation of full URL for live sports event
* Add support for filter only show cached streams
* Update default configs
* properly handle prowlarr exception on MaxProcessLimitReached
* Add redis lock mechanism to prevent concurrent live scrapers from multiple instances of stremio addon installation users
* Refactor ZileanScraper to use parallel requests for searching and filtering streams with new endpoints
* Fix DLHD scraping & enable DLHD without MediaFlow
* Switch to httpx for async HTTP requests
* Implement caching for PikPak token to reduce login error
* Refactor torrent cleanup logic
* Add inactivity monitor extension to close idle spiders
Introduced `InactivityMonitor` to automatically close spiders that remain inactive for a specified time. This extension checks activity at regular intervals and uses configurable settings for check intervals and inactivity timeouts. If no items are scraped within the timeout period, the spider is closed to free resources.
* Handle TypeError in dynamic sorting of streams
* Refactor torrent info scraper to support pre-processing.
Introduced a pre-processing function mapping to handle specific indexer requirements before parsing the HTML. Added a custom pre-processing function for "TheRARBG" to handle URL adjustments, improving modularity and readability in the `get_torrent_info` function.
* Refine error logging and fix hash key in spider
* Fix Prowlarr not stop on max process limit
* #315: Integrate ScrapeOps logging into all scrapers & spiders
Added ScrapeOps logging to Zilean, Torrentio, Prowlarr, and Prowlarr Feed scrapers to enhance request tracking and error handling. Configured ScrapeOps API key in settings and updated Pipfile/Pipfile.lock with scrapeops-python-requests and scrapeops-scrapy dependencies.
* Refactor scraper cache status handler
* verify torrent before parsing on prowlarr & prioritize magnet on badass_torrents
* Add support for provide locally hosted mediaflow proxy public address and reduce the time leg on private ip address checking
* handle RD exception cases
* do not setup scrapeops when api key is none
* Enhanced dynamic sorting of torrent streams
Revised the dynamic_sort_key function to handle different key types more efficiently with match-case. Simplified error handling and improved logging to capture sorting data in the case of exceptions.
* add missing last update date for metadata
* update domain for nowmesports
Viren070
mhdzumair