Add option to disable image proxy checker

This commit introduces a new configuration option to disable the image proxy
checker. A new environment variable PIXIVFE_PROXY_CHECK_ENABLED and
corresponding ServerConfig field are added. The main.go file is updated
to conditionally initialise the proxy checker based on this setting.

Documentation is also updated to explain the new configuration options.
This commit is contained in:
perennial
2024-09-24 19:07:06 +10:00
parent 305f574f15
commit ae031dbd8b
4 changed files with 44 additions and 17 deletions
+1
View File
@@ -11,6 +11,7 @@ PIXIVFE_HOST='127.0.0.1'
# PIXIVFE_REQUESTLIMIT=
# PIXIVFE_IMAGEPROXY=
# PIXIVFE_ACCEPTLANGUAGE=
# PIXIVFE_PROXY_CHECK_ENABLED=
# PIXIVFE_PROXY_CHECK_INTERVAL=
# PIXIVFE_TOKEN_LOAD_BALANCING=
# PIXIVFE_REPO_URL=
+2
View File
@@ -58,6 +58,7 @@ type ServerConfig struct {
ProxyServer_staging string `env:"PIXIVFE_IMAGEPROXY,overwrite"`
ProxyServer url.URL // proxy server URL, may or may not contain authority part of the URL
ProxyCheckEnabled bool `env:"PIXIVFE_PROXY_CHECK_ENABLED,overwrite"`
ProxyCheckInterval time.Duration `env:"PIXIVFE_PROXY_CHECK_INTERVAL,overwrite"`
// Development options
@@ -119,6 +120,7 @@ func (s *ServerConfig) LoadConfig() error {
s.AcceptLanguage = "en-US,en;q=0.5"
s.ProxyServer_staging = BuiltinProxyUrl
s.ProxyCheckEnabled = true
s.ProxyCheckInterval = 8 * time.Hour
s.TokenLoadBalancing = "round-robin"
s.MaxRetries = 5
+28 -10
View File
@@ -87,16 +87,6 @@ See [hosting an image proxy server](image-proxy-server.md) or the [list of publi
The value of the `Accept-Language` header used for requests to Pixiv's API. Change this to modify the response language.
## `PIXIVFE_PROXY_CHECK_INTERVAL`
**Required**: No
**Default:** `8h`
The interval in minutes between proxy checks. Defaults to 8 hours if not set.
Please specify this value in Go's `time.Duration` notation, e.g. `2h3m5s`.
You can disable this by setting the value to 0. Then, proxies will only be checked once at server initialization.
## `PIXIVFE_TOKEN_LOAD_BALANCING`
**Required**: No
@@ -113,6 +103,34 @@ Valid options:
This option is useful when you have multiple Pixiv accounts and want to distribute the load across them, reducing the risk of rate limiting for individual accounts by the Pixiv API.
## Image proxy checker configuration
PixivFE includes a [image proxy checker](https://codeberg.org/VnPower/PixivFE/src/branch/v2/server/proxy_checker/proxy_checker.go) that periodically tests the pre-defined list of image proxy servers to determine which ones are working. It maintains an updated list of functional proxies that can be used to make image requests to Pixiv.
The following variables control the behavior of the proxy checker.
### `PIXIVFE_PROXY_CHECK_ENABLED`
**Required**: No
**Default:** `true`
Controls whether the image proxy checker is enabled. Set to `false` to completely disable proxy checking.
When disabled, PixivFE will not perform any checks on the image proxy servers, which can be useful in environments where this behavior is not needed or causes issues.
### `PIXIVFE_PROXY_CHECK_INTERVAL`
**Required**: No
**Default:** `8h`
The interval between proxy checks. Defaults to 8 hours if not set.
Please specify this value in Go's [`time.Duration`](https://pkg.go.dev/time#ParseDuration) notation, e.g. `2h3m5s`.
You can disable periodic checks by setting the value to `0`. Then, proxies will only be checked once at server initialization.
## Exponential backoff configuration
PixivFE implements exponential backoff for API requests and token management to handle failures gracefully and manage rate limiting. The following environment variables can be used to configure this behavior, fine-tuning the exponential backoff behavior for both API requests and token management. If not set, the default values will be used.
+13 -7
View File
@@ -25,14 +25,20 @@ func main() {
audit.Init(config.GlobalConfig.InDevelopment)
template.Init(config.GlobalConfig.InDevelopment, "assets/views")
// Initialize and start the proxy checker
defer proxy_checker.StopProxyChecker()
firstCheckDone := proxy_checker.InitializeProxyChecker()
// Conditionally initialize and start the proxy checker
if config.GlobalConfig.ProxyCheckEnabled {
defer proxy_checker.StopProxyChecker()
firstCheckDone := proxy_checker.InitializeProxyChecker()
// Wait for the first proxy check to complete
log.Println("Waiting for initial proxy check to complete...")
<-firstCheckDone
log.Println("Initial proxy check completed. Starting server...")
// Wait for the first proxy check to complete
log.Println("Waiting for initial proxy check to complete...")
<-firstCheckDone
log.Println("Initial proxy check completed.")
} else {
log.Println("Skipping proxy checker initialization.")
}
log.Println("Starting server...")
handlers.InitializeRateLimiter()