Merge branch 'master' into patch-1

README.md

@@ -30,7 +30,7 @@ When working on this repo, it is advised that you review your changes locally be
 
 Please make sure you fork the repo and change the clone URL in the example below for your fork:
 
-- Linux Mint / Ubuntu 18.04 LTS / 19.10:
+- Linux Mint / Ubuntu 18.04 LTS / 19.10 / 20.04 LTS:
     - Preparations (only required once):
 
     ```bash

docs/ftldns/compile.md

@@ -19,12 +19,12 @@ sudo dnf install gcc gmp-devel gmp-static m4
 
 ---
 
-You'll also need to compile `nettle` as *FTL*DNS uses `libnettle` for handling DNSSEC. Compile and install a recent version of `nettle` (we recommend 3.5):
+You'll also need to compile `nettle` as *FTL*DNS uses `libnettle` for handling DNSSEC. Compile and install a recent version of `nettle` (we tested and recommend 3.6):
 
 ```bash
-wget https://ftp.gnu.org/gnu/nettle/nettle-3.5.tar.gz
-tar -xvzf nettle-3.5.tar.gz
-cd nettle-3.5
+wget https://ftp.gnu.org/gnu/nettle/nettle-3.6.tar.gz
+tar -xvzf nettle-3.6.tar.gz
+cd nettle-3.6
 ./configure
 make
 sudo make install
@@ -44,7 +44,7 @@ If you want to build another branch and not `master`, use checkout to get to thi
 *FTL*DNS can now be compiled and installed:
 
 ```bash
-make -j 4
+make -j $(nproc)
 sudo make install
 ```
 

docs/ftldns/dns-resolver.md

@@ -16,13 +16,13 @@ We place hooks in a lot of places in the resolver that branch out into `FTL` cod
 
 #### Remove limit on maximum cache size
 
-Users can configure the size of the resolver's name cache. The default is 150 names. Setting the cache size to zero disables caching. We think users should be allowed to set the cache size to any value they find appropriate. However, `dnsmasq`'s source code contains a condition that limits the maximum size of the cache to 10,000 names. We removed this hard-coded upper limit in [option.c](https://github.com/pi-hole/FTL/commit/ea3309e7b1991f50d40555b9a18f39894c237b29#diff-733116077302620357dcd8252f41449dR2582R258) and submitted a patch to remove this hard-coded limit in the upstream version of `dnsmasq`.
+Users can configure the size of the resolver's name cache. The default is 150 names. Setting the cache size to zero disables caching. We think users should be allowed to set the cache size to any value they find appropriate. However, `dnsmasq`'s source code contains a condition that limits the maximum size of the cache to 10,000 names. We removed this hard-coded upper limit in and submitted a patch to remove this hard-coded limit in the upstream version of `dnsmasq`. It was accepted for `dnsmasq v2.81`.
 
 #### Improve detection algorithm for determining the "best" forward destination
 
 The DNS forward destination determination algorithm in *FTL*DNS's is modified to be much less restrictive than the original algorithm in `dnsmasq`. We keep using the fastest responding server now for 1000 queries or 10 minutes (whatever happens earlier) instead of 50 queries or 10 seconds (default values in `dnsmasq`).
 We keep the exceptions, i.e., we try all possible forward destinations if `SERVFAIL` or `REFUSED` is received or if a timeout occurs.
 Overall, this change has proven to greatly reduce the number of actually performed queries in typical Pi-hole environments. It may even be understood as being preferential in terms of privacy (as we send queries much less often to all servers).
-This has been implemented in commit [d1c163e](https://github.com/pi-hole/FTL/commit/d1c163e499a5cd9f311610e9da1e9365bbf81e89) on the `FTLDNS` branch.
+This has been implemented in commit [d1c163e](https://github.com/pi-hole/FTL/commit/d1c163e499a5cd9f311610e9da1e9365bbf81e89).
 
 {!abbreviations.md!}

docs/ftldns/in-depth.md

@@ -23,10 +23,6 @@ Command-line arguments can be arbitrarily combined, e.g. `pihole-FTL debug test`
 - `/var/run/pihole-FTL.port` file containing port on which `FTL` is listening
 - `/var/run/pihole/FTL.sock` Unix socket
 
-## Domain lists format
-
-Since Pi-hole v4.0, we use a simpler domain list format for the two important blocklist files `gravity.list` and `black.list`. In contrast to the traditional HOSTS format (which caused a lot of overhead), the domain list format is the minimal possible solution for saving memory while still using plain text lists for your convenience. When *FTL*DNS imports these two files, they are walked by our improved list parser speeding up the loading of blocklists significantly. Regardless of which blocking mode (`IP` or `NXDOMAIN`) is selected, *FTL*DNS will always load the lists into its internal hashed cache to be able to determine the blocking status within a few milliseconds, even when you're using huge blocking lists on low-end devices. With everything we do, we design *FTL*DNS for maximum efficiency also on low-performance devices.
-
 ## Linux capabilities
 
 Capabilities (POSIX 1003.1e, [capabilities(7)](http://man7.org/linux/man-pages/man7/capabilities.7.html)) provide fine-grained control over superuser permissions, allowing the use of the `root` user to be avoided.

docs/ftldns/privacylevels.md

@@ -39,11 +39,4 @@ This setting disables
 - Query Log
 - Long-term database logging
 
-### Level 4 - disabled statistics (v4.1+)
-
-Disables all statistics processing. Even the query counters will not be available.
-Additionally, you can disable logging to the file `/var/log/pihole.log` using `sudo pihole logging off`.
-
-Note that - due to the disabled query processing - regex blocking is **not** available on level 4.
-
 {!abbreviations.md!}

docs/guides/caddy-configuration.md

@@ -2,6 +2,8 @@
 
 If you'd like to use [Caddy](https://caddyserver.com/) as your main web server with Pi-hole, you'll need to make a few changes.
 
+> Note: This guide only deals with setting up caddy as a reverse-proxy and not as a replacement for lighttpd (Although caddy is capable of doing so, but it is beyond the scope of this guide).
+
 ## Modifying lighttpd configuration
 
 First, change the listen port in this file: `/etc/lighttpd/lighttpd.conf:`
@@ -24,9 +26,47 @@ or
 sudo service lighttpd restart
 ```
 
+## Installing Caddy
+
+Follow the instructions on the [Caddy download](https://caddyserver.com/docs/download) documentation page.
+
 ## Setting up your Caddyfile
 
-Now set up a "virtual host" in your Caddyfile (default `/etc/caddy/Caddyfile`). There are many options you can add, but at a minimum, you need to make a "default" host by binding `0.0.0.0:80`. This will accept requests for any interface.
+Now set up a "virtual host" in your Caddyfile (default `/etc/caddy/Caddyfile`). There are many options you can add, but at a minimum, you need to make a "default" host by binding `:80`. This will accept requests for any interface on port `80`.
+
+### Caddyfile (for Caddy v2)
+
+```
+http://pi.hole {
+  reverse_proxy localhost:1080
+}
+```
+
+- If you'd like to enable HTTPS on your site, make sure your server is reachable via your domain name (ex: myawesomesite.com) and is pointing to the right IP address.
+- Additionally you need to open ports :80 and :443 (Apart from the one's required specifically for pi-hole) for your server before setting up HTTPS.
+
+The following configuration will automatically fetch and setup HTTPS for your domain using Lets-Encrypt
+
+```
+myawesomesite.com {
+  reverse_proxy localhost:1080
+}
+```
+
+Additionally you can make pihole reacheable via a subdomain and optionally can you enable Zstandard and Gzip compression as follows:
+
+```
+pihole.myawesomesite.com {
+  reverse_proxy localhost:1080
+  encode zstd gzip
+}
+```
+
+Finally, run `sudo systemctl caddy reload` to reload Caddy with the new configuration.
+
+### Caddyfile (for Caddy v1)
+
+Caddy v1 is no longer actively supported, but the following is a config example if you're running an old installation.
 
 ```
 blackhole:80, pi.hole:80, 0.0.0.0:80 {
@@ -44,19 +84,17 @@ blackhole:80, pi.hole:80, 0.0.0.0:80 {
 }
 ```
 
-In this case, I've chosen to also add blackhole and pi.hole as valid names to open the admin page with.
-
-Finally, restart your Caddy server: `sudo service caddy restart`
+In this example, `blackhole` and `pi.hole` are added as valid names with which to open the admin page.
 
 ## Verifying your setup
 
-First, make sure that any other sites you're serving from caddy are still functioning. For example, if you have a block for `myawesomesite.com:80` in your Caddyfile, open up a browser to `http://myawesomesite.com` and verify it still loads.
+First, make sure that any other sites you're serving from caddy are still functioning. For example, if you have a block for `myawesomesite.com:80` or similar in your Caddyfile, open up a browser to `http://myawesomesite.com` (or `https://` if you have enabled it) and verify it still loads.
 
 Next, verify you can load the admin page. Open up `http://pi.hole/admin` (or use the IP address of your server) and verify that you can access the admin page.
 
 Finally, verify that requests for ads are being black holed:
 
-```bash
+```console
 $ curl -H "Host: badhost" pi.hole/
 <html>
 <head>
@@ -75,3 +113,7 @@ Lastly, ensure that requests for JavaScript files from advertisement domains are
 curl -H "Host: badhost" pi.hole/malicious.js
 var x = "Pi-hole: A black hole for Internet advertisements."
 ```
+
+For more information visit Caddy's documentation [website](https://caddyserver.com/docs/).
+
+For usage questions or support, visit the [Caddy Community forums](https://caddy.community/).