From 968e0cf13709ad4fadb1fcd1d18d73d55e3fd096 Mon Sep 17 00:00:00 2001 From: DL6ER Date: Fri, 28 Sep 2018 12:11:43 +0200 Subject: [PATCH 1/9] Add backup instructions for long-term database. As we utilize the online backup functionality of SQLite3, a backup can be generated without having to stop/restart pihole-FTL, i.e., without interruption of DNS service. Signed-off-by: DL6ER --- docs/ftldns/database.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/ftldns/database.md b/docs/ftldns/database.md index 4c21efe..3cbb214 100644 --- a/docs/ftldns/database.md +++ b/docs/ftldns/database.md @@ -13,6 +13,12 @@ sudo service pihole-FTL start ``` Note that DNS resolution will not be available as long as `pihole-FTL` is stopped. +The database can be backed up while FTL is running when using the SQLite3 Online backup method, e.g., +``` +sqlite3 /etc/pihole/pihole-FTL.db ".backup /home/pi/pihole-FTL.db.backup" +``` +will create `/home/pi/pihole-FTL.db.backup` which is a copy of your long-term database. + Another way of controlling the size of the long-term database is setting a maximum age for log queries to keep using the config parameter [`MAXDBDAYS`](configfile.md#maxdbdays). It defaults to 365 days, i.e. queries that are older than one year get periodically removed to limit the growth of the long-term database file. The config parameter [`DBIMPORT`](configfile.md#dbimport) controls whether `FTL` loads information from the database on startup. It need to do this to populate the internal datastructure with the most recent history. However, as importing from the database on disk can delay FTL on very large deploys, it can be disabled using this option. From 3d486e4705005598c29754273f63d24fcb6a8f9a Mon Sep 17 00:00:00 2001 From: Jasper Danan Date: Wed, 3 Oct 2018 18:59:16 +1000 Subject: [PATCH 2/9] Ported Whitelisting and Blacklisting Guide Ported this wiki page (https://github.com/pi-hole/pi-hole/wiki/Whitelisting-and-Blacklisting) to Guides section under "Editing Whitelisting and Blacklisting" Signed-off-by: Jasper Danan --- docs/guides/whitelist-blacklist.md | 85 ++++++++++++++++++++++++++++++ mkdocs.yml | 1 + 2 files changed, 86 insertions(+) create mode 100644 docs/guides/whitelist-blacklist.md diff --git a/docs/guides/whitelist-blacklist.md b/docs/guides/whitelist-blacklist.md new file mode 100644 index 0000000..3d9ac13 --- /dev/null +++ b/docs/guides/whitelist-blacklist.md @@ -0,0 +1,85 @@ +### What to Whitelist or Blacklist + +[This extension for Google Chrome](https://chrome.google.com/webstore/detail/whitelist-assistant-by-dn/fdmpekabnlekabjlimjkfmdjajnddgpc) can help you in finding out which domains you need to whitelist. + + +### How to Whitelist or Blacklist + +There are scripts to aid users in adding or removing domains to the whitelist or blacklist. + +The scripts will first parse `whitelist.txt` or `blacklist.txt` for any changes, and if any additions or deletions are detected, it will reload `dnsmasq` so that they are effective immediately. + +Each script accepts the following parameters: + + + + + + + + + + + + + + + + + + + + + + + + +
[domain]Fully qualified domain name you wish to add or remove. You can pass any number of domains.
-dRemoval mode. Domains will be removed from the list, rather than added
-nrUpdate blacklist without refreshing dnsmasq
-fForce delete cached blocklist content
-qQuiet mode. Console output is minimal. Useful for calling from another script (see gravity.sh)
+ +Domains passed are parsed by the script to ensure they are valid domains. If a domain is invalid it will be ignored. + + +##### Example `pihole -w` usages + + + + + + + + + + + + + + + + +
pihole -w domain1 [domain2...]Attempt to add one or more domains to the whitelist and reload dnsmasq.
pihole -w -nr domain1 [domain2...]Attempt to add one or more domains to the whitelist, but do not reload dnsmasq.
pihole -w -f domain1 [domain2...]Attempt to add one or more domains to the whitelist and force dnsmasq to reload
+ +To remove domains from the whitelist: +Add `-d` as an additional argument (e.g `pihole -w -d domain1 [domain2...]`) + + +##### Example `pihole -b` usages + + + + + + + + + + + + + + + + +
pihole -b domain1 [domain2...]Attempt to add one or more domains to the blacklist and reload dnsmasq.
pihole -b -nr domain1 [domain2...]Attempt to add one or more domains to the blacklist, but do not reload dnsmasq.
pihole -b -f domain1 [domain2...]Attempt to add one or more domains to the blacklist and force dnsmasq to reload
+ +To remove domains from the blacklist: +Add `-d` as an additional argument (e.g `pihole -b -d domain1 [domain2...]`) \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index 445d1f7..2b2b03a 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -63,6 +63,7 @@ pages: - 'Guides': - 'Pi-hole as All-Around DNS Solution': guides/unbound.md - 'Configuring DNS-Over-HTTPS on Pi-hole': guides/dns-over-https.md + - 'Editing Whitelisting and Blacklisting': guides/whitelist-blacklist.md - 'Pi-hole and OpenVPN Server': - 'Overview': 'guides/vpn/overview.md' - 'Installation': 'guides/vpn/installation.md' From 8410f36625f5019943ed0e184724bbab0f70115d Mon Sep 17 00:00:00 2001 From: Jasper Danan Date: Sat, 6 Oct 2018 14:50:57 +1000 Subject: [PATCH 3/9] Signed-off-by: Jasper Danan Changed title from 'Editing Whitelisting and Blacklisting' to 'Editing Whitelist and Blacklist' --- mkdocs.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mkdocs.yml b/mkdocs.yml index 2b2b03a..8605415 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -63,7 +63,7 @@ pages: - 'Guides': - 'Pi-hole as All-Around DNS Solution': guides/unbound.md - 'Configuring DNS-Over-HTTPS on Pi-hole': guides/dns-over-https.md - - 'Editing Whitelisting and Blacklisting': guides/whitelist-blacklist.md + - 'Editing Whitelist and Blacklist': guides/whitelist-blacklist.md - 'Pi-hole and OpenVPN Server': - 'Overview': 'guides/vpn/overview.md' - 'Installation': 'guides/vpn/installation.md' From d3386a2472aff813fa99c93446f95524c72cc5b8 Mon Sep 17 00:00:00 2001 From: DL6ER Date: Sun, 7 Oct 2018 18:07:45 +0200 Subject: [PATCH 4/9] Section the split and backup methods to avoid confusion. Signed-off-by: DL6ER --- docs/ftldns/database.md | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/docs/ftldns/database.md b/docs/ftldns/database.md index 3cbb214..2210646 100644 --- a/docs/ftldns/database.md +++ b/docs/ftldns/database.md @@ -4,6 +4,13 @@ We update the database file periodically and on exit of *FTL*DNS (triggered e.g. The location of the database can be configures by the config parameter [`DBFILE`](configfile.md#dbfile). It defaults to `/etc/pihole/pihole-FTL.db`. If the given file does not exist, *FTL*DNS will create a new (empty) database file. +Another way of controlling the size of the long-term database is setting a maximum age for log queries to keep using the config parameter [`MAXDBDAYS`](configfile.md#maxdbdays). It defaults to 365 days, i.e. queries that are older than one year get periodically removed to limit the growth of the long-term database file. + +The config parameter [`DBIMPORT`](configfile.md#dbimport) controls whether `FTL` loads information from the database on startup. It need to do this to populate the internal datastructure with the most recent history. However, as importing from the database on disk can delay FTL on very large deploys, it can be disabled using this option. + +--- +### Split database + You can split your long-term database by periodically rotating the database file (do this only when `pihole-FTL` is *not* running). The individual database contents can easily be merged when required. This could be implemented by running a monthly `cron` job such as: ``` @@ -13,15 +20,14 @@ sudo service pihole-FTL start ``` Note that DNS resolution will not be available as long as `pihole-FTL` is stopped. +### Backup database + The database can be backed up while FTL is running when using the SQLite3 Online backup method, e.g., ``` sqlite3 /etc/pihole/pihole-FTL.db ".backup /home/pi/pihole-FTL.db.backup" ``` will create `/home/pi/pihole-FTL.db.backup` which is a copy of your long-term database. -Another way of controlling the size of the long-term database is setting a maximum age for log queries to keep using the config parameter [`MAXDBDAYS`](configfile.md#maxdbdays). It defaults to 365 days, i.e. queries that are older than one year get periodically removed to limit the growth of the long-term database file. - -The config parameter [`DBIMPORT`](configfile.md#dbimport) controls whether `FTL` loads information from the database on startup. It need to do this to populate the internal datastructure with the most recent history. However, as importing from the database on disk can delay FTL on very large deploys, it can be disabled using this option. --- From 2a88ac016b18e1e36bae875e2a491f43aef37c85 Mon Sep 17 00:00:00 2001 From: DL6ER Date: Sun, 7 Oct 2018 18:10:34 +0200 Subject: [PATCH 5/9] Remove "for v4.0+" part in the header as this is now the regular version of Pi-hole. Signed-off-by: DL6ER --- mkdocs.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mkdocs.yml b/mkdocs.yml index 445d1f7..b684295 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -1,4 +1,4 @@ -site_name: "Pi-hole documentation for v4.0+" +site_name: "Pi-hole documentation" site_url: https://docs.pi-hole.net repo_url: "https://github.com/pi-hole/pi-hole/" edit_uri: "" From 221caebfac6d8e53d97d70946065271684c01cb4 Mon Sep 17 00:00:00 2001 From: root Date: Tue, 9 Oct 2018 14:40:47 +0000 Subject: [PATCH 6/9] Added NGINX Configuration page Signed-off-by: root --- docs/guides/nginx-configuration.md | 88 ++++++++++++++++++++++++++++++ mkdocs.yml | 1 + 2 files changed, 89 insertions(+) create mode 100644 docs/guides/nginx-configuration.md diff --git a/docs/guides/nginx-configuration.md b/docs/guides/nginx-configuration.md new file mode 100644 index 0000000..ec8adfa --- /dev/null +++ b/docs/guides/nginx-configuration.md @@ -0,0 +1,88 @@ +### Notes & Warnings +- If you're using php5, change all instances of `php7.0-fpm` to `php5-fpm` and change `/run/php/php7.0-fpm.sock` to `/var/run/php5-fpm.sock` + +### Basic requirements +1. Stop default lighttpd +`service lighttpd stop` +2. Install necessary packages +`apt-get -y install nginx php7.0-fpm php7.0-zip apache2-utils` +3. Disable lighttpd at startup +`systemctl disable lighttpd` +4. Enable php7.0-fpm at startup +`systemctl enable php7.0-fpm` +5. Enable nginx at startup +`systemctl enable nginx` +6. Edit `/etc/nginx/sites-available/default` to: + +``` +server { + listen 80 default_server; + listen [::]:80 default_server; + + root /var/www/html; + server_name _; + autoindex off; + + index pihole/index.php index.php index.html index.htm; + + location / { + expires max; + try_files $uri $uri/ =404; + } + + location ~ \.php$ { + include snippets/fastcgi-php.conf; + fastcgi_pass unix:/run/php/php7.0-fpm.sock; + auth_basic "Restricted"; #For Basic Auth + auth_basic_user_file /etc/nginx/.htpasswd; #For Basic Auth + } + + location /*.js { + index pihole/index.js; + auth_basic "Restricted"; #For Basic Auth + auth_basic_user_file /etc/nginx/.htpasswd; #For Basic Auth + } + + location /admin { + root /var/www/html; + index index.php index.html index.htm; + auth_basic "Restricted"; #For Basic Auth + auth_basic_user_file /etc/nginx/.htpasswd; #For Basic Auth + } + + location ~ /\.ht { + deny all; + } +} +``` + +7.1. Create username for authentication for the admin - we don't want other people in our network change our black and whitelist ;) +`htpasswd -c /etc/nginx/.htpasswd exampleuser` + +7.2. Change ownership of html directory to nginx user + `chown -R www-data:www-data /var/www/html` + +7.3. Make sure html directory is writable + `chmod -R 755 /var/www/html` + +7.4. Start php7.0-fpm daemon + `service php7.0-fpm start` + +7.5. Start nginx webserver + `service nginx start` + +### Optional configuration +- If you want to use your custom domain to access admin page (e.g.: `http://mydomain.internal/admin/settings.php` instead of `http://pi.hole/admin/settings.php`), make sure `mydomain.internal` is assigned to `server_name` in `/etc/nginx/sites-available/default`. E.g.: `server_name mydomain.internal;` + +- If you want to use block page for any blocked domain subpage (aka Nginx 404), add this to Pihole server block in your Nginx configuration file: +``` +error_page 404 /pihole/index.php +``` +- When using nginx to serve pihole, Let's Encrypt can be used to directly configure nginx. Make sure to use your hostname instead of _ in `server_name _;` line above. +``` +add-apt-repository ppa:certbot/certbot +apt-get install certbot python-certbot-nginx + +certbot --nginx -m "$email" -d "$domain" -n --agree-tos --no-eff-email +``` + diff --git a/mkdocs.yml b/mkdocs.yml index b684295..afc91b9 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -63,6 +63,7 @@ pages: - 'Guides': - 'Pi-hole as All-Around DNS Solution': guides/unbound.md - 'Configuring DNS-Over-HTTPS on Pi-hole': guides/dns-over-https.md + - 'Configuring NGINX for PiHole': guides/nginx-configuration.md - 'Pi-hole and OpenVPN Server': - 'Overview': 'guides/vpn/overview.md' - 'Installation': 'guides/vpn/installation.md' From e113df3507292224376313373927900742e17b8a Mon Sep 17 00:00:00 2001 From: root Date: Tue, 9 Oct 2018 16:02:40 +0000 Subject: [PATCH 7/9] Added Caddy configuration from Wiki Signed-off-by: root --- docs/guides/caddy-configuration.md | 63 ++++++++++++++++++++++++++++++ mkdocs.yml | 1 + 2 files changed, 64 insertions(+) create mode 100644 docs/guides/caddy-configuration.md diff --git a/docs/guides/caddy-configuration.md b/docs/guides/caddy-configuration.md new file mode 100644 index 0000000..567710a --- /dev/null +++ b/docs/guides/caddy-configuration.md @@ -0,0 +1,63 @@ +# Caddy Webserver Frontend + +If you'd like to use Caddy as your main web server with PiHole, you'll need to make a few changes. + +## Modifying lighttpd configuration +First, change the listen port in this file: +`/etc/lighttpd/lighttpd.conf:` + +``` +server.port = 1080 +``` + +In this case I chose 1080 somewhat at random. Use whatever feels right to you. + +Next, restart the lighttpd server with either of these commands: +- `sudo systemctl restart lighttpd` +- `sudo service lighttpd restart.` + +## Setting up your Caddyfile +Now we need to set up a "virtual host" in our Caddyfile (default `/etc/caddy/Caddyfile`). There are many more options you can add, but at bare minimum you need to make a "default" host by binding `0.0.0.0:80` which will accept requests for any host. +```YAML +blackhole:80, pi.hole:80, 0.0.0.0:80 { + root /var/www/html/pihole + log /var/log/caddy/blackhole.log + + rewrite { + ext js + to index.js + } + + proxy / localhost:1080 { + transparent + } +} +``` +In this case I've chosen to also add blackhole and pi.hole as valid names to open the admin page with. + +Finally, restart your Caddy server: +- `sudo service caddy restart` + +## Verifying your set up +First, make sure that any other sites you're serving from caddy are still functioning. For example, if you have a block for `myawesomesite.com:80` in your Caddyfile, open up a browser to `http://myawesomesite.com` and verify it still loads. + +Next, verify you can load the admin page. Open up `http://pi.hole/admin` (or use the IP address of your server) and verify that you can access the admin page. + +Finally, verify that requests for ads are being black holed: +```BASH +$ curl -H "Host: badhost" pi.hole/ + + + + + + + +``` +Replace the URL `pi.hole` with the IP address or alternate DNS name you're using if necessary. + +Lastly, ensure that requests for JavaScript files from advertisement domains are being served properly: +```BASH +curl -H "Host: badhost" pi.hole/malicious.js +var x = "Pi-hole: A black hole for Internet advertisements." +``` diff --git a/mkdocs.yml b/mkdocs.yml index b684295..915d10b 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -63,6 +63,7 @@ pages: - 'Guides': - 'Pi-hole as All-Around DNS Solution': guides/unbound.md - 'Configuring DNS-Over-HTTPS on Pi-hole': guides/dns-over-https.md + - 'Configuring Caddy for Pi-Hole': guides/caddy-configuration.md - 'Pi-hole and OpenVPN Server': - 'Overview': 'guides/vpn/overview.md' - 'Installation': 'guides/vpn/installation.md' From 4f6deca45e78c9af3165773feb3c34b5d479eed7 Mon Sep 17 00:00:00 2001 From: root Date: Thu, 11 Oct 2018 13:57:02 +0000 Subject: [PATCH 8/9] Fixed Pi-hole references Signed-off-by: root --- docs/guides/caddy-configuration.md | 3 +-- mkdocs.yml | 2 +- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/guides/caddy-configuration.md b/docs/guides/caddy-configuration.md index 567710a..ee997e2 100644 --- a/docs/guides/caddy-configuration.md +++ b/docs/guides/caddy-configuration.md @@ -1,6 +1,5 @@ -# Caddy Webserver Frontend -If you'd like to use Caddy as your main web server with PiHole, you'll need to make a few changes. +If you'd like to use Caddy as your main web server with Pi-hole, you'll need to make a few changes. ## Modifying lighttpd configuration First, change the listen port in this file: diff --git a/mkdocs.yml b/mkdocs.yml index 915d10b..086d474 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -63,7 +63,7 @@ pages: - 'Guides': - 'Pi-hole as All-Around DNS Solution': guides/unbound.md - 'Configuring DNS-Over-HTTPS on Pi-hole': guides/dns-over-https.md - - 'Configuring Caddy for Pi-Hole': guides/caddy-configuration.md + - 'Configuring Caddy for Pi-hole': guides/caddy-configuration.md - 'Pi-hole and OpenVPN Server': - 'Overview': 'guides/vpn/overview.md' - 'Installation': 'guides/vpn/installation.md' From 189aaafe746207ccd3e38051219d496786209228 Mon Sep 17 00:00:00 2001 From: root Date: Thu, 11 Oct 2018 14:04:20 +0000 Subject: [PATCH 9/9] Fixed Pi-hole references, and numbers Signed-off-by: root --- docs/guides/nginx-configuration.md | 14 +++++++------- mkdocs.yml | 2 +- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/guides/nginx-configuration.md b/docs/guides/nginx-configuration.md index ec8adfa..92821c6 100644 --- a/docs/guides/nginx-configuration.md +++ b/docs/guides/nginx-configuration.md @@ -56,29 +56,29 @@ server { } ``` -7.1. Create username for authentication for the admin - we don't want other people in our network change our black and whitelist ;) +7. Create username for authentication for the admin - we don't want other people in our network change our black and whitelist ;) `htpasswd -c /etc/nginx/.htpasswd exampleuser` -7.2. Change ownership of html directory to nginx user +8. Change ownership of html directory to nginx user `chown -R www-data:www-data /var/www/html` -7.3. Make sure html directory is writable +9. Make sure html directory is writable `chmod -R 755 /var/www/html` -7.4. Start php7.0-fpm daemon +10. Start php7.0-fpm daemon `service php7.0-fpm start` -7.5. Start nginx webserver +11. Start nginx webserver `service nginx start` ### Optional configuration - If you want to use your custom domain to access admin page (e.g.: `http://mydomain.internal/admin/settings.php` instead of `http://pi.hole/admin/settings.php`), make sure `mydomain.internal` is assigned to `server_name` in `/etc/nginx/sites-available/default`. E.g.: `server_name mydomain.internal;` -- If you want to use block page for any blocked domain subpage (aka Nginx 404), add this to Pihole server block in your Nginx configuration file: +- If you want to use block page for any blocked domain subpage (aka Nginx 404), add this to Pi-hole server block in your Nginx configuration file: ``` error_page 404 /pihole/index.php ``` -- When using nginx to serve pihole, Let's Encrypt can be used to directly configure nginx. Make sure to use your hostname instead of _ in `server_name _;` line above. +- When using nginx to serve Pi-hole, Let's Encrypt can be used to directly configure nginx. Make sure to use your hostname instead of _ in `server_name _;` line above. ``` add-apt-repository ppa:certbot/certbot apt-get install certbot python-certbot-nginx diff --git a/mkdocs.yml b/mkdocs.yml index afc91b9..50b795e 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -63,7 +63,7 @@ pages: - 'Guides': - 'Pi-hole as All-Around DNS Solution': guides/unbound.md - 'Configuring DNS-Over-HTTPS on Pi-hole': guides/dns-over-https.md - - 'Configuring NGINX for PiHole': guides/nginx-configuration.md + - 'Configuring NGINX for Pi-hole': guides/nginx-configuration.md - 'Pi-hole and OpenVPN Server': - 'Overview': 'guides/vpn/overview.md' - 'Installation': 'guides/vpn/installation.md'