Commit Graph

559 Commits

Author SHA1 Message Date
DL6ER 7359b496ff Move database.maxHistory to webserver.api.maxHistory
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-04-25 22:26:25 +02:00
DL6ER b3ef031a22 Various memory improvements to (a) avoid crash during the final cleanup (don't allow reusing of pointer to memory that may already have been munmapped) , and (b) ensure we have release all memory after exporting to the on-disk database (finalize all prepared statements in queries_to_database())
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-04-18 21:13:48 +02:00
DL6ER fba171f151 Delete only up to 1% of the queries in the database in one go to avoid long blocking times with many queries to be deleted.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-03-13 11:41:14 +01:00
DL6ER d0e3500c4e Even when database.network.parseARPcache is set to false, FTL should be cleaning the network table, adding in clients from its own knowledge (those that sent queries), and add local interfaces
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-03-13 11:10:40 +01:00
DL6ER 44c3fe4ef4 Merge remote-tracking branch 'origin/development' into new/http
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-03-13 10:30:39 +01:00
DL6ER b87ef114ef Merge pull request #1532 from pi-hole/new/adb_style_blocking
Add support for Adblock Plus domain lists
2023-03-11 15:18:29 -05:00
DL6ER 62ebd05496 Apply Pi-hole SQLite3 patches
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-03-11 07:52:57 +01:00
DL6ER 1fe7bb6946 Update embedded SQLite3 engine to version 3.41.1
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-03-11 07:45:25 +01:00
DL6ER da118e87ce Remove last traces of temporarily added benchmarking tools. Also remove the hint about ABP domains, this can easily be checked in gravity
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-03-05 13:56:55 +01:00
DL6ER 06f0e0340e Remove debugging timing output
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-02-26 16:52:23 +01:00
DL6ER ae9b291082 Set abp_domains = 1 during the CI tests.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-02-26 07:19:48 +01:00
DL6ER ab2f652e22 Use property "abp_domains" from info table to decide whether ABP blocking is to be used or not. Also log when FTL enabled ABP-style blocking
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-02-25 22:29:17 +01:00
DL6ER ec82cd3dd4 Add timing for ABP style detection
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-02-25 18:37:40 +01:00
DL6ER 4ea0bf7dea Always update last_disk_db_idx after storing queries on disk as sqlite3_changes() sometimes reports to few rows being inserted. This was a rather tedious debugging...
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-02-25 18:23:13 +01:00
Dan Schaper 6f8ba76077 Update src/database/gravity-db.c
Spespellingllling

Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
2023-02-15 14:01:37 -08:00
Christian König d9c753be27 Fix spellcheck to get things deployed
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-02-15 21:55:47 +01:00
DL6ER 80b5fa008c Fix handling of rare (but possible) gravity database issues such as "list not available"
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-02-15 21:27:12 +01:00
DL6ER 75cd6913ee Do not use a new option but instead automatically detect if ABP-style domains are present in the database. This ensures that this addition comes at no extra costs to any installs using pure HOSTS-style adlists.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-02-15 21:19:31 +01:00
DL6ER 8794b1684d Add ABP format blocking support for gravity. Note that the option needs to be switched on by setting GRAVITY_ABP_STYLE=true in pihole-FTL.conf to avoid running this computationally expensive task on the vast majority of user databases only fed from properly formatted HOSTS lists. Gravity can enable the setting when it detects ABP format automatically.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-02-15 19:48:19 +01:00
DL6ER 49c978a9e8 Add filesystem type in disk usage warning (if available)
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-02-07 19:18:12 +01:00
DL6ER 4ee054b962 Only warn once about disk shortage in case database and log file are on the same device (e.g. both on / )
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-02-06 21:16:23 +01:00
DL6ER d7da3f7996 Improve disk shortage warning to point out more precisely which filesystem is getting full. We do this by mentioning the mountpoint instead of the filename
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-02-06 21:12:47 +01:00
DL6ER 9833fe1af0 Reload gravity database when 'updated' property in table 'info' has changed
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-02-05 16:44:14 +01:00
DL6ER cf82495d6c Create new webtheme() PHP function and make the webtheme an enum
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-02-03 18:12:55 +01:00
DL6ER da6a1ae053 Add {GET,DELETE} /api/info/messages
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-01-30 21:00:08 +01:00
DL6ER 5bfc1fd122 Add /api/search as a batter replacement of the currently existing "pihole -q" command
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-01-29 19:55:12 +01:00
DL6ER 8753a8d69b Add more compiler warnings and fix a few things they pointed out worth improving/being more explicit about. This adds GCC-12 compatibility out of the box.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-01-29 18:12:19 +01:00
DL6ER 48fc06d46b Add POST /api/teleporter to upload and install backed up configuration
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-01-25 21:51:12 +01:00
DL6ER e4383775d1 Add /api/action/gravity which can be used to trigger a run of pihole -g. The output is live streamed using HTTP/1.1 chunked encoding.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-01-24 20:15:45 +01:00
DL6ER 0856153699 Add ability to list allowed values (for enums) in JSON form. Add pretty-printing for TOML and update all help test descriptions from the official Pi-hole documentation
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-01-22 12:01:42 +01:00
DL6ER 999a9735cb Do not store the payload on the stack but allocate heap memory for it. Using dozens of KB for each API connection is dangerous, hundreds of KB large payloads is surely a very bad idea. (rmemeber, the answer to how-much-it-too-much is the rather vague: too much is when the stack overflows)
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-01-09 20:08:39 +01:00
DL6ER f199ac2f08 Automatically migrate the API password hash and the lists of clients and domains to be excluded from setupVars.conf
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-01-08 21:21:38 +01:00
DL6ER e145d20d28 Rewrite the entire config-related code to allow for changing data without having to restart. Hereby, we greatly reduce code duplication in the TOML routines so we won't have to touch tme in the future when adding additional options.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-01-07 18:06:41 +01:00
DL6ER f2d68f20d7 Also verify endpoint structure: Query endpoints from FTL and check if all properties mentioned in the docs are present (and of correct type) and that there are no extra properties we forgot to document. Furthermore, also verify that the provided examples are of correct type, too.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-01-03 21:51:13 +01:00
DL6ER c7803df604 Add /api/stats/upstreams and /api/stats/database/upstreams documentation
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-01-02 21:19:11 +01:00
DL6ER fa71150336 Order network devices descending by lastQuery
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-01-02 20:38:20 +01:00
DL6ER 507df1c157 Move /api/network to /api/ftl/network
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-01-02 20:37:50 +01:00
DL6ER 3389b1f0c2 Add /api/ftl/config which can be used to retrieve the entire configuration of FTL. This endpoint only supports GET so far, however, we will add POST in the future so that configuration values can be changed through the API
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-01-01 18:19:36 +01:00
DL6ER a9ea62a3fc Merge branch 'development' into new/http
Signed-off-by: DL6ER <dl6er@dl6er.de>
2022-12-30 15:12:14 +01:00
DL6ER f0b133b285 Always store time we start to save to the database (not only in debug mode). This avoids errorneous timing reports in case of errors.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2022-12-27 07:31:39 +01:00
DL6ER 3f1be06814 Fix logic for status code parsing
Signed-off-by: DL6ER <dl6er@dl6er.de>
2022-12-09 16:43:43 +01:00
DL6ER 983544e6d3 Fix incorrect DNSSEC-related warning during history import
Signed-off-by: DL6ER <dl6er@dl6er.de>
2022-12-03 12:43:20 +01:00
DL6ER 5252aeb077 Reference sqlite3ErrName instead of copying it. This requires SQLite3 patching
Signed-off-by: DL6ER <dl6er@dl6er.de>
2022-11-25 11:35:08 +01:00
DL6ER 51b2b71756 Print extended error code if anything in dbquery() fails. This will be helpful in a lot of cases where we'd only be logging 'disk I/O error' but a more specififc error is available
Signed-off-by: DL6ER <dl6er@dl6er.de>
2022-11-25 11:32:15 +01:00
DL6ER cafa2eff8f Merge pull request #1482 from pi-hole/update/sqlite_3.40.0
Update embedded SQLite3 engine to version 3.40.0
2022-11-17 12:51:42 +01:00
DL6ER bb57105001 Update embedded SQLite3 engine to version 3.40.0
Signed-off-by: DL6ER <dl6er@dl6er.de>
2022-11-16 17:58:46 +01:00
DL6ER af495664f9 Adlist warning -> ADLIST WARNING
Signed-off-by: DL6ER <dl6er@dl6er.de>
2022-11-15 21:28:38 +01:00
DL6ER fc212c383a Always ensure FTL messages are unique
Signed-off-by: DL6ER <dl6er@dl6er.de>
2022-11-15 19:33:47 +01:00
DL6ER 7ecd358f96 Allow FTL to analyze stale cache replies. They are assigned to a new query type (17)
Signed-off-by: DL6ER <dl6er@dl6er.de>
2022-11-09 20:09:48 +01:00
RD WebDesign e24e162da4 Fix comment
Signed-off-by: RD WebDesign <github@rdwebdesign.com.br>
2022-10-11 00:54:01 -03:00