pi-hole/FTL
1
0
Fork 0
mirror of https://github.com/pi-hole/FTL.git synced 2024-10-26 16:52:18 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
3,846 Commits 20 Branches 106 Tags
8cdc8ed48a4baeb15c23e886ae1ce206fab79ecd
Commit Graph

385 Commits

Author SHA1 Message Date
DL6ER 8cdc8ed48a Merge branch 'development' into new/optimize-queries 2022-01-05 20:50:27 +01:00
DL6ER 912ce90e47 Update SQLite to v3.37.1 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2022-01-02 15:19:28 +01:00
DL6ER 4516ee22b0 Do not use AUTOINCREMENT as it will add gaps between the IDs even if nothing is actually INSERTed as we chose the OR IGNORE path. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-12-16 22:30:30 +01:00
DL6ER f350ce5729 Create and use link table for additional_info column 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-12-16 22:10:17 +01:00
DL6ER d134a4943a Use the SQLITE_PREPARE_PERSISTENT flag is a hint to the query planner that the prepared statement will be retained for a long time and probably reused many times. Without this flag, SQLite3 assumes that the prepared statement will be used just once or at most a few times and then destroyed relatively soon. The current implementation acts on this hint by avoiding the use of [lookaside memory] so as not to deplete the limited store of lookaside memory. Also, update the tests for the expected database schema after renaming the table and adding the new view. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-12-16 12:28:19 +01:00
DL6ER beae1b3bf5 Use table name query_storage 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-12-16 11:52:08 +01:00
DL6ER 1cd6f04fb6 Utilize new queries_storage and link-tables for the queries table 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-12-16 10:58:12 +01:00
DL6ER 6c82a9c4f8 Optimize queries table by separating domain, client, and forward strings into a linking table and referencing them by an ID. This also allows storing (IP,hostname) pair for clients. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-12-16 10:58:01 +01:00
DL6ER 5b9eefd120 Merge pull request #1251 from pi-hole/update/sqlite_3.37.0 
Update SQLite engine from 3.36.0 to 3.37.0
 2021-12-02 06:59:05 +01:00
DL6ER db5e13d361 Update SQLite engine from 3.36.0 to 3.37.0 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-12-01 19:40:36 +01:00
DL6ER 4c2c59b214 Warn about resource shortages (15min load average exceeding number of cores, available disk space for database and log file, and available shared memory (RAM)) 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-11-29 14:11:15 +01:00
DL6ER 028989bcae Reduce code duplication by using labels + ensure we do not run dbclose() multiple times on database issues (e.g., when database is locked) 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-11-14 13:01:39 +01:00
DL6ER ff34a0c84b Add DNSMASQ_WARN message type whenever dnsmasq logs a warning. Also, do not flush the message table on reload (but only on restart) so that config-related (i.e. one-time) warnings are not accidentally deleted. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-11-13 10:28:59 +01:00
Christian König 0fde381a76 Print FTL version when calling the built in sqlite function only in interactive mode 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2021-10-24 20:42:51 +02:00
DL6ER c5e05a5880 Improve response time calculation. We are interested in the time it takes until we have everything we need to serve content. Typically, this is the first reply (either from cache or from upstream), but sometimes it is a later reply (e.g. when content can only partially be served from cache and needs extra forwarding). 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-10-14 20:57:24 +02:00
DL6ER d317a3f190 Merge branch 'development' into tweak/rate_limiting 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-10-03 14:53:07 +02:00
DL6ER 5ab7bf7797 Merge pull request #1162 from yubiuser/db_permissions 
Change database permission to 664
 2021-09-26 09:01:52 +02:00
DL6ER 5bd6b0910a Fix indentation 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-09-26 08:55:35 +02:00
yubiuser 8e363468b2 Set database permissions everytime the database is initialized 
Signed-off-by: yubiuser <ckoenig@posteo.de>
 2021-09-26 08:52:58 +02:00
yubiuser 6c7838ef3a Change database permission to 664 
Signed-off-by: yubiuser <ckoenig@posteo.de>
 2021-09-26 08:52:42 +02:00
DL6ER ac0346b9e2 Improvements based on static-analysis of source code 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-09-23 20:24:57 +02:00
DL6ER 671456f98e Fix rate-limiting queries only being resetted when rate-limiting actually triggered. This caused every device to be rate-limited sooner or later. This should not happen. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-09-20 19:53:48 +02:00
DL6ER 3ad3126494 Log how long a query will be refused when rate-limiting 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-09-20 11:33:13 +02:00
DL6ER b59a5c5dd7 Clients can still "collect" rate-limiting points while being rate-limited. If they are still querying like hell, they will never get un-blocked again. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-09-18 20:33:42 +02:00
DL6ER 296f2eb85a Add regex extension ";reply=NXDOMAIN,NODATA,REFUSED,IP,NONE" 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-09-09 21:51:09 +02:00
DL6ER 5f4d0395a7 Abort database routines early if database is known to be broken due to database file corruption. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-09-09 13:46:01 +02:00
DL6ER 2ec4312efb Ensure we can the correct error string when "ip neigh show" or "ip address show" fails. Before, we picked up the error from the logg() which was likely always a not ver helpful "Success" message 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-09-07 21:37:55 +02:00
DL6ER 07da984a96 Improvements suggested by cppcheck 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-09-07 21:36:21 +02:00
DL6ER 64f230f551 Update DB counters still within the running TRANSACTION to reduce disk I/O 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-09-03 13:11:14 +02:00
DL6ER 6e1cdaec85 Only open database when really necessary. This may reduce disk activity slightly and save a bit of CPU time. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-09-03 12:52:30 +02:00
DL6ER 127860e195 Merge pull request #1156 from pi-hole/tweak/blocking_while_busy 
Customizable locking while database is busy
 2021-08-23 20:19:36 +02:00
DL6ER f73c9d2907 Remove redundant upstream->count 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-08-22 11:58:51 +02:00
DL6ER 90982afdb3 Ensure busy blocking is also done when database was not available initially (incl. when forking a TCP worker) 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-08-19 10:06:24 +02:00
DL6ER d57311e56f Allow users to configure how FTL reacts to queries when the gravity database is not available 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-08-18 11:41:16 +02:00
DL6ER 79e853c80a Log how many queries have been saved in the final query storing 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-08-10 14:17:58 +02:00
DL6ER c47fc3b357 Set extended DNS error to UNSET (-1) when importing from the database 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-08-08 20:50:04 +02:00
DL6ER 62973a082f Log rate-limiting of clients to the message table 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-08-06 19:45:35 +02:00
DL6ER f0f6b1de6f Further reduce memory footprint of FTL by about 12%. We don't store the char pointer of the extended DNS errors because we can get this at any time. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-07-02 18:02:21 +02:00
DL6ER 979951c588 Reduce memory footprint of FTL by 11%. We don't store the rowid of a query in memory because we don't really need that. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-07-02 17:26:41 +02:00
DL6ER 4f8cbe29f7 Store real over-time counts of forwarded queries. So far, we counted only the first server a query was sent to. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-06-25 13:10:39 +02:00
DL6ER 4cd2b0362c Merge branch 'development' into update/dnsmasq-v2.86 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-06-24 12:00:46 +02:00
DL6ER 7e0fc46b22 Update SQLite engine to 3.36.0 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-06-23 10:22:16 +02:00
DL6ER a4005d8cc6 Merge branch 'development' into update/dnsmasq-v2.86 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-06-22 09:53:56 +02:00
DL6ER 77b578edf5 Merge pull request #1134 from pi-hole/fix/lock_gravityClose 
Improve locking during heavy TCP forking
 2021-05-22 22:26:03 +02:00
DL6ER 69b4f1f775 Do not relock during database loading 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-05-12 11:54:03 +02:00
DL6ER ada54a7274 Add location logging to no-lock message 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-05-10 17:36:20 +02:00
DL6ER 90764f4be2 Check if mutex is already locked before trying to lock again in forks with timeouts 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-05-10 09:51:08 +02:00
DL6ER 809e721d8e Lock when terminating forks to ensure we remap shared memory. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-04-27 12:15:27 +02:00
DL6ER 49094570c4 Enable compile-time flag SQLITE_ENABLE_DBPAGE_VTAB to add support for .recover shell option. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-04-23 17:29:10 +02:00
Petr Čech 74a6b073f1 Merge pull request #1120 from petrcech/patch-1 
Fix compilation failure
 2021-04-21 22:05:03 +02:00