diff --git a/README.md b/README.md index c71246e..57ddd3d 100644 --- a/README.md +++ b/README.md @@ -9,6 +9,8 @@ The stack now uses safer defaults for production: * The published SOCKS entrypoint now requires credentials from a Docker secret. * CoreDNS recursive resolution for the catch-all zone is restricted to trusted/internal networks. * Pi-hole ARP cache parsing is disabled in Docker (`FTLCONF_database_network_parseARPcache=false`) to prevent recurring netlink `neigh`/ARP errors. +* Fixed `10.5.0.x` service IPs are retained for the stack's internal routing/DNS design, but host/LAN-specific values now default to portable settings and can be overridden with environment variables. +* Optional integrations that need operator-specific credentials, such as Tailscale, are no longer enabled with placeholder defaults in the base stack. Before starting in production, set these secret files: @@ -18,11 +20,28 @@ printf 'proxyuser:REPLACE_WITH_A_LONG_RANDOM_PASSWORD\n' > secrets/3proxy_users. printf 'REPLACE_WITH_A_LONG_RANDOM_NAMECOIN_RPC_PASSWORD\n' > secrets/namecoin_rpc_password.txt ``` +Optional Tailscale note: + +* The `tailscale` service is now behind the Compose `tailscale` profile. +* It is not started by default. +* To enable it, export a real auth key and start that profile explicitly: + +```sh +export TS_AUTHKEY=tskey-your-real-key +docker compose --profile tailscale up -d tailscale +``` + +Release default note for `.zil`: + +* `zildns` now uses explicit public resolvers inside its container so its Zilliqa RPC lookups do not depend on Docker's embedded DNS path. +* This change is limited to the `zildns` container; the stack's internal fixed `10.5.0.x` service IP design remains unchanged. + SOCKS auth note: * `dark3proxy` now reads credentials from `secrets/3proxy_users.txt`. * Each non-comment line in that secret must be `username:password`. * If you intentionally want an internal-only unauthenticated proxy, set `PROXY_REQUIRE_AUTH=false` for `dark3proxy` and remove the public `2000:1080` port mapping. +* The router now defaults to **no auth bypass CIDRs**. If you want trusted subnets to skip SOCKS auth, set `ROUTER_NO_AUTH_CIDRS` in your shell or `.env`, for example `ROUTER_NO_AUTH_CIDRS=192.168.1.0/24`. Pi-hole note: disabling ARP parsing avoids noisy `Failed to read ARP cache` messages in containerized setups where neighbor-table netlink operations are not @@ -54,6 +73,18 @@ Public proxy access is exposed on a single host port: Local admin endpoint: * `127.0.0.1:2002` -> 3proxy admin (`dark3proxy:8161`), host-local only. +* `127.0.0.1:2003` -> Pi-hole web UI (`darkpihole:80`), host-local by default. +* `127.0.0.1:2004` -> status dashboard (`darkstatus:8080`), host-local by default. + +These host-local admin bindings can be overridden with environment variables when +needed: + +```sh +export PIHOLE_WEB_BIND_HOST=192.168.1.31 +export PIHOLE_WEB_BIND_PORT=2003 +export STATUS_DASHBOARD_BIND_HOST=192.168.1.31 +export STATUS_DASHBOARD_BIND_PORT=2004 +``` Router internals for `:2000`: @@ -261,17 +292,24 @@ records in Namecoin itself and wait for local `namecoind` sync completion. Quick test: ```sh -docker exec darkpihole dig @10.5.0.4 A oxen.loki +LOKI_ADDR="$(docker logs darklokinet 2>&1 | sed -n 's/.*endpoint:\([a-z0-9]\{52\}\.loki\).*/\1/p' | tail -n1)" +docker exec darkpihole dig @10.5.0.4 A "$LOKI_ADDR" ``` > **Note:** Lokinet may need a short bootstrap period after startup before -> `.loki` names resolve. +> `.loki` names resolve. Older sample names such as `oxen.loki` are no longer a +> reliable health check; resolving the container's self-published `.loki` +> address is a better validation of the local Lokinet path. -## Monitoring the proxy +## Monitoring and stack status -A lightweight Prometheus stack is included to expose 3proxy metrics and give -you visibility into traffic volumes, connected sessions, etc. +Two lightweight tools are included for visibility: + +* `3proxy_exporter` exposes 3proxy counters in Prometheus format for external + scraping. +* `status_dashboard` serves a local-only status page for the whole stack, + including container state, probe results, proxy usernames, and recent logs. ### how it works @@ -279,32 +317,39 @@ you visibility into traffic volumes, connected sessions, etc. status socket. Samples look like `PROXY CONNS 12` or `SOCKS IN 345`. * `monitor/exporter.py` polls that socket every few seconds and exports the counters on HTTP port **9100** in Prometheus format. -* The `docker-compose.yml` file defines two monitoring services: - `3proxy_exporter` and `prometheus`. - Prometheus scrapes the exporter and exposes the metrics for inspection or - external dashboarding. +* `monitor/status_dashboard.py` talks to the local Docker socket, runs targeted + DNS and SOCKS checks, and serves both HTML and JSON for stack status. ### building & running -The exporter lives in `monitor/`; build the image and start the stack: +The monitoring utilities live in `monitor/`; build the images and start the +stack: ```sh -# build everything including the monitoring services -docker-compose build 3proxy_exporter prometheus +# build the exporter and status dashboard +docker-compose build 3proxy_exporter status_dashboard docker-compose up -d ``` -Prometheus will be accessible on port **9090**. If you want dashboards, you -can point any external Grafana or compatible tool at Prometheus and use metrics -such as `proxy_conns` and `proxy_bytes_in`. +By default the status page is available on **http://127.0.0.1:2004/** and the +raw JSON is available on **http://127.0.0.1:2004/api/status**. If you override +the bind host/port via `STATUS_DASHBOARD_BIND_HOST` or +`STATUS_DASHBOARD_BIND_PORT`, use that address instead. + +If you want external dashboards or alerts, point a Prometheus-compatible +collector at the exporter and use metrics such as `proxy_conns` and +`proxy_bytes_in`. ### tips -* Alerts can be added in Prometheus rules, e.g. fire when `proxy_conns` - exceeds a threshold for several minutes. -* If you don’t want the full stack, you can still query the monitor port - directly with `nc`; nothing in the proxy depends on the exporter. +* Alerts can be added in your external Prometheus rules, e.g. fire when + `proxy_conns` exceeds a threshold for several minutes. +* You can still query the monitor port directly with `nc`; nothing in the + proxy depends on the exporter. +* The status dashboard can show logs and configured proxy usernames, so keep it + behind a trusted admin network or add your own access controls before + exposing it broadly. * The `scripts/validate-config.sh` script warns if a 3proxy config lacks a `monitor` line. @@ -321,8 +366,12 @@ any warnings or errors. The script covers: * CoreDNS/PopuraDNS Corefile syntax * Unbound configuration syntax * Presence of required secret files +* Portable host-local defaults for Pi-hole, the status dashboard, and router auth bypass CIDRs * Simple 3proxy configuration sanity (presence of `socks`/`proxy` rules) * Detection of mutable `:latest` image tags in deployment compose files +* Verification that critical resolver/backend services keep explicit healthchecks +* Live CoreDNS smoke checks for `.onion`, `.i2p`, `.eth`, and `.zil` +* Verification that Tailscale stays opt-in and does not ship a placeholder auth key * Verification that restart policies exist The GitHub Actions workflow also includes a **smoke test** job that spins up diff --git a/docker-compose.lock.yml b/docker-compose.lock.yml index ea8571b..ccd1517 100644 --- a/docker-compose.lock.yml +++ b/docker-compose.lock.yml @@ -7,7 +7,7 @@ services: container_name: dark3proxy-exporter depends_on: dark3proxy: - condition: service_started + condition: service_healthy required: true image: darkproxy-3proxy-exporter:local networks: @@ -20,7 +20,7 @@ services: container_name: darkalfis dns: - 10.5.0.6 - image: cofob/alfis + image: cofob/alfis@sha256:0c5788b1e409557bb814dc2d055d584db8c48b3d42b375bf13feb59583af0585 networks: darkproxy: ipv4_address: 10.5.0.3 @@ -79,7 +79,7 @@ services: PROXY_USERS_FILE: /run/secrets/PROXY_USERS ROUTER_I2P_MAP_FILE: /var/lib/i2pdns/map.json ROUTER_I2P_POOL_CIDR: 172.31.0.0/16 - ROUTER_NO_AUTH_CIDRS: 192.168.1.0/24 + ROUTER_NO_AUTH_CIDRS: "" ROUTER_REQUIRE_AUTH: "true" ROUTER_USERS_FILE: /run/secrets/PROXY_USERS YGG_CONNECT_WAIT_SECONDS: "8" @@ -145,7 +145,7 @@ services: - -conf=/emc/emercoin.conf - -printtoconsole container_name: darkemer - image: wg00/emercoin:0.8.4 + image: wg00/emercoin:0.8.4@sha256:b890987fb4b158305040dc76b32cd24ed5173dd7bc4aba983141ea0fe82e2996 mem_limit: "1073741824" mem_reservation: "268435456" networks: @@ -173,9 +173,21 @@ services: dockerfile: Dockerfile cpus: 0.5 container_name: darkens + dns: + - 10.5.0.4 environment: ENS_RPC_URL: https://ethereum-rpc.publicnode.com ENSDNS_TTL: "60" + healthcheck: + test: + - CMD + - python + - -c + - import socket; from dnslib import DNSRecord; q=DNSRecord.question('vitalik.eth','TXT'); s=socket.socket(socket.AF_INET, socket.SOCK_DGRAM); s.settimeout(2); s.sendto(q.pack(), ('127.0.0.1', 53)); data, _ = s.recvfrom(4096); DNSRecord.parse(data) + timeout: 10s + interval: 1m0s + retries: 3 + start_period: 15s mem_limit: "268435456" mem_reservation: "67108864" networks: @@ -205,8 +217,8 @@ services: test: - CMD - bash - - -c - - pgrep i2pd && pgrep yggdrasil + - -lc + - pgrep i2pd >/dev/null && pgrep yggdrasil >/dev/null && [ -s /var/lib/i2pd/addressbook/addresses.csv ] timeout: 10s interval: 30s retries: 3 @@ -238,6 +250,15 @@ services: target: /run/secrets/tz sysctls: net.ipv6.conf.all.disable_ipv6: "0" + volumes: + - type: volume + source: i2pd_state + target: /var/lib/i2pd + volume: {} + - type: volume + source: yggdrasil_state + target: /var/lib/yggdrasil + volume: {} i2pdns: build: context: /home/blade/darkproxy/i2pdns @@ -250,6 +271,16 @@ services: I2PDNS_MAP_FILE: /var/lib/i2pdns/map.json I2PDNS_POOL_CIDR: 172.31.0.0/16 I2PDNS_TTL: "60" + healthcheck: + test: + - CMD + - python + - -c + - import socket; from dnslib import DNSRecord; q=DNSRecord.question('stats.i2p','A'); s=socket.socket(socket.AF_INET, socket.SOCK_DGRAM); s.settimeout(2); s.sendto(q.pack(), ('127.0.0.1', 53)); data, _ = s.recvfrom(4096); DNSRecord.parse(data) + timeout: 10s + interval: 1m0s + retries: 3 + start_period: 10s mem_limit: "134217728" mem_reservation: "67108864" networks: @@ -279,6 +310,14 @@ services: - source: /dev/net/tun target: /dev/net/tun permissions: rwm + healthcheck: + test: + - CMD-SHELL + - pidof lokinet >/dev/null && ip link show lokinet0 >/dev/null 2>&1 + timeout: 10s + interval: 1m0s + retries: 3 + start_period: 45s mem_limit: "268435456" mem_reservation: "67108864" networks: @@ -314,6 +353,14 @@ services: -addnode=3.228.193.128:8334 \ -addnode=3.66.245.44:8334 \ -addnode=212.51.144.42:8334 + healthcheck: + test: + - CMD-SHELL + - 'curl -fsS --user "namecoinrpc:$$(cat /run/secrets/NAMECOIN_RPC_PASSWORD)" --data-binary ''{"jsonrpc":"1.0","id":"health","method":"getblockcount","params":[]}'' -H ''content-type: text/plain;'' http://127.0.0.1:8336/ >/dev/null' + timeout: 10s + interval: 1m0s + retries: 3 + start_period: 30s image: ukd1/namecoind@sha256:6fdf63f61f687ee639893a8cdcd910bb24c5eb3a3493ff72142c911d7019200b mem_limit: "2147483648" mem_reservation: "536870912" @@ -342,13 +389,23 @@ services: container_name: darknamecoin depends_on: namecoind: - condition: service_started + condition: service_healthy required: true environment: NAMECOIN_RPC_PASSWORD_FILE: /run/secrets/NAMECOIN_RPC_PASSWORD NAMECOIN_RPC_TIMEOUT: "8" NAMECOIN_RPC_URL: http://darknamecoind:8336/ NAMECOIN_RPC_USER: namecoinrpc + healthcheck: + test: + - CMD + - python + - -c + - import socket; from dnslib import DNSRecord; q=DNSRecord.question('d.bit','TXT'); s=socket.socket(socket.AF_INET, socket.SOCK_DGRAM); s.settimeout(2); s.sendto(q.pack(), ('127.0.0.1', 53)); data, _ = s.recvfrom(4096); DNSRecord.parse(data) + timeout: 10s + interval: 1m0s + retries: 3 + start_period: 15s mem_limit: "268435456" mem_reservation: "67108864" networks: @@ -369,10 +426,13 @@ services: - SYS_NICE cpus: 1.5 container_name: darkpihole + entrypoint: + - /usr/local/bin/darkproxy-pihole-entrypoint.sh environment: FTLCONF_database_network_parseARPcache: "false" + FTLCONF_dns_listeningMode: all FTLCONF_dns_upstreams: 10.5.0.4#53 - PIHOLE_DNS_: 10.5.0.4 + PIHOLE_DNS_1: 10.5.0.4 TEMPERATUREUNIT: f TZ: America/Detroit WEBPASSWORD_FILE: /run/secrets/PIHOLE_WEBPASSWORD @@ -390,7 +450,7 @@ services: interval: 1m0s retries: 3 start_period: 30s - image: pihole/pihole@sha256:1c32c36b862a12762656b6471c854cebc01fe945639ba3a893611337c2c95e99 + image: pihole/pihole@sha256:712b39f1fdb55121cef509813dfbe02d2bdef9c28e07404fa1d422f5157323b2 mem_limit: "536870912" mem_reservation: "134217728" networks: @@ -430,9 +490,47 @@ services: target: /etc/dnsmasq.d bind: create_host_path: true - prometheus: - container_name: darkprom - image: prom/prometheus@sha256:e4254400b85610324913f0dc4acf92603d9984e7519414c5a12811aa6146acc3 + - type: bind + source: /home/blade/darkproxy/scripts/pihole-entrypoint.sh + target: /usr/local/bin/darkproxy-pihole-entrypoint.sh + read_only: true + bind: + create_host_path: true + status_dashboard: + build: + context: /home/blade/darkproxy/monitor + dockerfile: status.Dockerfile + container_name: darkstatus + depends_on: + dark3proxy: + condition: service_healthy + required: true + pihole: + condition: service_healthy + required: true + environment: + DASHBOARD_PORT: "8080" + DNS_PORT: "53" + DNS_SERVER: darkpihole + DOCKER_PROJECT: darkproxy + DOCKER_SOCKET: /var/run/docker.sock + LOG_TAIL_LINES: "120" + PROXY_HOST: dark3proxy + PROXY_PORT: "1080" + PROXY_USERS_FILE: /run/secrets/PROXY_USERS + SOCKET_TIMEOUT: "5" + STATUS_CACHE_SECONDS: "15" + healthcheck: + test: + - CMD + - python + - -c + - import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/healthz', timeout=5).read() + timeout: 10s + interval: 1m0s + retries: 3 + start_period: 20s + image: darkproxy-status-dashboard:local networks: darkproxy: ipv4_address: 10.5.0.21 @@ -440,50 +538,22 @@ services: ports: - mode: ingress host_ip: 127.0.0.1 - target: 9090 - published: "9090" + target: 8080 + published: "2004" protocol: tcp + restart: unless-stopped + secrets: + - source: PROXY_USERS + target: /run/secrets/PROXY_USERS security_opt: - no-new-privileges:true volumes: - type: bind - source: /home/blade/darkproxy/monitor/prometheus.yml - target: /etc/prometheus/prometheus.yml + source: /var/run/docker.sock + target: /var/run/docker.sock read_only: true bind: create_host_path: true - tailscale: - build: - context: /home/blade/darkproxy/tailscale - dockerfile: Dockerfile - cap_add: - - NET_ADMIN - - SYS_MODULE - container_name: darkscale - devices: - - source: /dev/net/tun - target: /dev/net/tun - permissions: rwm - environment: - TS_AUTHKEY: tskey-YOUR-AUTH-KEY-HERE - TS_EXTRA_ARGS: --accept-dns=false --advertise-exit-node - TS_STATE_DIR: /var/lib/tailscale - hostname: darkproxy-exit - networks: - darkproxy: - ipv4_address: 10.5.0.10 - platform: linux/amd64 - restart: unless-stopped - security_opt: - - no-new-privileges:true - sysctls: - net.ipv4.ip_forward: "1" - net.ipv6.conf.all.forwarding: "1" - volumes: - - type: volume - source: tailscale_data - target: /var/lib/tailscale - volume: {} tor_yggdrasil: build: context: /home/blade/darkproxy/tor_yggdrasil_docker @@ -524,6 +594,14 @@ services: unbound: cpus: 4 container_name: darkunbound + healthcheck: + test: + - CMD-SHELL + - drill @127.0.0.1 google.com >/dev/null 2>&1 + timeout: 10s + interval: 1m0s + retries: 3 + start_period: 20s image: mvance/unbound@sha256:76906da36d1806f3387338f15dcf8b357c51ce6897fb6450d6ce010460927e90 mem_limit: "805306368" mem_reservation: "268435456" @@ -562,12 +640,25 @@ services: dockerfile: Dockerfile cpus: 0.5 container_name: darkzil + dns: + - 1.1.1.1 + - 8.8.8.8 environment: ZILDNS_CACHE_SECONDS: "300" ZILDNS_PREWARM_DOMAIN: brad.zil ZILDNS_TTL: "120" ZILDNS_ZNS_NETWORK: mainnet ZILDNS_ZNS_URL: https://api.zilliqa.com + healthcheck: + test: + - CMD + - node + - -e + - const net=require('net'); const socket=net.createConnection({host:'127.0.0.1', port:53}); socket.setTimeout(2000); socket.on('connect',()=>{socket.end(); process.exit(0);}); socket.on('timeout',()=>{socket.destroy(); process.exit(1);}); socket.on('error',()=>process.exit(1)); + timeout: 10s + interval: 1m0s + retries: 3 + start_period: 20s mem_limit: "268435456" mem_reservation: "67108864" networks: @@ -597,10 +688,12 @@ volumes: name: emc_data i2p_dns_map: name: i2p_dns_map + i2pd_state: + name: i2pd_state namecoin_data: name: namecoin_data - tailscale_data: - name: tailscale_data + yggdrasil_state: + name: yggdrasil_state secrets: NAMECOIN_RPC_PASSWORD: name: darkproxy_NAMECOIN_RPC_PASSWORD diff --git a/docker-compose.yml b/docker-compose.yml index 05f7211..31853db 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -16,7 +16,7 @@ services: PROXY_REQUIRE_AUTH: "true" PROXY_USERS_FILE: /run/secrets/PROXY_USERS ROUTER_REQUIRE_AUTH: "true" - ROUTER_NO_AUTH_CIDRS: 192.168.1.0/24 + ROUTER_NO_AUTH_CIDRS: ${ROUTER_NO_AUTH_CIDRS:-} ROUTER_USERS_FILE: /run/secrets/PROXY_USERS ROUTER_I2P_MAP_FILE: /var/lib/i2pdns/map.json ROUTER_I2P_POOL_CIDR: 172.31.0.0/16 @@ -80,12 +80,15 @@ services: tz: /run/secrets/tz secrets: - tz + volumes: + - i2pd_state:/var/lib/i2pd + - yggdrasil_state:/var/lib/yggdrasil restart: unless-stopped cpus: "4.0" mem_reservation: 512m mem_limit: 2g healthcheck: - test: ["CMD", "bash", "-c", "pgrep i2pd && pgrep yggdrasil"] + test: ["CMD", "bash", "-lc", "pgrep i2pd >/dev/null && pgrep yggdrasil >/dev/null && [ -s /var/lib/i2pd/addressbook/addresses.csv ]"] interval: 30s timeout: 10s retries: 3 @@ -120,6 +123,12 @@ services: - "i2p_dns_map:/var/lib/i2pdns" sysctls: - "net.ipv6.conf.all.disable_ipv6=0" + healthcheck: + test: ["CMD", "python", "-c", "import socket; from dnslib import DNSRecord; q=DNSRecord.question('stats.i2p','A'); s=socket.socket(socket.AF_INET, socket.SOCK_DGRAM); s.settimeout(2); s.sendto(q.pack(), ('127.0.0.1', 53)); data, _ = s.recvfrom(4096); DNSRecord.parse(data)"] + interval: 1m + timeout: 10s + retries: 3 + start_period: 10s networks: darkproxy: ipv4_address: 10.5.0.16 @@ -153,7 +162,7 @@ services: ipv4_address: 10.5.0.7 alfis: - image: cofob/alfis + image: cofob/alfis@sha256:0c5788b1e409557bb814dc2d055d584db8c48b3d42b375bf13feb59583af0585 platform: linux/amd64 # platform removed; image is amd64 only container_name: darkalfis @@ -223,11 +232,18 @@ services: cpus: "4.0" mem_reservation: 256m mem_limit: 768m + healthcheck: + test: ["CMD-SHELL", "drill @127.0.0.1 google.com >/dev/null 2>&1"] + interval: 1m + timeout: 10s + retries: 3 + start_period: 20s pihole: container_name: darkpihole - image: pihole/pihole@sha256:1c32c36b862a12762656b6471c854cebc01fe945639ba3a893611337c2c95e99 + image: pihole/pihole@sha256:712b39f1fdb55121cef509813dfbe02d2bdef9c28e07404fa1d422f5157323b2 platform: linux/amd64 + entrypoint: ["/usr/local/bin/darkproxy-pihole-entrypoint.sh"] security_opt: - no-new-privileges:true # For DHCP it is recommended to remove these ports and instead add: network_mode: "host" @@ -243,15 +259,15 @@ services: # - "67:67/udp" # Uncomment the line below if you are using Pi-hole as your NTP server # - "123:123/udp" - - "127.0.0.1:2003:80/tcp" # pihole web port (localhost only) + - "${PIHOLE_WEB_BIND_HOST:-127.0.0.1}:${PIHOLE_WEB_BIND_PORT:-2003}:80/tcp" # pihole web port (host-local by default) environment: # Set the appropriate timezone for your location (https://en.wikipedia.org/wiki/List_of_tz_database_time_zones), e.g: TZ: 'America/Detroit' # WEBPASSWORD: 'set a secure password here or it will be random' # FTLCONF_webserver_api_password: 'darkproxy' # If using Docker's default `bridge` network setting the dns listening mode should be set to 'all' -# FTLCONF_dns_listeningMode: 'all' - PIHOLE_DNS_: 10.5.0.4 + FTLCONF_dns_listeningMode: 'all' + PIHOLE_DNS_1: 10.5.0.4 TEMPERATUREUNIT: f WEBTHEME: lcars WEBPASSWORD_FILE: /run/secrets/PIHOLE_WEBPASSWORD @@ -266,6 +282,7 @@ services: - './pihole/etc-pihole:/etc/pihole' # Uncomment the below if you have custom dnsmasq config files that you want to persist. Not needed for most starting fresh with Pi-hole v6. If you're upgrading from v5 you and have used this directory before, you should keep it enabled for the first v6 container start to allow for a complete migration. It can be removed afterwards. Needs environment variable FTLCONF_misc_etc_dnsmasq_d: 'true' - './pihole/etc-dnsmasq.d:/etc/dnsmasq.d' + - './scripts/pihole-entrypoint.sh:/usr/local/bin/darkproxy-pihole-entrypoint.sh:ro' cap_add: # See https://github.com/pi-hole/docker-pi-hole#note-on-capabilities # Required if you are using Pi-hole as your DHCP server, else not needed @@ -291,7 +308,7 @@ services: ipv4_address: 10.5.0.6 emc: - image: wg00/emercoin:0.8.4 + image: wg00/emercoin:0.8.4@sha256:b890987fb4b158305040dc76b32cd24ed5173dd7bc4aba983141ea0fe82e2996 platform: linux/amd64 container_name: darkemer security_opt: @@ -327,8 +344,16 @@ services: environment: ENS_RPC_URL: https://ethereum-rpc.publicnode.com ENSDNS_TTL: 60 + dns: + - "10.5.0.4" sysctls: - "net.ipv6.conf.all.disable_ipv6=0" + healthcheck: + test: ["CMD", "python", "-c", "import socket; from dnslib import DNSRecord; q=DNSRecord.question('vitalik.eth','TXT'); s=socket.socket(socket.AF_INET, socket.SOCK_DGRAM); s.settimeout(2); s.sendto(q.pack(), ('127.0.0.1', 53)); data, _ = s.recvfrom(4096); DNSRecord.parse(data)"] + interval: 1m + timeout: 10s + retries: 3 + start_period: 15s networks: darkproxy: ipv4_address: 10.5.0.11 @@ -351,8 +376,17 @@ services: ZILDNS_TTL: 120 ZILDNS_CACHE_SECONDS: 300 ZILDNS_PREWARM_DOMAIN: brad.zil + dns: + - "1.1.1.1" + - "8.8.8.8" sysctls: - "net.ipv6.conf.all.disable_ipv6=0" + healthcheck: + test: ["CMD", "node", "-e", "const net=require('net'); const socket=net.createConnection({host:'127.0.0.1', port:53}); socket.setTimeout(2000); socket.on('connect',()=>{socket.end(); process.exit(0);}); socket.on('timeout',()=>{socket.destroy(); process.exit(1);}); socket.on('error',()=>process.exit(1));"] + interval: 1m + timeout: 10s + retries: 3 + start_period: 20s networks: darkproxy: ipv4_address: 10.5.0.15 @@ -376,6 +410,12 @@ services: - /dev/net/tun:/dev/net/tun sysctls: - "net.ipv6.conf.all.disable_ipv6=0" + healthcheck: + test: ["CMD-SHELL", "pidof lokinet >/dev/null && ip link show lokinet0 >/dev/null 2>&1"] + interval: 1m + timeout: 10s + retries: 3 + start_period: 45s networks: darkproxy: ipv4_address: 10.5.0.14 @@ -417,6 +457,12 @@ services: - namecoin_data:/data sysctls: - "net.ipv6.conf.all.disable_ipv6=0" + healthcheck: + test: ["CMD-SHELL", "curl -fsS --user \"namecoinrpc:$(cat /run/secrets/NAMECOIN_RPC_PASSWORD)\" --data-binary '{\"jsonrpc\":\"1.0\",\"id\":\"health\",\"method\":\"getblockcount\",\"params\":[]}' -H 'content-type: text/plain;' http://127.0.0.1:8336/ >/dev/null"] + interval: 1m + timeout: 10s + retries: 3 + start_period: 30s networks: darkproxy: ipv4_address: 10.5.0.13 @@ -434,7 +480,8 @@ services: mem_reservation: 64m mem_limit: 256m depends_on: - - namecoind + namecoind: + condition: service_healthy environment: NAMECOIN_RPC_URL: http://darknamecoind:8336/ NAMECOIN_RPC_USER: namecoinrpc @@ -444,6 +491,12 @@ services: - NAMECOIN_RPC_PASSWORD sysctls: - "net.ipv6.conf.all.disable_ipv6=0" + healthcheck: + test: ["CMD", "python", "-c", "import socket; from dnslib import DNSRecord; q=DNSRecord.question('d.bit','TXT'); s=socket.socket(socket.AF_INET, socket.SOCK_DGRAM); s.settimeout(2); s.sendto(q.pack(), ('127.0.0.1', 53)); data, _ = s.recvfrom(4096); DNSRecord.parse(data)"] + interval: 1m + timeout: 10s + retries: 3 + start_period: 15s networks: darkproxy: ipv4_address: 10.5.0.12 @@ -453,6 +506,7 @@ services: context: ./tailscale dockerfile: Dockerfile platform: linux/amd64 + profiles: ["tailscale"] container_name: darkscale hostname: darkproxy-exit security_opt: @@ -463,8 +517,7 @@ services: devices: - /dev/net/tun:/dev/net/tun environment: - # ⚠️ Get your own auth key from https://login.tailscale.com/admin/settings/keys - - TS_AUTHKEY=${TS_AUTHKEY:-tskey-YOUR-AUTH-KEY-HERE} + - TS_AUTHKEY=${TS_AUTHKEY:-} - TS_EXTRA_ARGS=--accept-dns=false --advertise-exit-node - TS_STATE_DIR=/var/lib/tailscale volumes: @@ -487,26 +540,60 @@ services: security_opt: - no-new-privileges:true depends_on: - - dark3proxy + dark3proxy: + condition: service_healthy networks: darkproxy: ipv4_address: 10.5.0.20 - prometheus: - image: prom/prometheus@sha256:e4254400b85610324913f0dc4acf92603d9984e7519414c5a12811aa6146acc3 - container_name: darkprom + status_dashboard: + container_name: darkstatus + build: + context: ./monitor + dockerfile: status.Dockerfile + image: darkproxy-status-dashboard:local platform: linux/amd64 security_opt: - no-new-privileges:true + depends_on: + dark3proxy: + condition: service_healthy + pihole: + condition: service_healthy + environment: + DASHBOARD_PORT: 8080 + DOCKER_PROJECT: darkproxy + DOCKER_SOCKET: /var/run/docker.sock + DNS_SERVER: darkpihole + DNS_PORT: 53 + PROXY_HOST: dark3proxy + PROXY_PORT: 1080 + PROXY_USERS_FILE: /run/secrets/PROXY_USERS + STATUS_CACHE_SECONDS: 15 + SOCKET_TIMEOUT: 5 + LOG_TAIL_LINES: 120 + secrets: + - PROXY_USERS volumes: - - ./monitor/prometheus.yml:/etc/prometheus/prometheus.yml:ro + - /var/run/docker.sock:/var/run/docker.sock:ro ports: - - "127.0.0.1:9090:9090" + - "${STATUS_DASHBOARD_BIND_HOST:-127.0.0.1}:${STATUS_DASHBOARD_BIND_PORT:-2004}:8080" + healthcheck: + test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/healthz', timeout=5).read()"] + interval: 1m + timeout: 10s + retries: 3 + start_period: 20s networks: darkproxy: ipv4_address: 10.5.0.21 + restart: unless-stopped volumes: + i2pd_state: + name: i2pd_state + yggdrasil_state: + name: yggdrasil_state darkalfis_data: name: darkalfis_data emc_data: diff --git a/ensdns/server.py b/ensdns/server.py index 4c93a2f..96780c4 100644 --- a/ensdns/server.py +++ b/ensdns/server.py @@ -2,6 +2,7 @@ import os import socket import threading +import time from dnslib import A, AAAA, CNAME, QTYPE, RR, TXT, DNSHeader, DNSRecord, RCODE from web3 import HTTPProvider, Web3 @@ -11,6 +12,7 @@ RPC_URL = os.getenv("ENS_RPC_URL", "https://ethereum-rpc.publicnode.com") LISTEN_HOST = os.getenv("ENSDNS_LISTEN_HOST", "0.0.0.0") LISTEN_PORT = int(os.getenv("ENSDNS_LISTEN_PORT", "53")) DEFAULT_TTL = int(os.getenv("ENSDNS_TTL", "60")) +RECONNECT_INTERVAL = float(os.getenv("ENSDNS_RECONNECT_INTERVAL", "30")) ENS_REGISTRY_ADDRESS = Web3.to_checksum_address("0x00000000000C2E074eC69A0dFb2997BA6C7d2e1e") @@ -44,6 +46,10 @@ PUBLIC_RESOLVER_ABI = [ EMPTY_ADDRESS = "0x0000000000000000000000000000000000000000" +class ENSBackendUnavailable(RuntimeError): + pass + + def namehash(name: str) -> bytes: node = b"\x00" * 32 labels = [label for label in name.strip().lower().split(".") if label] @@ -59,18 +65,58 @@ def normalize_qname(qname: str) -> str: class ENSResolver: def __init__(self, rpc_url: str): - self.web3 = Web3(HTTPProvider(rpc_url, request_kwargs={"timeout": 10})) - if not self.web3.is_connected(): - raise RuntimeError(f"Could not connect to Ethereum RPC: {rpc_url}") - self.registry = self.web3.eth.contract(address=ENS_REGISTRY_ADDRESS, abi=ENS_REGISTRY_ABI) + self.rpc_url = rpc_url + self._lock = threading.Lock() + self._web3 = None + self._registry = None + self._last_connect_attempt = 0.0 + self._last_error = None + + def _connect(self, force: bool = False) -> bool: + now = time.monotonic() + with self._lock: + if self._web3 is not None and self._registry is not None and self._web3.is_connected(): + return True + if not force and now - self._last_connect_attempt < RECONNECT_INTERVAL: + return False + + self._last_connect_attempt = now + web3 = Web3(HTTPProvider(self.rpc_url, request_kwargs={"timeout": 10})) + if not web3.is_connected(): + self._web3 = None + self._registry = None + self._last_error = f"Could not connect to Ethereum RPC: {self.rpc_url}" + print(f"[ensdns] {self._last_error}", flush=True) + return False + + self._web3 = web3 + self._registry = web3.eth.contract(address=ENS_REGISTRY_ADDRESS, abi=ENS_REGISTRY_ABI) + self._last_error = None + print(f"[ensdns] connected to {self.rpc_url}", flush=True) + return True + + def _ensure_backend(self): + if not self._connect(): + detail = self._last_error or f"Could not connect to Ethereum RPC: {self.rpc_url}" + raise ENSBackendUnavailable(detail) + return self._web3, self._registry def resolve(self, name: str): + web3, registry = self._ensure_backend() node = namehash(name) - resolver_addr = self.registry.functions.resolver(node).call() + try: + resolver_addr = registry.functions.resolver(node).call() + except Exception as exc: + with self._lock: + self._web3 = None + self._registry = None + self._last_error = f"ENS registry lookup failed: {exc}" + raise ENSBackendUnavailable(self._last_error) from exc + if not resolver_addr or resolver_addr == EMPTY_ADDRESS: return None - resolver = self.web3.eth.contract(address=resolver_addr, abi=PUBLIC_RESOLVER_ABI) + resolver = web3.eth.contract(address=resolver_addr, abi=PUBLIC_RESOLVER_ABI) addr = None contenthash = None @@ -105,7 +151,13 @@ def answer_query(record: DNSRecord, ens: ENSResolver) -> DNSRecord: reply.header.rcode = RCODE.NXDOMAIN return reply - resolved = ens.resolve(qname) + try: + resolved = ens.resolve(qname) + except ENSBackendUnavailable as exc: + print(f"[ensdns] backend unavailable while resolving {qname}: {exc}", flush=True) + reply.header.rcode = RCODE.SERVFAIL + return reply + if resolved is None: reply.header.rcode = RCODE.NXDOMAIN return reply @@ -163,8 +215,9 @@ def serve_tcp(sock: socket.socket, ens: ENSResolver): def main(): ens = ENSResolver(RPC_URL) - print(f"[ensdns] connected to {RPC_URL}", flush=True) print(f"[ensdns] listening on {LISTEN_HOST}:{LISTEN_PORT} (udp/tcp)", flush=True) + if not ens._connect(force=True): + print(f"[ensdns] starting degraded; RPC unavailable at {RPC_URL}", flush=True) udp_sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) udp_sock.bind((LISTEN_HOST, LISTEN_PORT)) diff --git a/monitor/status.Dockerfile b/monitor/status.Dockerfile new file mode 100644 index 0000000..c7a31e8 --- /dev/null +++ b/monitor/status.Dockerfile @@ -0,0 +1,19 @@ +FROM python:3.12-alpine + +WORKDIR /app +COPY status_dashboard.py /app/status_dashboard.py + +ENV DASHBOARD_PORT=8080 \ + DOCKER_PROJECT=darkproxy \ + DOCKER_SOCKET=/var/run/docker.sock \ + DNS_SERVER=darkpihole \ + DNS_PORT=53 \ + PROXY_HOST=dark3proxy \ + PROXY_PORT=1080 \ + PROXY_USERS_FILE=/run/secrets/PROXY_USERS \ + STATUS_CACHE_SECONDS=15 \ + SOCKET_TIMEOUT=5 \ + LOG_TAIL_LINES=120 + +EXPOSE 8080 +CMD ["python", "/app/status_dashboard.py"] diff --git a/monitor/status_dashboard.py b/monitor/status_dashboard.py new file mode 100644 index 0000000..c3c7cb8 --- /dev/null +++ b/monitor/status_dashboard.py @@ -0,0 +1,880 @@ +import http.client +import json +import os +import random +import re +import socket +import struct +import threading +import time +import urllib.parse +from datetime import datetime, timezone +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer + + +DASHBOARD_PORT = int(os.getenv("DASHBOARD_PORT", "8080")) +DOCKER_SOCKET = os.getenv("DOCKER_SOCKET", "/var/run/docker.sock") +DOCKER_PROJECT = os.getenv("DOCKER_PROJECT", "darkproxy") +DNS_SERVER = os.getenv("DNS_SERVER", "darkpihole") +DNS_PORT = int(os.getenv("DNS_PORT", "53")) +PROXY_HOST = os.getenv("PROXY_HOST", "dark3proxy") +PROXY_PORT = int(os.getenv("PROXY_PORT", "1080")) +PROXY_USERS_FILE = os.getenv("PROXY_USERS_FILE", "/run/secrets/PROXY_USERS") +SOCKET_TIMEOUT = float(os.getenv("SOCKET_TIMEOUT", "5")) +STATUS_CACHE_SECONDS = int(os.getenv("STATUS_CACHE_SECONDS", "15")) +LOG_TAIL_LINES = int(os.getenv("LOG_TAIL_LINES", "120")) + +ERROR_RE = re.compile(r"error|failed|not found|panic|traceback|unhealthy|refused|parse", re.IGNORECASE) +IGNORE_RE = { + "darkscale": re.compile(r"tpmrm0|Tailscale is stopped", re.IGNORECASE), + "darkpihole": re.compile(r"refused to do a recursive query", re.IGNORECASE), + "darki2p": re.compile( + r"SessionCreated read error: End of file|" + r"Connect error Operation canceled|" + r"Connect error Network is unreachable|" + r"RouterInfo for .* not found|" + r"RouterInfo not found, failed to send messages|" + r"NetDbReq: .* not found after 5 attempts", + re.IGNORECASE, + ), +} +RCODE_NAMES = { + 0: "NOERROR", + 1: "FORMERR", + 2: "SERVFAIL", + 3: "NXDOMAIN", + 4: "NOTIMP", + 5: "REFUSED", +} +DNS_CHECKS = [ + { + "name": "google.com", + "record_type": "A", + "severity": "critical", + "label": "Public DNS path", + "note": "Pi-hole -> CoreDNS -> Unbound", + }, + { + "name": "facebookcorewwwi.onion", + "record_type": "A", + "severity": "critical", + "label": ".onion resolution", + "note": "Tor DNS path", + }, + { + "name": "stats.i2p", + "record_type": "A", + "severity": "critical", + "label": ".i2p resolution", + "note": "i2pdns bridge path", + }, + { + "name": "vitalik.eth", + "record_type": "TXT", + "severity": "warning", + "label": ".eth resolution", + "note": "ENS resolver path", + }, + { + "name": "brad.zil", + "record_type": "TXT", + "severity": "warning", + "label": ".zil resolution", + "note": "Zilliqa resolver path", + }, + { + "name": "d.bit", + "record_type": "TXT", + "severity": "info", + "label": ".bit resolution", + "note": "Sync-dependent Namecoin path", + }, +] + +STATUS_CACHE = {"expires_at": 0.0, "payload": None} +CACHE_LOCK = threading.Lock() + + +class UnixSocketHTTPConnection(http.client.HTTPConnection): + def __init__(self, socket_path: str, timeout: float): + super().__init__("localhost", timeout=timeout) + self.socket_path = socket_path + + def connect(self): + self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + self.sock.settimeout(self.timeout) + self.sock.connect(self.socket_path) + + +def now_iso() -> str: + return datetime.now(timezone.utc).isoformat() + + +def docker_request(path: str) -> tuple[int, bytes]: + conn = UnixSocketHTTPConnection(DOCKER_SOCKET, timeout=SOCKET_TIMEOUT) + try: + conn.request("GET", path) + response = conn.getresponse() + return response.status, response.read() + finally: + conn.close() + + +def docker_json(path: str): + status, payload = docker_request(path) + if status >= 400: + raise RuntimeError(f"Docker API request failed for {path}: HTTP {status}") + return json.loads(payload.decode("utf-8")) + + +def docker_bytes(path: str) -> bytes: + status, payload = docker_request(path) + if status >= 400: + raise RuntimeError(f"Docker API request failed for {path}: HTTP {status}") + return payload + + +def decode_docker_stream(payload: bytes) -> str: + # Docker may frame stdout/stderr for non-TTY containers using an 8-byte header. + chunks = [] + offset = 0 + while offset + 8 <= len(payload): + if payload[offset + 1 : offset + 4] != b"\x00\x00\x00": + return payload.decode("utf-8", errors="replace") + frame_len = struct.unpack(">I", payload[offset + 4 : offset + 8])[0] + frame_start = offset + 8 + frame_end = frame_start + frame_len + if frame_end > len(payload): + return payload.decode("utf-8", errors="replace") + chunks.append(payload[frame_start:frame_end]) + offset = frame_end + if offset == len(payload): + return b"".join(chunks).decode("utf-8", errors="replace") + return payload.decode("utf-8", errors="replace") + + +def service_log_text(container_id: str, tail: int) -> str: + query = urllib.parse.urlencode({"stdout": 1, "stderr": 1, "tail": max(1, min(tail, 500))}) + payload = docker_bytes(f"/containers/{container_id}/logs?{query}") + return decode_docker_stream(payload).strip() + + +def extract_error_excerpt(name: str, log_text: str) -> list[str]: + ignore = IGNORE_RE.get(name) + matches = [] + for line in log_text.splitlines(): + if not ERROR_RE.search(line): + continue + if ignore and ignore.search(line): + continue + matches.append(line) + return matches[-6:] + + +def read_proxy_users() -> list[dict[str, str]]: + users = [] + try: + with open(PROXY_USERS_FILE, "r", encoding="utf-8") as handle: + for raw_line in handle: + line = raw_line.strip() + if not line or line.startswith("#") or ":" not in line: + continue + username, password = line.split(":", 1) + if username: + users.append({"username": username, "password": password}) + except FileNotFoundError: + return [] + return users + + +def encode_dns_name(name: str) -> bytes: + parts = name.rstrip(".").split(".") + encoded = bytearray() + for part in parts: + label = part.encode("idna") + encoded.append(len(label)) + encoded.extend(label) + encoded.append(0) + return bytes(encoded) + + +def read_dns_name(payload: bytes, offset: int) -> tuple[str, int]: + labels = [] + jumped = False + next_offset = offset + while True: + length = payload[offset] + if length == 0: + offset += 1 + if not jumped: + next_offset = offset + break + if length & 0xC0 == 0xC0: + pointer = struct.unpack(">H", payload[offset : offset + 2])[0] & 0x3FFF + offset = pointer + if not jumped: + next_offset += 2 + jumped = True + continue + offset += 1 + labels.append(payload[offset : offset + length].decode("utf-8", errors="replace")) + offset += length + if not jumped: + next_offset = offset + return ".".join(labels), next_offset + + +def parse_dns_answers(payload: bytes, qtype: int) -> tuple[str, list[str]]: + _, _, _, answer_count, _, _ = struct.unpack(">HHHHHH", payload[:12]) + offset = 12 + for _ in range(1): + _, offset = read_dns_name(payload, offset) + offset += 4 + answers = [] + for _ in range(answer_count): + _, offset = read_dns_name(payload, offset) + record_type, _, _, rdlength = struct.unpack(">HHIH", payload[offset : offset + 10]) + offset += 10 + rdata = payload[offset : offset + rdlength] + offset += rdlength + if record_type == 1 and qtype == 1 and rdlength == 4: + answers.append(socket.inet_ntoa(rdata)) + elif record_type == 16 and qtype == 16 and rdlength > 0: + strings = [] + inner_offset = 0 + while inner_offset < len(rdata): + chunk_len = rdata[inner_offset] + inner_offset += 1 + strings.append(rdata[inner_offset : inner_offset + chunk_len].decode("utf-8", errors="replace")) + inner_offset += chunk_len + answers.append("".join(strings)) + rcode = payload[3] & 0x0F + return RCODE_NAMES.get(rcode, f"RCODE_{rcode}"), answers + + +def dns_query(name: str, record_type: str) -> dict[str, object]: + qtype = 1 if record_type == "A" else 16 + query_id = random.randint(0, 65535) + flags = 0x0100 + header = struct.pack(">HHHHHH", query_id, flags, 1, 0, 0, 0) + question = encode_dns_name(name) + struct.pack(">HH", qtype, 1) + message = header + question + start = time.monotonic() + with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock: + sock.settimeout(SOCKET_TIMEOUT) + sock.sendto(message, (DNS_SERVER, DNS_PORT)) + payload, _ = sock.recvfrom(4096) + duration_ms = int((time.monotonic() - start) * 1000) + status, answers = parse_dns_answers(payload, qtype) + return {"status": status, "answers": answers, "latency_ms": duration_ms} + + +def run_dns_check(spec: dict[str, str]) -> dict[str, object]: + try: + result = dns_query(spec["name"], spec["record_type"]) + outcome = "ok" if result["status"] == "NOERROR" and result["answers"] else "failed" + if spec["severity"] == "info" and result["status"] in {"NOERROR", "NXDOMAIN"}: + outcome = "ok" + detail = result["status"] + if result["answers"]: + detail = f"{result['status']} ({', '.join(result['answers'][:2])})" + return { + "kind": "dns", + "label": spec["label"], + "target": spec["name"], + "record_type": spec["record_type"], + "severity": spec["severity"], + "outcome": outcome, + "detail": detail, + "note": spec["note"], + "latency_ms": result["latency_ms"], + } + except Exception as exc: + return { + "kind": "dns", + "label": spec["label"], + "target": spec["name"], + "record_type": spec["record_type"], + "severity": spec["severity"], + "outcome": "failed", + "detail": str(exc), + "note": spec["note"], + "latency_ms": -1, + } + + +def recv_exact(sock: socket.socket, size: int) -> bytes: + chunks = [] + remaining = size + while remaining > 0: + data = sock.recv(remaining) + if not data: + raise RuntimeError("unexpected EOF") + chunks.append(data) + remaining -= len(data) + return b"".join(chunks) + + +def socks_reply_message(code: int) -> str: + return { + 0x00: "succeeded", + 0x01: "general failure", + 0x02: "connection not allowed", + 0x03: "network unreachable", + 0x04: "host unreachable", + 0x05: "connection refused", + 0x06: "TTL expired", + 0x07: "command not supported", + 0x08: "address type not supported", + }.get(code, f"reply {code}") + + +def run_proxy_check(users: list[dict[str, str]]) -> dict[str, object]: + if not users: + return { + "kind": "proxy", + "label": "SOCKS ingress", + "target": f"{PROXY_HOST}:{PROXY_PORT}", + "severity": "warning", + "outcome": "skipped", + "detail": "No proxy users found in secret", + "note": "Shows auth + CONNECT readiness", + "latency_ms": -1, + } + user = users[0] + host = "example.com".encode("idna") + start = time.monotonic() + try: + with socket.create_connection((PROXY_HOST, PROXY_PORT), timeout=SOCKET_TIMEOUT) as sock: + sock.settimeout(SOCKET_TIMEOUT) + sock.sendall(b"\x05\x01\x02") + greeting = recv_exact(sock, 2) + if greeting != b"\x05\x02": + raise RuntimeError(f"unexpected auth method {greeting!r}") + + username = user["username"].encode("utf-8") + password = user["password"].encode("utf-8") + sock.sendall(bytes([0x01, len(username)]) + username + bytes([len(password)]) + password) + auth_reply = recv_exact(sock, 2) + if auth_reply[1] != 0x00: + raise RuntimeError("authentication failed") + + request = b"\x05\x01\x00\x03" + bytes([len(host)]) + host + struct.pack(">H", 80) + sock.sendall(request) + header = recv_exact(sock, 4) + reply_code = header[1] + addr_type = header[3] + if addr_type == 0x01: + recv_exact(sock, 4) + elif addr_type == 0x03: + domain_len = recv_exact(sock, 1)[0] + recv_exact(sock, domain_len) + elif addr_type == 0x04: + recv_exact(sock, 16) + recv_exact(sock, 2) + if reply_code != 0x00: + raise RuntimeError(socks_reply_message(reply_code)) + except Exception as exc: + return { + "kind": "proxy", + "label": "SOCKS ingress", + "target": f"{PROXY_HOST}:{PROXY_PORT}", + "severity": "critical", + "outcome": "failed", + "detail": str(exc), + "note": "Shows auth + CONNECT readiness", + "latency_ms": -1, + } + + return { + "kind": "proxy", + "label": "SOCKS ingress", + "target": f"{PROXY_HOST}:{PROXY_PORT}", + "severity": "critical", + "outcome": "ok", + "detail": f"Authenticated as {user['username']} and opened CONNECT tunnel", + "note": "Shows auth + CONNECT readiness", + "latency_ms": int((time.monotonic() - start) * 1000), + } + + +def service_severity(state: str, health: str, restart_count: int, error_excerpt: list[str]) -> str: + if state in {"exited", "dead"} or health == "unhealthy": + return "critical" + if state != "running" or health == "starting" or restart_count > 0 or error_excerpt: + return "degraded" + return "healthy" + + +def list_project_containers() -> list[dict[str, object]]: + filters = urllib.parse.quote(json.dumps({"label": [f"com.docker.compose.project={DOCKER_PROJECT}"]}, separators=(",", ":"))) + raw_containers = docker_json(f"/containers/json?all=1&filters={filters}") + services = [] + for item in raw_containers: + name = item.get("Names", [item["Id"]])[0].lstrip("/") + inspect = docker_json(f"/containers/{item['Id']}/json") + state = inspect["State"]["Status"] + health = inspect["State"].get("Health", {}).get("Status", "none") + restart_count = inspect.get("RestartCount", 0) + logs = service_log_text(item["Id"], LOG_TAIL_LINES) + error_excerpt = extract_error_excerpt(name, logs) + services.append( + { + "id": item["Id"], + "service": item.get("Labels", {}).get("com.docker.compose.service", name), + "container_name": name, + "state": state, + "status_text": item.get("Status", state), + "health": health, + "restart_count": restart_count, + "image": item.get("Image", ""), + "ports": item.get("Ports", []), + "severity": service_severity(state, health, restart_count, error_excerpt), + "error_excerpt": error_excerpt, + } + ) + return sorted(services, key=lambda svc: svc["service"]) + + +def summarize_status(services: list[dict[str, object]], checks: list[dict[str, object]]) -> dict[str, object]: + critical_services = sum(1 for svc in services if svc["severity"] == "critical") + degraded_services = sum(1 for svc in services if svc["severity"] == "degraded") + failed_critical_checks = sum(1 for check in checks if check["severity"] == "critical" and check["outcome"] != "ok") + failed_warning_checks = sum(1 for check in checks if check["severity"] == "warning" and check["outcome"] != "ok") + + if critical_services or failed_critical_checks: + overall = "critical" + elif degraded_services or failed_warning_checks: + overall = "degraded" + else: + overall = "healthy" + + return { + "overall": overall, + "service_counts": { + "total": len(services), + "healthy": sum(1 for svc in services if svc["severity"] == "healthy"), + "degraded": degraded_services, + "critical": critical_services, + }, + "check_counts": { + "total": len(checks), + "ok": sum(1 for check in checks if check["outcome"] == "ok"), + "failed": sum(1 for check in checks if check["outcome"] == "failed"), + "skipped": sum(1 for check in checks if check["outcome"] == "skipped"), + }, + } + + +def build_status_payload() -> dict[str, object]: + services = list_project_containers() + users = read_proxy_users() + checks = [run_dns_check(spec) for spec in DNS_CHECKS] + checks.append(run_proxy_check(users)) + summary = summarize_status(services, checks) + return { + "generated_at": now_iso(), + "project": DOCKER_PROJECT, + "summary": summary, + "services": services, + "checks": checks, + "users": [{"username": entry["username"]} for entry in users], + } + + +def get_status_payload(force_refresh: bool = False) -> dict[str, object]: + now = time.monotonic() + with CACHE_LOCK: + if not force_refresh and STATUS_CACHE["payload"] is not None and now < STATUS_CACHE["expires_at"]: + return STATUS_CACHE["payload"] + payload = build_status_payload() + STATUS_CACHE["payload"] = payload + STATUS_CACHE["expires_at"] = now + STATUS_CACHE_SECONDS + return payload + + +def find_service(identifier: str) -> dict[str, object] | None: + payload = get_status_payload() + for service in payload["services"]: + if service["service"] == identifier or service["container_name"] == identifier: + return service + return None + + +HTML = """ + + + + + darkproxy status + + + +
+
+
+

darkproxy status

+
Loading…
+
+ + JSON +
+ +
+ +
+
+

Services

+ + + + + + + + + + + + +
ServiceStateHealthRestartsErrors
+
+ +
+

Users

+
Configured proxy usernames only. Passwords are never shown.
+
    +
    +
    + +
    +

    Checks

    + + + + + + + + + + + + +
    CheckTargetSeverityResultLatencyDetail
    +
    + +
    +
    +

    Logs

    + + +
    + 
    Select a service to view recent logs.
    +
    +
    + + + + +""" + + +class Handler(BaseHTTPRequestHandler): + def send_json(self, payload: dict[str, object], status: int = 200): + body = json.dumps(payload, indent=2).encode("utf-8") + self.send_response(status) + self.send_header("Content-Type", "application/json; charset=utf-8") + self.send_header("Content-Length", str(len(body))) + self.end_headers() + try: + self.wfile.write(body) + except BrokenPipeError: + return + + def send_html(self, body: str, status: int = 200): + payload = body.encode("utf-8") + self.send_response(status) + self.send_header("Content-Type", "text/html; charset=utf-8") + self.send_header("Content-Length", str(len(payload))) + self.end_headers() + try: + self.wfile.write(payload) + except BrokenPipeError: + return + + def do_GET(self): + parsed = urllib.parse.urlparse(self.path) + params = urllib.parse.parse_qs(parsed.query) + try: + if parsed.path == "/": + self.send_html(HTML) + return + if parsed.path == "/api/status": + force_refresh = params.get("refresh", ["0"])[0] == "1" + self.send_json(get_status_payload(force_refresh=force_refresh)) + return + if parsed.path == "/healthz": + self.send_json({"ok": True, "generated_at": now_iso()}) + return + if parsed.path == "/api/users": + users = [{"username": entry["username"]} for entry in read_proxy_users()] + self.send_json({"users": users}) + return + if parsed.path == "/api/logs": + service_name = params.get("service", [""])[0] + if not service_name: + self.send_json({"error": "missing service parameter"}, status=400) + return + service = find_service(service_name) + if service is None: + self.send_json({"error": f"unknown service {service_name}"}, status=404) + return + tail = int(params.get("tail", ["200"])[0]) + self.send_json( + { + "service": service["service"], + "container_name": service["container_name"], + "logs": service_log_text(service["id"], tail), + } + ) + return + self.send_json({"error": "not found"}, status=404) + except BrokenPipeError: + return + except Exception as exc: + self.send_json({"error": str(exc)}, status=500) + + def log_message(self, format, *args): + return + + +if __name__ == "__main__": + server = ThreadingHTTPServer(("0.0.0.0", DASHBOARD_PORT), Handler) + server.serve_forever() diff --git a/scripts/validate-config.sh b/scripts/validate-config.sh index 40688ed..3699ba1 100644 --- a/scripts/validate-config.sh +++ b/scripts/validate-config.sh @@ -40,7 +40,7 @@ if command -v docker >/dev/null 2>&1; then exit 1 fi set +e - timeout 8s docker run --rm "$COREDNS_IMAGE" -conf /Corefile -dns.port=0 + timeout -k 2s 8s docker run --rm "$COREDNS_IMAGE" -conf /Corefile -dns.port=0 COREDNS_STATUS=$? set -e if [ "$COREDNS_STATUS" -ne 0 ] && [ "$COREDNS_STATUS" -ne 124 ]; then @@ -50,7 +50,7 @@ if command -v docker >/dev/null 2>&1; then elif command -v coredns >/dev/null 2>&1 && coredns -plugins 2>/dev/null | grep -q '^meshname$'; then echo "[validate-config] checking PopuraDNS/Corefile with local compiled coredns" set +e - timeout 8s coredns -conf PopuraDNS/Corefile -dns.port=0 + timeout -k 2s 8s coredns -conf PopuraDNS/Corefile -dns.port=0 COREDNS_STATUS=$? set -e if [ "$COREDNS_STATUS" -ne 0 ] && [ "$COREDNS_STATUS" -ne 124 ]; then @@ -61,6 +61,22 @@ else echo "[validate-config] cannot check Corefile syntax (need docker or a compiled local coredns with meshname plugin)" fi +# enforce portable defaults for host/LAN-specific settings in the base stack +if ! grep -q 'ROUTER_NO_AUTH_CIDRS: ${ROUTER_NO_AUTH_CIDRS:-}' docker-compose.yml; then + echo "[validate-config] error: ROUTER_NO_AUTH_CIDRS should default to empty and be overridden explicitly per deployment" >&2 + exit 1 +fi + +if ! grep -q '\${PIHOLE_WEB_BIND_HOST:-127.0.0.1}:\${PIHOLE_WEB_BIND_PORT:-2003}:80/tcp' docker-compose.yml; then + echo "[validate-config] error: Pi-hole web binding should default to 127.0.0.1 and be configurable via env vars" >&2 + exit 1 +fi + +if ! grep -q '\${STATUS_DASHBOARD_BIND_HOST:-127.0.0.1}:\${STATUS_DASHBOARD_BIND_PORT:-2004}:8080' docker-compose.yml; then + echo "[validate-config] error: status dashboard binding should default to 127.0.0.1 and be configurable via env vars" >&2 + exit 1 +fi + # ensure onion and i2p zones are present in Corefile for zone in "onion.:53" "i2p.:53" "eth.:53" "bit.:53" "alt.:53" "loki.:53" "zil.:53" "web3.:53" "exit.:53" "onion4.:53" "onion6.:53"; do if ! grep -q "$zone" PopuraDNS/Corefile; then @@ -100,6 +116,16 @@ if command -v dig >/dev/null 2>&1; then echo "[validate-config] i2p query failed" >&2 exit 1 fi + echo "[validate-config] performing live DNS query for .eth via CoreDNS" + if ! dig @10.5.0.4 -p 53 TXT vitalik.eth +short | grep -q 'address='; then + echo "[validate-config] eth query failed" >&2 + exit 1 + fi + echo "[validate-config] performing live DNS query for .zil via CoreDNS" + if ! dig @10.5.0.4 -p 53 TXT brad.zil +short | grep -q 'zil_address='; then + echo "[validate-config] zil query failed" >&2 + exit 1 + fi fi # ensure referenced secrets exist @@ -117,8 +143,8 @@ done # check for mutable :latest tags in deployment compose files if grep -qE 'image: .*:latest' docker-compose.yml; then - echo "[validate-config] warning: some services use the ':latest' image tag;" \ - "pin to a specific version before deploying to production." + echo "[validate-config] error: mutable ':latest' image tags are not allowed in the release compose file" >&2 + exit 1 fi # warn if there are no healthchecks at all @@ -139,15 +165,19 @@ fi # unbound configuration syntax check if command -v docker >/dev/null 2>&1; then - echo "[validate-config] checking unbound configuration in compiled repo image" - UNBOUND_IMAGE="$(docker build -q -f unbound_arm/Dockerfile unbound_arm | tail -n1)" + echo "[validate-config] checking unbound configuration in runtime image" + UNBOUND_IMAGE="$(awk '/image: mvance\/unbound@sha256:/{print $2; exit}' "$TEMP_FILE")" + if [ -z "$UNBOUND_IMAGE" ]; then + echo "[validate-config] error: failed to resolve unbound runtime image from rendered compose config" >&2 + exit 1 + fi docker run --rm --entrypoint /bin/sh \ -v "${PWD}/unbound/unbound.conf:/opt/unbound/etc/unbound/unbound.conf:ro" \ -v "${PWD}/unbound/forward-records.conf:/opt/unbound/etc/unbound/forward-records.conf:ro" \ -v "${PWD}/unbound/a-records.conf:/opt/unbound/etc/unbound/a-records.conf:ro" \ -v "${PWD}/unbound/srv-records.conf:/opt/unbound/etc/unbound/srv-records.conf:ro" \ "$UNBOUND_IMAGE" -lc \ - 'getent passwd _unbound >/dev/null || useradd -r -s /usr/sbin/nologin _unbound; unbound-checkconf /opt/unbound/etc/unbound/unbound.conf' + 'getent passwd _unbound >/dev/null || useradd -r -s /usr/sbin/nologin _unbound; mkdir -p /opt/unbound/etc/unbound/dev /opt/unbound/etc/unbound/var/log; : > /opt/unbound/etc/unbound/dev/null; [ -s /opt/unbound/etc/unbound/var/root.key ] || /opt/unbound/sbin/unbound-anchor -a /opt/unbound/etc/unbound/var/root.key >/dev/null 2>&1 || :; [ -e /opt/unbound/etc/unbound/var/root.key ] || : > /opt/unbound/etc/unbound/var/root.key; /opt/unbound/sbin/unbound-checkconf /opt/unbound/etc/unbound/unbound.conf' elif command -v unbound-checkconf >/dev/null 2>&1; then echo "[validate-config] checking unbound configuration locally" unbound-checkconf unbound/unbound.conf @@ -177,7 +207,13 @@ if grep -q 'CHANGE_ME_NAMECOIN_RPC_PASSWORD' docker-compose.yml; then fi if grep -q 'tskey-YOUR-AUTH-KEY-HERE' docker-compose.yml; then - echo "[validate-config] warning: tailscale is still configured with the placeholder auth key" + echo "[validate-config] error: tailscale placeholder auth key is still present in docker-compose.yml" >&2 + exit 1 +fi + +if ! grep -q 'profiles: \["tailscale"\]' docker-compose.yml; then + echo "[validate-config] error: tailscale should be opt-in via a compose profile" >&2 + exit 1 fi # simple sanity checks @@ -187,9 +223,24 @@ if grep -q "container_name: darkproxy" docker-compose.yml; then fi # ensure Pi-hole DNS environment is correctly formatted -if grep -q "PIHOLE_DNS_" docker-compose.yml | grep -q "PIHOLE_DNS_:"; then +if grep -q "PIHOLE_DNS_:" docker-compose.yml; then echo "[validate-config] warning: PIHOLE_DNS_ variable ends with underscore;" \ - "consider using PIHOLE_DNS_1, PIHOLE_DNS_2 etc." + "consider using PIHOLE_DNS_1, PIHOLE_DNS_2 etc." +fi + +# ensure critical services have healthchecks in the rendered compose config +if [ -f "$TEMP_FILE" ]; then + for service in dark3proxy i2pd_yggdrasil i2pdns unbound pihole ensdns zildns lokinet namecoind namecoindns status_dashboard; do + if ! awk -v svc="$service" ' + $0 ~ "^ " svc ":" { in_svc=1; next } + in_svc && $0 ~ "^ [^ ]" { exit !found } + in_svc && $0 ~ "^ healthcheck:" { found=1 } + END { if (in_svc) exit !found; exit 0 } + ' "$TEMP_FILE"; then + echo "[validate-config] error: service $service is missing a healthcheck in rendered compose config" >&2 + exit 1 + fi + done fi echo "[validate-config] all validations passed" diff --git a/tailscale/post-rules.sh b/tailscale/post-rules.sh index 670d24a..4f2db29 100644 --- a/tailscale/post-rules.sh +++ b/tailscale/post-rules.sh @@ -8,14 +8,16 @@ TAILSCALED_PID=$! # Wait for Tailscale to initialize sleep 2 -# Authenticate with Tailscale via auth key -if [ -n "$TS_AUTHKEY" ] && [ "$TS_AUTHKEY" != "tskey-YOUR-AUTH-KEY-HERE" ]; then - # tailscale up flags change over time; avoid deprecated options. - /usr/local/bin/tailscale up --authkey="$TS_AUTHKEY" ${TS_EXTRA_ARGS:-"--accept-dns=false --advertise-exit-node"} || true -else - echo "TS_AUTHKEY is not set to a real key; skipping tailscale up" +if [ -z "${TS_AUTHKEY:-}" ]; then + echo "TS_AUTHKEY must be set when enabling the tailscale profile" >&2 + kill "$TAILSCALED_PID" + wait "$TAILSCALED_PID" || true + exit 1 fi +# tailscale up flags change over time; avoid deprecated options. +/usr/local/bin/tailscale up --authkey="$TS_AUTHKEY" ${TS_EXTRA_ARGS:-"--accept-dns=false --advertise-exit-node"} + # Wait a bit for Tailscale to be ready sleep 3 diff --git a/zildns/Dockerfile b/zildns/Dockerfile index 432ba51..06d204a 100644 --- a/zildns/Dockerfile +++ b/zildns/Dockerfile @@ -6,8 +6,10 @@ COPY package.json /app/package.json RUN npm install --omit=dev COPY server.js /app/server.js +COPY entrypoint.sh /app/entrypoint.sh +RUN chmod +x /app/entrypoint.sh EXPOSE 53/udp EXPOSE 53/tcp -CMD ["node", "/app/server.js"] +CMD ["/app/entrypoint.sh"] diff --git a/zildns/entrypoint.sh b/zildns/entrypoint.sh new file mode 100644 index 0000000..17f47b4 --- /dev/null +++ b/zildns/entrypoint.sh @@ -0,0 +1,9 @@ +#!/bin/sh +set -e + +# Fix Docker's broken internal DNS by using external DNS directly +echo "nameserver 1.1.1.1" > /etc/resolv.conf +echo "nameserver 8.8.8.8" >> /etc/resolv.conf + +# Start the Node.js server +exec node /app/server.js diff --git a/zildns/server.js b/zildns/server.js index 7444185..d1cbaff 100644 --- a/zildns/server.js +++ b/zildns/server.js @@ -12,7 +12,7 @@ const ZNS_URL = process.env.ZILDNS_ZNS_URL || "https://api.zilliqa.com"; const ZNS_NETWORK = process.env.ZILDNS_ZNS_NETWORK || "mainnet"; const PREWARM_DOMAIN = normalizePrewarmDomain(process.env.ZILDNS_PREWARM_DOMAIN || "brad.zil"); -const { Packet, createServer } = dns2; +const { Packet, UDPServer, TCPServer } = dns2; const RCODE = { NOERROR: 0, SERVFAIL: 2, @@ -58,32 +58,53 @@ function cacheSet(key, value) { }); } +function withTimeout(promise, ms) { + return Promise.race([ + promise, + new Promise((_, reject) => + setTimeout(() => reject(new Error(`Timeout after ${ms}ms`)), ms) + ) + ]); +} + async function resolveZilDomain(domain) { const cached = cacheGet(domain); if (cached !== null) { return cached; } - const [zilResult, ipfsResult] = await Promise.allSettled([ - resolution.addr(domain, "ZIL"), - resolution.ipfsHash(domain), - ]); + try { + const [zilResult, ipfsResult] = await withTimeout( + Promise.allSettled([ + resolution.addr(domain, "ZIL"), + resolution.ipfsHash(domain), + ]), + 5000 // 5 second timeout + ); - const result = { - zilAddress: zilResult.status === "fulfilled" && zilResult.value ? zilResult.value : null, - ipfsHash: ipfsResult.status === "fulfilled" && ipfsResult.value ? ipfsResult.value : null, - }; + const result = { + zilAddress: zilResult.status === "fulfilled" && zilResult.value ? zilResult.value : null, + ipfsHash: ipfsResult.status === "fulfilled" && ipfsResult.value ? ipfsResult.value : null, + }; - const hasData = Boolean(result.zilAddress || result.ipfsHash); - const resolved = hasData ? result : null; - cacheSet(domain, resolved); - return resolved; + const hasData = Boolean(result.zilAddress || result.ipfsHash); + const resolved = hasData ? result : null; + cacheSet(domain, resolved); + return resolved; + } catch (err) { + console.error(`[zildns] API failed for ${domain}:`, err.message); + // Cache negative result for a short time to avoid repeated API failures + cacheSet(domain, null); + return null; + } } function makeResponse(request) { const response = Packet.createResponseFromRequest(request); response.header.aa = 1; response.header.ra = 1; + response.header.z = 0; + response.header.tc = 0; return response; } @@ -123,13 +144,12 @@ function addCnameAnswer(response, name, resolved) { }); } -async function handleRequest(request, send) { +async function buildResponse(request) { const response = makeResponse(request); const question = request.questions && request.questions[0]; if (!question) { - send(response); - return; + return response; } const qname = normalizeName(question.name); @@ -138,15 +158,13 @@ async function handleRequest(request, send) { if (!qname.endsWith(".zil")) { response.header.rcode = RCODE.NXDOMAIN; - send(response); - return; + return response; } const resolved = await resolveZilDomain(qname); if (!resolved) { response.header.rcode = RCODE.NXDOMAIN; - send(response); - return; + return response; } if (isType("TXT") || isType("ANY")) { @@ -157,35 +175,9 @@ async function handleRequest(request, send) { addCnameAnswer(response, question.name, resolved); } - send(response); + return response; } -const server = createServer({ - udp: true, - tcp: true, - handle: (request, send) => { - handleRequest(request, send).catch((err) => { - const response = makeResponse(request); - response.header.rcode = RCODE.SERVFAIL; - send(response); - console.error("[zildns] request failed:", err && err.message ? err.message : String(err)); - }); - }, -}); - -server.on("listening", () => { - console.log(`[zildns] listening on ${LISTEN_HOST}:${LISTEN_PORT} (udp/tcp)`); - console.log(`[zildns] zns provider ${ZNS_URL} (${ZNS_NETWORK})`); -}); - -server.on("close", () => { - console.log("[zildns] server closed"); -}); - -server.on("error", (err) => { - console.error("[zildns] server error:", err); -}); - async function maybePrewarm() { if (!PREWARM_DOMAIN) { console.log("[zildns] prewarm disabled"); @@ -206,17 +198,48 @@ async function maybePrewarm() { async function start() { await maybePrewarm(); - - server.listen({ - udp: { - address: LISTEN_HOST, - port: LISTEN_PORT, - }, - tcp: { - address: LISTEN_HOST, - port: LISTEN_PORT, - }, + + const udpServer = new UDPServer((request, send) => { + buildResponse(request) + .then((response) => { + send(response); + }) + .catch((err) => { + console.error("[zildns] request failed:", err.message); + const response = makeResponse(request); + response.header.rcode = RCODE.SERVFAIL; + send(response); + }); }); + + const tcpServer = new TCPServer((request, send) => { + buildResponse(request) + .then((response) => { + send(response); + }) + .catch((err) => { + console.error("[zildns] request failed:", err.message); + const response = makeResponse(request); + response.header.rcode = RCODE.SERVFAIL; + send(response); + }); + }); + + udpServer.on("error", (err) => { + console.error("[zildns] udp error:", err); + }); + + tcpServer.on("error", (err) => { + console.error("[zildns] tcp error:", err); + }); + + udpServer.listen(LISTEN_PORT, LISTEN_HOST); + tcpServer.listen(LISTEN_PORT, LISTEN_HOST); + + console.log(`[zildns] listening on ${LISTEN_HOST}:${LISTEN_PORT} (udp/tcp)`); + console.log(`[zildns] zns provider ${ZNS_URL} (${ZNS_NETWORK})`); + + return { udpServer, tcpServer }; } start().catch((err) => {