# `utils/lxc.sh`

With the use of Linux Containers (LXC) we can scale our tasks over a stack of containers, what we call the: lxc suite. The SearXNG suite config is loaded by default, every time you start the `lxc.sh` script (you do not need to care about).

## Install LXD

Before you can start with containers, you need to install and initiate LXD once:

    $ snap install lxd
    $ lxd init --auto

To make use of the containers from the SearXNG suite, you have to build the LXC suite containers initial. But be warned, this might take some time:

    $ sudo -H ./utils/lxc.sh build

A cup of coffee later, your LXC suite is build up and you can run whatever task you want / in a selected or even in all LXC suite containers.

### Internet Connectivity & Docker

There is a conflict in the `iptables` setup of Docker & LXC. If you have docker installed, you may find that the internet connectivity of your LXD containers no longer work.

Whenever docker is started (reboot) it sets the iptables policy for the `FORWARD` chain to `DROP` [ref]:

    $ sudo -H iptables-save | grep FORWARD
    :FORWARD ACCEPT [7048:7851230]
    :FORWARD DROP [7048:7851230]

A handy solution of this problem might be to reset the policy for the `FORWARD` chain after the network has been initialized. For this create a file in the `if-up` section of the network (`/etc/network/if-up.d/iptable`) and insert the following lines:

    #!/bin/sh
    iptables -F FORWARD
    iptables -P FORWARD ACCEPT

Don’t forget to set the execution bit:

    sudo chmod ugo+x /etc/network/if-up.d/iptable

Reboot your system and check the iptables rules:

    $ sudo -H iptables-save | grep FORWARD
    :FORWARD ACCEPT [7048:7851230]
    :FORWARD ACCEPT [7048:7851230]

## SearXNG LXC suite

The intention of the SearXNG LXC suite is to build up a suite of containers for development tasks or buildhosts with a very small set of simple commands. At the end of the `--help` output the SearXNG suite from the SearXNG suite config is introduced:

    $ sudo -H ./utils/lxc.sh --help
    ...
    LXC suite: searxng
      Suite includes installation of SearXNG
      images:     ubu2004 ubu2204 fedora35 archlinux
      containers: searxng-ubu2004 searxng-ubu2204 searxng-fedora35 searxng-archlinux

As shown above there are images and containers build up on this images. To show more info about the containers in the SearXNG LXC suite call `show suite`. If this is the first time you make use of the SearXNG LXC suite, no containers are installed and the output is:

    $ sudo -H ./utils/lxc.sh show suite

    LXC suite (searxng-*)
    =====================

    +------+-------+------+------+------+-----------+
    | NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
    +------+-------+------+------+------+-----------+

    WARN:  container searxng-ubu2004 does not yet exists
    WARN:  container searxng-ubu2204 does not yet exists
    WARN:  container searxng-fedora35 does not yet exists
    WARN:  container searxng-archlinux does not yet exists

If you do not want to run a command or a build in all containers, you can build just one. Here by example in the container that is build upon the archlinux image:

    $ sudo -H ./utils/lxc.sh build searxng-archlinux
    $ sudo -H ./utils/lxc.sh cmd searxng-archlinux pwd

Otherwise, to apply a command to all containers you can use:

    $ sudo -H ./utils/lxc.sh build
    $ sudo -H ./utils/lxc.sh cmd -- ls -la .

### Running commands

Inside containers, you can run scripts from the DevOps tooling box or run what ever command you need. By example, to start a bash use:

    $ sudo -H ./utils/lxc.sh cmd searxng-archlinux bash
    INFO:  [searxng-archlinux] bash
    [root@searxng-archlinux SearXNG]#

### Good to know

Each container shares the root folder of the repository and the command `utils/lxc.sh cmd` handle relative path names transparent:

    $ pwd
    /share/SearXNG

    $ sudo -H ./utils/lxc.sh cmd searxng-archlinux pwd
    INFO:  [searxng-archlinux] pwd
    /share/SearXNG

The path `/share/SearXNG` will be different on your HOST system. The commands in the container are executed by the `root` inside of the container. Compare output of:

    $ ls -li Makefile
    47712402 -rw-rw-r-- 1 markus markus 2923 Apr 19 13:52 Makefile

    $ sudo -H ./utils/lxc.sh cmd searxng-archlinux ls -li Makefile
    INFO:  [searxng-archlinux] ls -li Makefile
    47712402 -rw-rw-r-- 1 root root 2923 Apr 19 11:52 Makefile
    ...

Since the path `/share/SearXNG` of the HOST system is wrapped into the container under the same name, the shown `Makefile` (inode `47712402`) in the output is always the identical `/share/SearXNG/Makefile` from the HOST system. In the example shown above the owner of the path in the container is the `root` user of the container (and the timezone in the container is different to HOST system).

### Install suite

To install the complete SearXNG suite into all LXC containers leave the container argument empty and run:

    $ sudo -H ./utils/lxc.sh build
    $ sudo -H ./utils/lxc.sh install suite

To build & install suite only in one container you can use by example:

    $ sudo -H ./utils/lxc.sh build searxng-archlinux
    $ sudo -H ./utils/lxc.sh install suite searxng-archlinux

The command above installs a SearXNG suite (see Installation Script). To install a nginx reverse proxy (or alternatively use apache):

    $ sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx

Same operation just in one container of the suite:

    $ sudo -H ./utils/lxc.sh cmd searxng-archlinux FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx

The FORCE_TIMEOUT environment is set to zero to run the script without user interaction.

To get the IP (URL) of the SearXNG service in the containers use `show suite` command. To test instances from containers just open the URLs in your WEB-Browser:

    $ sudo ./utils/lxc.sh show suite | grep SEARXNG_URL

    [searxng-ubu2110]      SEARXNG_URL          : http://n.n.n.170/searxng
    [searxng-ubu2004]      SEARXNG_URL          : http://n.n.n.160/searxng
    [searxnggfedora35]     SEARXNG_URL          : http://n.n.n.150/searxng
    [searxng-archlinux]    SEARXNG_URL          : http://n.n.n.140/searxng

### Clean up

If there comes the time you want to get rid off all the containers and clean up local images just type:

    $ sudo -H ./utils/lxc.sh remove
    $ sudo -H ./utils/lxc.sh remove images

## Setup SearXNG buildhost

You can install the SearXNG buildhost environment into one or all containers. The installation procedure to set up a build host takes its time. Installation in all containers will take more time (time for another cup of coffee).

    sudo -H ./utils/lxc.sh cmd -- ./utils/searxng.sh install buildhost

To build (live) documentation inside a archlinux container:

    sudo -H ./utils/lxc.sh cmd searxng-archlinux make docs.clean docs.live
    ...
    [I 200331 15:00:42 server:296] Serving on http://0.0.0.0:8080

To get IP of the container and the port number live docs is listening:

    $ sudo ./utils/lxc.sh show suite | grep docs.live
    ...
    [searxng-archlinux]  INFO:  (eth0) docs.live:  http://n.n.n.140:8080/

## Command Help

The `--help` output of the script is largely self-explanatory:

    usage::
      lxc.sh build        [containers|<name>]
      lxc.sh copy         [images]
      lxc.sh remove       [containers|<name>|images]
      lxc.sh [start|stop] [containers|<name>]
      lxc.sh show         [images|suite|info|config [<name>]]
      lxc.sh cmd          [--|<name>] '...'
      lxc.sh install      [suite|base [<name>]]

    build
      :containers:   build, launch all containers and 'install base' packages
      :<name>:       build, launch container <name>  and 'install base' packages
    copy:
      :images:       copy remote images of the suite into local storage
    remove
      :containers:   delete all 'containers' or only <container-name>
      :images:       delete local images of the suite
    start/stop
      :containers:   start/stop all 'containers' from the suite
      :<name>:       start/stop container <name> from suite
    show
      :info:         show info of all (or <name>) containers from LXC suite
      :config:       show config of all (or <name>) containers from the LXC suite
      :suite:        show services of all (or <name>) containers from the LXC suite
      :images:       show information of local images
    cmd
      use single quotes to evaluate in container's bash, e.g.: 'echo $(hostname)'
      --             run command '...' in all containers of the LXC suite
      :<name>:       run command '...' in container <name>
    install
      :base:         prepare LXC; install basic packages
      :suite:        install LXC searxng suite into all (or <name>) containers

    LXC suite: searxng
      Suite includes installation of SearXNG
      images:     ubu2004 ubu2204 fedora35 archlinux
      containers: searxng-ubu2004 searxng-ubu2204 searxng-fedora35 searxng-archlinux

## SearXNG suite config

The SearXNG suite is defined in the file git://utils/lxc-searxng.env:

    # -*- coding: utf-8; mode: sh indent-tabs-mode: nil -*-
    # SPDX-License-Identifier: AGPL-3.0-or-later
    # shellcheck shell=bash

    # This file is a setup of a LXC suite.  It is sourced from different context, do
    # not manipulate the environment directly, implement functions and manipulate
    # environment only in subshells.

    lxc_set_suite_env() {

        export LXC_SUITE_NAME="searxng"

        # name of https://images.linuxcontainers.org
        export LINUXCONTAINERS_ORG_NAME="${LINUXCONTAINERS_ORG_NAME:-images}"
        export LXC_HOST_PREFIX="${LXC_SUITE_NAME:-searx}"
        export LXC_SUITE=(

            # end of standard support see https://wiki.ubuntu.com/Releases
            "$LINUXCONTAINERS_ORG_NAME:ubuntu/20.04"  "ubu2004" # LTS EOSS April 2025
            "$LINUXCONTAINERS_ORG_NAME:ubuntu/22.04"  "ubu2204" # LTS EOSS April 2027

            # EOL see https://fedoraproject.org/wiki/Releases
            "$LINUXCONTAINERS_ORG_NAME:fedora/35"     "fedora35"

            # rolling releases see https://www.archlinux.org/releng/releases/
            "$LINUXCONTAINERS_ORG_NAME:archlinux"     "archlinux"
        )
    }

    lxc_suite_install_info() {
        (
            lxc_set_suite_env
            cat <<EOF
    LXC suite: ${LXC_SUITE_NAME}
      Suite includes installation of SearXNG
      images:     ${LOCAL_IMAGES[*]}
      containers: ${CONTAINERS[*]}
    EOF
        )
    }

    lxc_suite_install() {
        (
            lxc_set_suite_env
            FORCE_TIMEOUT=0 "${LXC_REPO_ROOT}/utils/searxng.sh" install all
            rst_title "Suite installation finished ($(hostname))" part
            if ask_yn "Developer install? (wraps source from HOST into the running instance)" Yn; then
                "${LXC_REPO_ROOT}/utils/searxng.sh" searxng.install.link_src "$(pwd)"
            fi
            lxc_suite_info
            echo
        )
    }

    lxc_suite_info() {
        (
            lxc_set_suite_env
            for ip in $(global_IPs) ; do
                if [[ $ip =~ .*:.* ]]; then
                    info_msg "(${ip%|*}) IPv6:       http://[${ip#*|}]"
                else
                    # IPv4:
                    # shellcheck disable=SC2034,SC2031
                    info_msg "(${ip%|*}) docs-live:  http://${ip#*|}:8080/"
                fi
            done
            "${LXC_REPO_ROOT}/utils/searxng.sh" searxng.instance.env
        )
    }