Sandbox Config

This config resides in /etc/lithos/sandboxes/NAME.yaml (by default). Where NAME is the name of a sandbox.

The configuration file contains security and resource limits for the container. Including:

  • A directory where image resides
  • Set of directories that are mounted inside the container (i.e. all writable directories for the container, the /tmp…)
  • User and group limits

See overview for guidelines.

Reference

config-file

The path for the processes config. In most cases should be left unset. Default is null which is results into /etc/lithos/processes/NAME.yaml with all other settings being defaults.

image-dir

Directory where application images are. Every subdir of the image-dir may be mounted as a root file system in the container. Required.

image-dir-levels

(default 1) A number of directory components required for image name in image-dir

log-file

The file name where to put supervisor log of the container. Default is /var/log/lithos/SANDBOX_NAME.yaml.

log-level

(default warn). The logging level of the supervisor.

readonly-paths

The mapping of virtual_directory: host_system_directory of folders which are visible for the container in read-only mode. (Note currently if you have submounts in the source directory, thay may be available as writeable). See Volumes for more details.

writable-paths

The mapping of virtual_directory: host_system_directory of folders which are visible for the container in writable mode. See Volumes for more details.

allow-users

List of ranges of user ids which can be used by container. For containers without user namespaces, it’s just a limit of the user-id setting.

Example:

allow-users: [1, 99, 1000-2000]

For containers which have uid maps enabled in sandbox this is a list of users available after uid mapping applied. For example, the following maps uid 100000 as root in namespace (e.g. for file permissions), but doesn’t allow to start process as root (even if it’s 100000 ouside):

uid-map: [{outside: 100000, inside: 0, count: 65536}]
allow-users: [1-65535]

For containers which do have uid maps enabled in container config, it limits all the user ids available to the namespace (i.e. for the outside setting of the uid map).

default-user

(no default) A user id used in the container if no user-id is specified in container config. By default user-id is required.

Note: default-user value must be contained in the allow-users range

allow-groups

List of ranges of group ids for the container. Works similarly to allow-users.

default-group

(default 0) A group id used in the container if no group-id is specified in container config.

Note: default-group value must be contained in the allow-users range

allow-tcp-ports

List of ranges of allowed TCP ports for container. This is currently not enforced in any way except:

  1. Ports < 1024 are restricted by OS for non-root (but may be allowed here)
  2. It restricts bind-port setting in container config

Note

if you have overlapping TCP port for different sandboxes, only single file descriptor will be used for each port. The config for opening port will be used arbitrary from single config amonst all users, which have obvious security implications.

Warning

tcp-ports bind at port in host namespace, i.e. it effectively discards bridged-network for that port this is both the feature and might be a pitfall. So most of the time you should avoid non-empty allow-tcp-ports if using bridged-network.

additional-hosts

Mapping of hostname: ip for names that will be added to /etc/hosts file. This is occasinally used for cheap but static service discovery.

uid-map, gid-map

The list of mapping for uids(gids) in the user namespace of the container. If they are not specified the user namespace is not used. This setting allows to run processes with uid zero without the risk of being the root on host system.

Here is a example of maps:

uid-map:
- {inside: 0, outside: 1000, count: 1}
- {inside: 1, outside: 1, count: 1}
gid-map:
- {inside: 0, outside: 100, count: 1}

Note

Currently you may have uid-map either in a sandbox or in a container config, not both.

used-images-list

(optional) A text file that is used by lithos_clean to keep images alive. It’s not used by any other means except lithos_clean utility.

Each line of the file should contain image name relative to the image_dir.

It’s expected that the list is kept up by some orchestration system or by deployment scripts or by any other tool meaningful for ops team.

This setting is only useful if auto-clean is true (default)

auto-clean

(default true) Clean images of this sandbox when running lithos_clean. This is a subject of the following caveats:

  1. Lithos clean is not run by lithos automatically, you ought to run it using cron tab
  2. If same image-dir is used for multiple sandboxes it will be cleaned if at least one of them has non-falsy auto-clean.
resolv-conf

(default /etc/resolv.conf) default place to copy resolv.conf from for containers.

Note: Container itself can override it’s own resolv.conf file, but can’t read original /etc/resolv.conf if this setting is changed.

hosts-file

(default /etc/hosts) default place to copy hosts from for containers.

Note: Container itself can override it’s own hosts file, but can’t read original /etc/hosts if this setting is changed.

bridged-network

(default is absent) a network bridge configuration for all the cotainers in the bridge

Example:

bridged-network:
  bridge: br0
  network: 10.0.0.0/24
  default_gateway: 10.0.0.1
  after-setup-command: [/usr/bin/arping, -U, -c1, '@{container_ip}']

Note

when bridged network is active your Process Config should contain a list of ip addresses one for each container.

Note

this setting does not affect tcp-ports. So usually you should keep allow-tcp-ports setting empty when using bridged network.

Options:

after-setup-command

Command to run after setting up container namespace but before running actual container. The example shown above sends unsolicited arp packet to notify router and other machines on the network that MAC address corresponding to container’s IP is changed.

Command must have absolute path, and has almost empty environment, so don’t assume PATH is there if you’re writing a script. Command runs in container’s network namespace but with all other namespaces in host system (in particular in host filesystem and with permissions of root in host system)

Replacement variables that work in command-line:

  • @{container_ip} – replaced with IP address of a container being set up

Few examples:

  1. [/usr/bin/arping, -U, -c1, '@{container_ip}'] – default in v0.17.x. This notifies other peers that MAC address for this IP changed.
  2. [/usr/bin/arping, -c1, '10.0.0.1'] – other way to do that, that often does the same as in (1) a side-effect (where 10.0.0.1 is a default gateway)
  3. [/usr/bin/ping, -c1, '10.0.0.1'] – doing same as (2) but using ICMP instead of ARP directly

Most of the time containers should work with empty after-setup-command, but because container gets new MAC address each time it starts, there might be a small delay (~ 5 sec) after container’s start where packets going to that IP are lost (so it appears that host is unavailable).

secrets-private-key

(default is absent) Use the specified private key(s) to decode secrets in container’s secret-environ setting.

The key in this file is openssh-compatible ed25519 private key (RSA keys are not supported). File can contain multiple keys (concatenated), if secret matches any of them it will be decoded.

To create a key use normal ssh-keygen and leave the password empty (password-protected keys aren’t supported):

ssh-keygen -t ed25519 -t /etc/lithos/keys/secret.key

Note: the key must be owned by root with permissions of 0600 (default for ssh-keygen).

secrets-namespaces

(default is [“”]) allow only secrets with listed namespaces. Useful only if secrets-private-key is set.

For example:

secrets-namespaces:
- project1.web
- project1.celery

The idea is you might want to use single secret private key for a whole cluster. But diferent services having different “namespaces”. This means you can use single public key for encyption and specify different namespace for each service. With this setup user can’t just copy a key from one service to another if that another service isn’t authorized to read the namespace using secrets-namespaces.

To encrypt secret for a specific namespace use:

lithos_crypt encrypt -k key.pub -d "secret" -n "project1.web"

By default both lithos_crypt and secrets-namespaces specify empty string as a namespace. This is good enough if you don’t have multiple teams sharing the same cluster.

Currently namespaces are limited to a regexp ^[a-zA-Z0-9_.-]*$

See Encrypted Variables for more info.