Sandbox Config¶
This config resides in /etc/lithos/sandboxes/NAME.yaml
(by default).
Where NAME
is the name of a sandbox.
The configuration file contains security and resource limits for the container. Including:
- A directory where image resides
- Set of directories that are mounted inside the container (i.e. all writable
directories for the container, the
/tmp
…) - User and group limits
See overview for guidelines.
Reference¶
-
config-file
¶ The path for the processes config. In most cases should be left unset. Default is
null
which is results into/etc/lithos/processes/NAME.yaml
with all other settings being defaults.
-
image-dir
¶ Directory where application images are. Every subdir of the
image-dir
may be mounted as a root file system in the container. Required.
-
log-file
¶ The file name where to put supervisor log of the container. Default is
/var/log/lithos/SANDBOX_NAME.yaml
.
-
log-level
¶ (default
warn
). The logging level of the supervisor.
-
readonly-paths
¶ The mapping of
virtual_directory: host_system_directory
of folders which are visible for the container in read-only mode. (Note currently if you have submounts in the source directory, thay may be available as writeable). See Volumes for more details.
-
writable-paths
¶ The mapping of
virtual_directory: host_system_directory
of folders which are visible for the container in writable mode. See Volumes for more details.
-
allow-users
¶ List of ranges of user ids which can be used by container. For containers without user namespaces, it’s just a limit of the
user-id
setting.Example:
allow-users: [1, 99, 1000-2000]
For containers which have uid maps enabled in sandbox this is a list of users available after uid mapping applied. For example, the following maps uid 100000 as root in namespace (e.g. for file permissions), but doesn’t allow to start process as root (even if it’s 100000 ouside):
uid-map: [{outside: 100000, inside: 0, count: 65536}] allow-users: [1-65535]
For containers which do have uid maps enabled in container config, it limits all the user ids available to the namespace (i.e. for the outside setting of the uid map).
-
default-user
¶ (no default) A user id used in the container if no
user-id
is specified in container config. By defaultuser-id
is required.Note:
default-user
value must be contained in theallow-users
range
-
allow-groups
¶ List of ranges of group ids for the container. Works similarly to
allow-users
.
-
default-group
¶ (default
0
) A group id used in the container if nogroup-id
is specified in container config.Note:
default-group
value must be contained in theallow-users
range
-
allow-tcp-ports
¶ List of ranges of allowed TCP ports for container. This is currently not enforced in any way except:
- Ports < 1024 are restricted by OS for non-root (but may be allowed here)
- It restricts
bind-port
setting in container config
Note
if you have overlapping TCP port for different sandboxes, only single file descriptor will be used for each port. The config for opening port will be used arbitrary from single config amonst all users, which have obvious security implications.
Warning
tcp-ports
bind at port in host namespace, i.e. it effectively discardsbridged-network
for that port this is both the feature and might be a pitfall. So most of the time you should avoid non-emptyallow-tcp-ports
if using bridged-network.
-
additional-hosts
¶ Mapping of
hostname: ip
for names that will be added to/etc/hosts
file. This is occasinally used for cheap but static service discovery.
-
uid-map, gid-map
¶ The list of mapping for uids(gids) in the user namespace of the container. If they are not specified the user namespace is not used. This setting allows to run processes with
uid
zero without the risk of being theroot
on host system.Here is a example of maps:
uid-map: - {inside: 0, outside: 1000, count: 1} - {inside: 1, outside: 1, count: 1} gid-map: - {inside: 0, outside: 100, count: 1}
Note
Currently you may have uid-map either in a sandbox or in a container config, not both.
-
used-images-list
¶ (optional) A text file that is used by
lithos_clean
to keep images alive. It’s not used by any other means exceptlithos_clean
utility.Each line of the file should contain image name relative to the
image_dir
.It’s expected that the list is kept up by some orchestration system or by deployment scripts or by any other tool meaningful for ops team.
This setting is only useful if
auto-clean
istrue
(default)
-
auto-clean
¶ (default
true
) Clean images of this sandbox when runninglithos_clean
. This is a subject of the following caveats:- Lithos clean is not run by lithos automatically, you ought to run it using cron tab
- If same
image-dir
is used for multiple sandboxes it will be cleaned if at least one of them has non-falsyauto-clean
.
-
resolv-conf
¶ (default
/etc/resolv.conf
) default place to copyresolv.conf
from for containers.Note: Container itself can override it’s own resolv.conf file, but can’t read original
/etc/resolv.conf
if this setting is changed.
-
hosts-file
¶ (default
/etc/hosts
) default place to copyhosts
from for containers.Note: Container itself can override it’s own
hosts
file, but can’t read original/etc/hosts
if this setting is changed.
-
bridged-network
¶ (default is absent) a network bridge configuration for all the cotainers in the bridge
Example:
bridged-network: bridge: br0 network: 10.0.0.0/24 default_gateway: 10.0.0.1 after-setup-command: [/usr/bin/arping, -U, -c1, '@{container_ip}']
Note
when bridged network is active your Process Config should contain a list of ip addresses one for each container.
Note
this setting does not affect
tcp-ports
. So usually you should keepallow-tcp-ports
setting empty when using bridged network.Options:
-
after-setup-command
¶ Command to run after setting up container namespace but before running actual container. The example shown above sends unsolicited arp packet to notify router and other machines on the network that MAC address corresponding to container’s IP is changed.
Command must have absolute path, and has almost empty environment, so don’t assume
PATH
is there if you’re writing a script. Command runs in container’s network namespace but with all other namespaces in host system (in particular in host filesystem and with permissions of root in host system)Replacement variables that work in command-line:
@{container_ip}
– replaced with IP address of a container being set up
Few examples:
[/usr/bin/arping, -U, -c1, '@{container_ip}']
– default in v0.17.x. This notifies other peers that MAC address for this IP changed.[/usr/bin/arping, -c1, '10.0.0.1']
– other way to do that, that often does the same as in (1) a side-effect (where 10.0.0.1 is a default gateway)[/usr/bin/ping, -c1, '10.0.0.1']
– doing same as (2) but using ICMP instead of ARP directly
Most of the time containers should work with empty
after-setup-command
, but because container gets new MAC address each time it starts, there might be a small delay (~ 5 sec) after container’s start where packets going to that IP are lost (so it appears that host is unavailable).
-
-
secrets-private-key
¶ (default is absent) Use the specified private key(s) to decode secrets in container’s
secret-environ
setting.The key in this file is openssh-compatible ed25519 private key (RSA keys are not supported). File can contain multiple keys (concatenated), if secret matches any of them it will be decoded.
To create a key use normal
ssh-keygen
and leave the password empty (password-protected keys aren’t supported):ssh-keygen -t ed25519 -t /etc/lithos/keys/secret.key
Note: the key must be owned by root with permissions of 0600 (default for ssh-keygen).
-
secrets-namespaces
¶ (default is [“”]) allow only secrets with listed namespaces. Useful only if
secrets-private-key
is set.For example:
secrets-namespaces: - project1.web - project1.celery
The idea is you might want to use single secret private key for a whole cluster. But diferent services having different “namespaces”. This means you can use single public key for encyption and specify different namespace for each service. With this setup user can’t just copy a key from one service to another if that another service isn’t authorized to read the namespace using
secrets-namespaces
.To encrypt secret for a specific namespace use:
lithos_crypt encrypt -k key.pub -d "secret" -n "project1.web"
By default both
lithos_crypt
andsecrets-namespaces
specify empty string as a namespace. This is good enough if you don’t have multiple teams sharing the same cluster.Currently namespaces are limited to a regexp
^[a-zA-Z0-9_.-]*$
See Encrypted Variables for more info.