Docker Functions

docker_functions.bsh

Set of functions to make using dockers easier

VSI_DOCKER_SHORT_VOLUME_PATTERN

Regex pattern for matching docker volume short form

Note

Works on:: C:*, c:/* .* /* - ‘^(([a-zA-Z]:[/])?[^:]*)’
Break the volume string up into BASH_REMATCH: 1 - Host path 2 - Docker path 3 - flags

is_dir_and_not_exist

Arguments:: $1 - String to check
Output:: Returns 0 for is a directory, and 1 for not a directory (is an internal mount)

See if the string looks like a non-existing directory

Docker currently creates directories that do not exist when mounting. This has a few bad side effects, such as the directories end up being own by root. In order to work around this “bug”, this function identifies if docker will assume a string is a directory/file or an internal mount.

Example

is_dir_and_not_exist this_looks_like_an_internal_volume
is_dir_and_not_exist ./this_looks/like_a/directory
is_dir_and_not_exist /this_also_looks/like_a/directory

if is_dir_and_not_exist ${my_mount}; then
  mkdir -p ${my_mount}
fi

See also

docker_sanitize_volume

is_internal_docker_volume

Arguments:: $1 - String to check
Output:: Returns 0 if it is a volume and 1 if it is not (i.e., it is a directory or invalid)

Checks to see if the string is a valid internal docker volume (as opposed to a Linux- or Windows-style directory)

See also

is_dir_and_not_exist

is_readonly_docker_volume

Arguments:

$1 - docker volume_flags string to check

Return Value:

0 if it is readonly
1 it if is writeable (according to docker flags only)

Checks to see if a volume string contains the read only attribute. Can be used both for the docker volume syntax and the docker compose short volume syntax

Note

It is still possible the underlying directory is not writeable; this will not check that. Used mostly for docker volumes where this is not an issue.

See also

docker_parse_volume_string: Parses a docker volume string and generates volume_flags

is_readonly_docker_compose_long_volume

Arguments:

$1 - A DOCKER_VOLUME_FLAGS string to check (Not the whole array)

Return Value:

0 if it is readonly
1 it if is writeable (according to docker flags only)

Checks to see if a volume string contains the read only attribute. Can be used both for the docker compose long volume syntax

Note

It is still possible the underlying directory is not writeable, this will not check that. Used mostly for docker volumes, where this is not an issue.

See also

docker_compose_volumes: Parses docker compose volumes and generates DOCKER_VOLUME_FLAGS

vsi::docker::is_docker_compose_volume_readonly

Arguments:

$1 - A VSI_COMPOSE_VOLUME_FLAGS string to check (Not the whole array)

Return Value:

0 if it is readonly
1 it if is writeable (according to docker flags only)

Checks to see if a volume flags string contains the read only attribute.

Note

It is still possible the underlying directory is not writeable, this will not check that. Used mostly for docker volumes, where this is not an issue.

See also

vsi::docker::get_compose_volumes: Parses docker compose volumes and generates VSI_COMPOSE_VOLUME_FLAGS

container_get_label

Arguments:

$1 - Container name
$2 - Label name

Output:

stdout - Prints out the label value (and evaluates environment variables)
Return 1 if label is blank (possibly not set), 0 for label found

Get a label value from a docker container

Note

Requires perl be installed

Bugs

Cannot determine the difference between blank and unset labels. See container_has_label to handle that ambiguity.

container_has_label

Arguments:

$1 - Container name
$2 - Label name

Output:

Return 1 for label is unset, 0 for label is set

Check to see if a container contains a certain label

container_get_label cannot tell whether a container has a label or not when the label is blank. The go template filter does not differentiate. This will verify if a blank label is set or unset.

Note

Requires jq be installed

docker_premkdir

Arguments:: $1 - Host directory name or docker volume

Convenient wrapper to deal with the MINGW screw ups

Preempt one of docker’s annoying behaviors of creating directories that don’t exist with root:root ownership, and just make the directory yourself.

When the argument is a docker volume, nothing should happen. Docker auto creates those just fine.

Note

Always creates the directory with 777 permission. This is to maximize the chances that it works. If you don’t want this, the easiest solution is to create the directory yourself. This is more of a last resort. For this reason, when the directory is created, a message is printed.

When a directory doesn’t exist, it is assumed it is a directory, and not a file because this is the same behavior as docker.

docker_host_dir

Arguments:: $1 - Directory name
Output:: stdout - Converted directory name

Normalize POSIX path to native path

On Windows:
Converts /d/foo/bar to D:foobar using cygpath

On everything else in the world:
WYSIWYG

docker_parse_volume_string

Arguments:

$1 - The volume string (source:destination:flags)

Output:

volume_host - Host path part of the string
volume_docker - Docker path part of the string
volume_flags - Extra optional flags part of the string; may be empty
Returns 1 if regex is not matched, else return 0

Split up the docker volume string

Docker volume strings can contain optional flags and, depending on windows or POSIX operating systems, have different rules, etc… This function will universally split the string up.

Note

Based off of docker 17.12 syntax. If new flags are added, the regex must be updated.

Not actually a bug, but a string like “d:/test:ro” would be parsed as host path “D:test” and docker path “ro”, NOT internal volume “d”, docker path “/test”, and flag read-only. This is not a bug because you can’t have a single letter docker volume, so this shouldn’t work anyways.

docker_sanitize_volume

Arguments:

$1 - The host directory/volume name
[$2] - The docker directory (optional. If omitted, copies host directory)

Output:

stdout - The argument that should go with the -v to docker for mounting

Create a directory before docker gets a chance to

A number of things can go wrong with something as simple as mounting a directory. If the directory doesn’t exist, root will end up owning the directory (which is often not desired). Also, on MINGW Linux/Windows path expansion ends up being an error to docker on Windows. So this function will take care of both of these problems in an OS agnostic way.

Usage

Mainly for the docker and docker compose CLI -v flags

Bugs

The docker directory is not optional if the host mount is a volume

parse-docker

Arguments:

$1.. - Arguments to be sent to docker command

Output:

docker_args - Arguments to docker, before the docker subcommand (run, up, down, etc…)
docker_command - Docker command specified
docker_command_args - Arguments for the specified command

Parse docker’s arguments and split up information (See OUTPUT)

Usage

Typically, in the calling function, you will define all the output variables as local so that they are captured by the calling function only.

Original command can be thought of as:: docker “${docker_args[@]}” “${docker_command}” “${docker_command_args[@]}”

Docker

Arguments:

$1.. - Arguments to be sent to docker command

Parameters:

[DRYRUN] - Optional variable inserted at the beginning of all docker commands. Useful for setting to “echo” or similar for dryrun mode
DOCKER_EXTRA_ARGS - Array of extra arguments inserted after docker but before the docker subcommand
DOCKER_EXTRA_{subcommand}_ARGS - Extra arguments inserted right after the docker subcommand
[DOCKER_AUTOREMOVE] - Automatically add the –rm flag to docker run commands. Default: 1

Output:

Runs docker command

Helper function to execute the right docker command, or just dryrun

Instead of calling the docker command directly, this Docker function should be called instead (for all dryrun-able activities. Simple commands like inspect, ps, etc. don’t make as much sense to dryrun, as the reset of the script will never be executed, thus rendering dryrun useless.)

Bugs

Does not handle the -v or --help case where no command is supplied

parse_docker_compose_volumes

Arguments:

$1 - Name of service of interest
[$2] - Prefix to volume names

Output:

DOCKER_VOLUME_LINES - Array of the docker lines. Lines starting with an S represent the short format. Lines starting with an L represent the beginning of a long format entry. Subsequent lines starting with an l are the additional lines of the long format entry. Order matters when in this format.

Get volume lines from docker compose config

Parses the output from docker compose config so that information about the available mounts are ready to be used by docker_compose_volumes

Example

docker-compose.yml
  version: "3.2"
  services:
    nb:
      image : some_image
      volumes:
      - /tmp:/mnt
      - type: bind
        source: /home/user/src
        target: /src
      - test:/opt
  volumes:
    test:

parse_docker_compose_volumes nb test_prefix_ < <(docker compose config)
# Or
# parse_docker_compose_volumes nb <<< "$(docker compose config)"

declare -p DOCKER_VOLUME_LINES
> declare -a DOCKER_VOLUME_LINES='([0]="S/tmp:/mnt" [1]="Ltype: bind"
                                   [2]="lsource: /home/user/src"
                                   [3]="ltarget: /src" [4]="Stest:/opt")'

Note

Useful for working around docker/compose#4728

Supports both short and long format

See also

docker_compose_volumes

docker_compose_list_internal_volumes

Input:: stdin - Docker compose yaml file
Output:: DOCKER_INTERNAL_VOLUMES - Array of the docker volumes

Parses the output from docker compose config to get the list of all internal volumes

Example

docker-compose.yml
  version: "3.2"
  services:
    nb:
      image : some_image
      volumes:
      - test:/opt
  volumes:
    test:
    stuff:

docker_compose_list_internal_volumes < <(docker compose config)
# Or
# docker_compose_list_internal_volumes <<< "$(docker compose config)"

declare -p DOCKER_INTERNAL_VOLUMES
> declare -a DOCKER_INTERNAL_VOLUMES='([0]="test" [1]="stuff")'

Note

Both test and stuff will be returned, even though stuff is not used.

docker_compose_volumes

Output:

DOCKER_VOLUME_SOURCES - List of sources
DOCKER_VOLUME_TARGETS - Corresponding list of targets
DOCKER_VOLUME_FLAGS - Corresponding list of flags. Can be a mix of short and long format flags. Needs more parsing before this mix is useful.
DOCKER_VOLUME_FORMATS - Corresponding list specifying either “long” or “short” as the format of each volume

Get volume lists from DOCKER_VOLUME_LINES

Usage

parse_docker_compose_volumes nb test_prefix_ < <(docker compose config)
docker_compose_volumes

See also

parse_docker_compose_volumes

vsi::docker::get_compose_volumes

Arguments:

$1 - Name of service of interest
[$2] - Prefix to volume names

Input:

stdin - yarp’d docker compose yaml file

Output:

VSI_COMPOSE_VOLUME_SOURCES - List of sources
VSI_COMPOSE_VOLUME_TARGETS - Corresponding list of targets
VSI_COMPOSE_VOLUME_FLAGS - Corresponding list of mount flags (newline separated partial dump from yarp)
VSI_COMPOSE_VOLUME_TYPES - Corresponding list of volume types (e.g. bind, volume, tmpfs, etc…)

Get volume lists from the yarp output of a docker-compose configuration file.

Usage

vsi::docker::get_compose_volumes nb < <(docker compose config | yarp)

See also

yarp

parse_docker_compose

Arguments:

$1.. - Arguments to be sent to docker compose command

Parameters:

[JUST_DOCKER_COMPOSE_DIR] - By default, the docker compose files will initially be searched for in the JUSTFILE directory, and then up the parent dirs until / is hit. However, if you wish to the disable this behavior and start searching for a docker compose file from the current directory where “just” is called, then set this var to an empty string. You can also set to a specific directory to start searching from within that directory.

Output:

docker_compose_args - Arguments to docker compose, before the docker compose command (run, up, down, etc…)
docker_compose_command - Docker compose command specified
docker_compose_command_args - Arguments for the specified command
docker_compose_files - Array of docker compose files that are used
docker_compose_project_name - Docker compose project name

Parse docker compose’s arguments and pull out a few important pieces of information (See OUTPUT)

Usage

Typically, in the calling function, you will define all the output variables as local, so that they are captured by the calling function only.

Original command can be thought of as:

docker compose "${docker_compose_args[@]}" "${docker_compose_command}" \
               "${docker_compose_command_args[@]}"

Example

To start searching for docker-compose.yml in the directory where “just” is called: (Justfile snippet)

target) # Test target
  JUST_DOCKER_COMPOSE_DIR="" Just-docker-compose run target
  ;;

To change the default search location for a project, in the project env file add

: ${JUST_DOCKER_COMPOSE_DIR="${PROJECT_CWD}/docker_dir"}

Note

Because the -f argument overrides all other forms of choosing the compose file, the docker_compose_args are updated to use this notation instead of other methods that may be used. This will not change the behavior of the docker compose command run, but will make adding additional compose files a lot easier.

compose_path_separator

Output:: stdout - IFS string

Reproduce docker compose logic for pathsep

docker compose uses some logic to determine the IFS separator for splitting up the COMPOSE_FILE environment variable. This should return the exact same result

Usage

local OLD_IFS="${IFS}"
local IFS="$(compose_path_separator)"
...
IFS="${OLD_IFS}"

Docker_compose

Arguments:

$1.. - Arguments to be sent to docker compose command

Parameters:

DOCKER_COMPOSE_EXTRA_ARGS - Array of extra arguments inserted after docker compose but before the docker subcommand
DOCKER_COMPOSE_EXTRA_{subcommand}_ARGS - Array of extra arguments inserted right after the docker subcommand
[DOCKER_COMPOSE_AUTOREMOVE] - Automatically add the –rm flag to docker compose run commands when using the Docker_compose helper function. Default: 1
DRYRUN] - Optional variable inserted at the beginning of all docker commands. Useful for setting to “echo” or similar for dryrun mode

Output:

Runs docker compose command

Helper function to execute (dryrun) the right command

Instead of calling the docker compose command directly, the Docker_compose command should be called instead (for all dryrun-able activities. Simple commands like ps, etc. don’t make as much sense to dryrun, as the reset of the script will never be executed, thus rendering dryrun useless.)

Bugs

Docker_compose should not be exec’d like Docker was. Since fifo buffers are created in common usage of Docker_compose, exec Docker_compose would result in fifo buffers being left behind in /tmp

get_docker_compose_version

Arguments:: $1… - docker compose yaml filenames
Output:: stdout - docker compose version. Nothing if none are found

Returns the first docker compose version found

get_docker_stage_names

Arguments:: $1 - Dockerfile filename
Output:: stdout - List of stages, newline separated

Get list of named stages in a Dockerfile

get_dockerfile_from_compose

Arguments:

$1 - docker compose file
$2 - Service name
[$3] - yarp of compose file

Output:

stdout - The Dockerfile. Not guaranteed to be a full path

Determine Dockerfile filename from the compose file

If you would like to set the “project directory” used by compose to be somewhere other then the directory in which the docker-compose.yml ($1) resides, you must provide the yarp of the compose file as $3 and set $1 to a “file” in the directory of the project dir. Note, however, that this file does not necessarily have to exist.

Just-docker-compose

Arguments:

$1… - Arguments to docker compose

Parameters:

JUST_PROJECT_PREFIX - Typically, the project name that is prepended to all the variables. EXAMPLE would cover EXAMPLE_VARIABLE
[COMPOSE_VERSION] - The docker compose yaml version number is auto determined by parsing the existing files, and is used in the auto generated files. In order to bypass this, you can specify the version number manually using this variable.

Docker_compose with advance just features

To provide a smoother just experience, advanced features are baked into Just-docker-compose to work with Just and just_entrypoint_functions to cover the last mile in getting docker started and setup with a predictable environment user permissions, mounts (and more…)

Creates a docker override yaml file and extends the current docker compose configuration.

See docker_compose_override generate_docker_compose_override for more information.

Determines which docker compose yaml files to use by mimicking the behavior of docker compose, which includes querying the COMPOSE_FILE environment variable or trying the default files.

Bugs

A temp file is left behind in dryrun mode so that dryrun mode will actually work. As long as the commands echo’d by dryrun are executed, the temp file will be cleaned up, as one of them is rm. So ultimately, this is by design.

Using this function with windows paths is not currently supported.

You can not use process substitution for the docker-compose.yml file because bash has to read it before docker compose reads it. So this will not work: Just-docker-compose -f <(echo “${compose_file}”) run test You have to use files.

docker_service_running

Arguments:

[$1…] - service names, if none are provided, all are used

Parameters:

COMPOSE_FILE - Colon delimited file listing docker compose files to use

Output:

stdout - State of service running, Up/Exited/Creating/etc…

Return Value:

0 - Service found
1 - Service not found

Checks to see if a service is running

Returns the state-status of any containers, using the docker compose service names.

docker_compose_service_names

Arguments:: $1 - docker compose yaml file
Output:: stdout - New line separated list of service names

Get service names from compose yaml file

docker_compose_sanitize_project_name

Arguments:

$1 - Directory name. Can also be any name really. But directory name is what docker compose uses to come up with the project name
[$2] - Optionally specify a prefix (like a username) to assist in making a (user) specific project name

Example

docker_compose_sanitize_project_name 'project/A@1.1_2'
  projectA112

docker_compose_sanitize_project_name 'project/A@1.1_2' 'a-user:7'
  auser7projectA112

docker_compose_sanitize_project_name '' 'a-user:7'
  auser7

Make a valid project name

Docker compose will auto generate a project name based off of the directory name. A docker compose project name can only have lowercase letters and numbers. This function will reproduce that same functionality so that you can have that value in bash, with some added benefits (prefix) to truly take control of your docker compose experience.

docker_cp_image

Arguments:

[$1]... - Same optional arguments as docker cp
${n-2} Image name
${n-1} SRC_PATH in image
${n} DEST_PATH on the host

A helper function to emulate docker cp from an image instead of from a container.

The SRC_PATH must be from an image, and the DEST_PATH must be on the host