udocker

Get started using Udocker

CS repository (udocker-copy)

בעיות נפוצות

ssh to a container

Udocker with gromacs/ambertools/vmd/open-babel/python

Installing R+RStudio on ubuntu container

 Using udocker to run containers

(Document compiled by aco@tauex.tau.ac.il)

The udocker platform lets anyone run containers from the Docker repository or from other repositories. udocker does not make use of Docker nor requires its installation

Containers are pulled to a local repository (directory hierarchy) which is owned by the user. Since containers may be too large to fit in your allowed storage quota you are advised to use a temporary scratch area.

Get started using Udocker

1) How to create a local repository in /vol/scratch

(Note: if you have a storage space of your own, you can substitute /vol/scratch with the path where your storage resides).

mkdir -p /vol/scratch/<your-username>/

udocker mkrepo /vol/scratch/<your-user-name>/<name of my repo>

Make sure to define an environment variable (add line in .bashrc or in .tcshrc):

To find out which shell you are using, run the command "echo $SHELL"

2) Setting UDOCKER_DIR environment variable

For bash users (run only once):

echo "export UDOCKER_DIR=/vol/scratch/<your-user-name>/<name of my repo>" >> ~/.bashrc && source ~/.bashrc

For tcsh users (run only once):

echo "setenv UDOCKER_DIR /vol/scratch/<your-user-name>/<name of my repo>" >> ~/.tcshrc && source ~/.tcshrc

If you don't create a repository and define the UDOCKER_DIR environment variable, udocker will create a repository under your home directory, in the directory ".udocker". This can be a problem if you do not have enough storage/quota. In most cases, it is recommended to use /vol/scratch.

Warning: Files and directories in /vol/scratch are automatically removed every week. This is not a problem as long as you don't save dynamic content (such as the code that you develop, output of your jobs, etc.) in /vol/scratch. You can always pull and create your containers again after they disappear. Make sure to use your home directory for dynamic content (see step 6 below related to --bindhome).

3) Pulling images and running containers

You can search container images in the Docker repository using

udocker search [-a] <STRING>                          #The -a option displays all matching entries without pausing.

Here we pull an image of the latest Ubuntu container from the Docker repository:

udocker pull ubuntu:latest

Best Practice use:

udocker pull ubuntu:focal

udocker create --name=<name of your container> ubuntu:focal

cat lsb-release

EOF

# or a one-liner (from host machine, not from inside container):

udocker run <name of your container> /bin/bash -c 'cat /etc/lsb-release'

This sequence allows the created container to be executed many times. Containers can also be pulled, created and executed in a single step. However, in this case a new container is created for every run invocation thus occupying more storage space.

Note: The command udocker is to be run only from host machine and not from inside container.

Which user am I inside container?

Within the container you are root, you can install packages and perform other tasks that only root can do. However, your root permissions are mapped to you real user and group IDs in the host machine.

4) Directory mapping

You can make you home directory appear inside the container by using the --bindhome flag:

udocker run --bindhome <name of your container>

You can map any host file or directory to appear inside the container by using the --volume flag:

udocker run --bindhome --volume=/usr/local:/mnt/local <name of your container>

This will map /usr/local from the host machine to /mnt/local within the container. You can issue multiple instances of --volume in your command if you wish.

5) Installing packages of software as needed

If you need additional packages to be used in the container, you can install them. For instance:

udocker run <name of your container>

(and then from within the container):

apt update

apt install nano

Warning:Common error בעיות נפוצות

Now you can run nano. You don't have to install it again in the future because the installation is persistent (as long as the container hasn't been deleted): udocker preserves all installation and modifications in its root file system. You can find it in $UDOCKER_DIR/containers/<name of your container>/ROOT

 A more detailed installations guide:

  1. From outside the container, change execmode:

    udocker setup --execmode=P2 <name of your container>

  2. Run the container:

    udocker run --containerauth --user=root <name of your container>

  3. Recommended - pre-installation:

    apt update; apt upgrdae -y; export DEBIAN_FRONTEND=noninteractive 

  4. Install:

    apt install python3 python3-dev python3-pip

    apt install nano

    apt install build-essential

  5. Exit the container:

    exit

  6. Return the execution mode:

    udocker setup --execmode=P1

  7. Run the container for use:

    udocker run --bindhome <name of your container>

  8. link:

https://docs.google.com/document/d/1MH_nzcGC0pHbCoyTUBOZsb38pILZ9CBfWitF...

Using X11 for graphical interface

You can run X11 applications and get the output on your host if you use X11 to login to the Linux host machine, or from Windows, if you use, for instance, MobaXterm.

 

First, find out the ip address of your host machine, then run following command inside the container:

For bash:

export DISPLAY=<host_ip>:0

For tcsh:

setenv DISPLAY <host_ip>:0

Then run whichever 11X application you'd like, xeyes, for example:

xeyes

If xeyes is not available within the Ubuntu container, you can install it:

apt update && apt install x11-apps

 

Run a container with a listening service:

In case your container is running a service that listens on a port, you can specify the port that it will listen to for remote connections. Run the container with the --publish flag:

udocker run --publish=<HOST_PORT:CONT_PORT> <container_name>

* HOST_PORT= The port that the host will listen on, you will use this port for remote connections

* CONT_PORT= The port of the container app, this port will be mapped to the HOST_PORT

To verify that the port has been published, run this command on the host:

netstat -plnt | grep <HOST_PORT>

(Go to top)

 

CS repository (udocker-copy)

udocker-copy is a utility which lets you copy containers from the CS udocker repository to your local udocker repository. In the CS repository, you'll find a variety of containers which built specifically for the CS school needs and can save you time building and customizing containers from scratch.

usage:

  • The utility should be used from a Linux CS school server's terminal.
  • If you wish to copy a container which has the same name as a container from your local repository, you'll be asked if you want to overwrite your container. It won't delete your container and create a new one. To avoid problems that may occur by overwriting, consider renaming your container first using the command: udocker rename <container-name> <new-container-name>

View the utility's help: udocker-copy

View a list of available containers in the CS repository: udocker-copy ps

Copy a container to your local repository: udocker-copy <container-name>

View a brief descriptions of all containers: udocker-copy desc

View a full description of a specific container: udocker-copy desc <container-name>

(Go to top)

בעיות נפוצות

1. Error when running various apt commands in Ubuntu:

Solution:

Run the command:

cp /etc/passwd . && sed -i '/_apt/ s/^/#/' passwd && mv passwd /etc/

(Go to top)

 

 

Additional information and documentation: