DCOS-Training

Notes for training on DC OS

View on GitHub

00 A review of Day 1

Additionally from yesterday - use Vagrant 1.9.1 if you get errors on Vagrant 1.9.5

Today’s training is more hands on and may be a little confusing due to all the errors and issues that may come up. Please be patient, keep your game faces on and your google search windows open. It gets a lot easier by Day 3. :)

01 Hands On: DC/OS - Scaling up and scaling down

Let’s start with a simple cluster setup:

  1. Reference: Git, Vagrant, and VirtualBox

  2. vagrant-hostmanager plugin

vagrant plugin install vagrant-hostmanager
  1. Clone-Configure-Deploy
git clone https://github.com/dcos/dcos-vagrant
cd dcos-vagrant
copy VagrantConfig-1m-1a-1p.yaml VagrantConfig.yaml
vagrant up

*note that we are using the 1-1-1 configuration today...

  1. Access the GUI http://m1.dcos/

  2. Install the DC/OS CLI

ci/dcos-install-cli.sh

Scale up and down by adding and removing public or private agent nodes

DC/OS does not allow changing the number of master nodes after installation.

Adding more nodes to an existing cluster requires your VagrantConfig.yaml to have both new and old nodes configured.

Add an Agent Node

To add a node, your VagrantConfig.yaml must have more agents specified than you currently have deployed.

Adding a node will not immediately change scheduled services, but may allow pending tasks to be scheduled using the newly available resources.

# Example initial cluster deploy
cp VagrantConfig-1m-3a-1p.yaml VagrantConfig.yaml
vagrant up m1 a1 p1 boot
# Add a private agent node
vagrant up a2

Remove an Agent Node

Removing an agent node will cause all tasks running on that node to be rescheduled elsewhere, if resources allow.

# Example initial cluster deploy
cp VagrantConfig-1m-3a-1p.yaml VagrantConfig.yaml
vagrant up
# Remove a private agent node
vagrant destroy -f a3

If the total amount of RAM on your machine is less than the memory specified in the yaml files, edit the files to reduce the resource requirements.

02 Hands On: Docker

Please ensure that you have logins on:

Open the Docker Toolbox terminal (Docker Quickstart Terminal) or the Docker terminal for this part.

Find Docker Images

$ docker images
$ docker search ubuntu
$ docker search nginx

Run Docker Job

$ docker run --rm ubuntu /bin/echo 'Hello world'

Run Docker Container

The -i switch signifies ‘interactive’

$ docker run --rm -t -i ubuntu:14.04 /bin/bash
$ echo 'Hello world'
$ exit

Run Docker Service

$ docker run -d --name my-nginx nginx

# list running containers
$ docker ps

# get container IP
$ NGINX_IP=$(docker inspect --format '' my-nginx)

# visit container address
$ curl http://${NGINX_IP}

# stop and remove container
$ docker rm -f my-nginx

Introspect Docker Containers

$ docker run -d --name my-ubuntu ubuntu:14.04 tail -f /dev/null

# Shell into running container
$ docker exec -it my-ubuntu /bin/bash
$ echo 'Hello world'
$ exit

# get the STDOUT & STDERR from the container
$ docker logs my-ubuntu

# get details about the running container
$ docker inspect my-ubuntu

# kill and remove container
$ docker rm -f my-ubuntu

Build Docker Image

OK, make sure you are in Docker hub, your username there is important.

$ mkdir -p demo/nginx
$ cd demo/nginx

# create a web page
$ echo "I just built my first Docker image \o/" > index.html

# create Dockerfile
$ cat > Dockerfile << EOF
FROM nginx
RUN apt-get update && apt-get install -y git
ADD index.html /usr/share/nginx/html/index.html
EOF

# log in to docker hub (not required for build, just getting getting username)
$ docker login

# get username
$ DOCKER_USER=$(docker info | grep Username | cut -d' ' -f2 | tee /dev/stderr)

# build docker image
$ docker build -t ${DOCKER_USER}/nginx-hello-world .

Push & Pull Docker Image to DockerHub

# publish image
$ docker push ${DOCKER_USER}/nginx-hello-world

# download published image
$ docker pull ${DOCKER_USER}/nginx-hello-world

Push Specific Image Version

When no image tag version is specified, latest is used by default.

# tag latest image with a fixed version
$ docker tag ${DOCKER_USER}/nginx-hello-world:latest ${DOCKER_USER}/nginx-hello-world:1.0.0

# publish image
$ docker push ${DOCKER_USER}/nginx-hello-world:1.0.0

# download published image
$ docker pull ${DOCKER_USER}/nginx-hello-world:1.0.0

Visit your container page on DockerHub: https://hub.docker.com/u/${DOCKER_USER}/nginx-hello-world/

Clean Up

# force delete all containers
$ docker ps -q -a | xargs docker rm -f

# delete all unused images
$ docker images -f "dangling=true" -q | xargs docker rmi

Other, less destructive clean up options:

# delete all stopped containers
$ docker ps -a -q -f status=exited | xargs docker rm -v

Now let’s quick build an example app

Fork Repo

  1. Visit the GitHub repo in a browser: https://github.com/karlkfi/minitwit
  2. Select Fork to open the fork screen

  3. Select a user or organization to copy the repo into

    The user or organization you choose will be referred to as ${GITHUB_USER} later in this document.

Clone Repo

$ mkdir -p ~/workspace
$ cd ~/workspace
$ git clone https://github.com/${GITHUB_USER}/minitwit
$ cd minitwit

Build Docker Image

$ docker build -t ${DOCKER_USER}/minitwit .

Run on Docker

$ docker run -d --name minitwit ${DOCKER_USER}/minitwit

Discover Container IP

$ docker inspect --format '' minitwit

View Container Logs

# print logs (STDOUT & STDERR)
$ docker logs minitwit

# print and tail the logs
$ docker logs minitwit -f

Stop Container

# stop container (could be started again later)
$ docker stop minitwit

# remove container (delete record and logs)
$ docker rm minitwit

Run on Docker with MySQL

MiniTwit (optionally?) depends on MySQL for data persistence.

# create mysql environment file
$ cat > mysql.env << EOF
MYSQL_ROOT_PASSWORD=root
MYSQL_DATABASE=minitwit
MYSQL_USER=minitwit
MYSQL_PASSWORD=minitwit
EOF

# start mysql server
$ docker run -d --name=mysql --env-file=mysql.env mysql:5.7.15

# find mysql IP
$ MYSQL_IP=$(docker inspect --format '' mysql)

# create minitwit environment file
$ cat > minitwit.env << EOF
SPRING_DATASOURCE_URL=jdbc:mysql://${MYSQL_IP}:3306/minitwit?autoReconnect=true&useSSL=false
SPRING_DATASOURCE_USERNAME=minitwit
SPRING_DATASOURCE_PASSWORD=minitwit
SPRING_DATASOURCE_DRIVER-CLASS-NAME=com.mysql.cj.jdbc.Driver
SPRING_DATASOURCE_PLATFORM=mysql
EOF

# start minitwit server
$ docker run -d --name minitwit --env-file=minitwit.env karlkfi/minitwit

Cleanup

# stop and remove both containers
$ docker rm -f minitwit mysql

03 Hands On: Back to DC/OS - Exploring the UI

Log in

  1. Enter the IP of a DC/OS master node (or a master node load balancer) in a browser
  2. Select a supported OAuth authorizer (Google, Github, or Microsoft)
  3. Follow the authorizer specific instructions to log in

Create a service

Install MinitTwit as a DC/OS Service.

  1. Select Services in the left navigation panel to access the service list page
  2. Select Deploy Service to open the service creation screen
  3. On the General tab, enter a service ID unique to the cluster (e.g. minitwit)
  4. On the General tab, enter the amount of Memory to allocate to the service (e.g. 512)
  5. On the Container Settings tab, enter the name or url of a Container Image (e.g. mhausenblas/minitwit)
  6. On the Network tab, under Network Type, select Bridge to enable mapping container ports to host ports
  7. On the Network tab, under Service Endpoints, enter the Container Port used by the service container (e.g. 80)
  8. On the Optional tab, under Accepted Resource Roles, enter slave_public to constrain deployment to public nodes
  9. In JSON Mode, under container.docker.portMappings[0], add "hostPort": 80, to specify which host port to use
  10. In JSON Mode, add "requirePorts": true, so that the service will only be deployed to nodes that have the specified host port available.
  11. Select Deploy to deploy the service

Locate service endpoint

  1. Select ‘Services` in the left navigation panel to access the service list page
  2. Select the name of the deployed service (e.g. minitwit) to access the service detail page
  3. Select the Task ID of the task with status Running to access the task detail page
  4. Select the first link in the Endpoints list to access the service itself

Destroy service

  1. Select Services in the left navigation panel to access the service list page
  2. Hover over the name of the deployed service (e.g. minitwit) to show the service actions button
  3. Select the service actions button to show a dropdown of service actions
  4. Select Destroy to destroy the service and its tasks

04 Hands On: DC/OS CLI

We’ll play a little more with the Minitwit service and see how it can be enabled with the command line.

Install

The DC/OS UI provides a link to install the CLI

Use the Linux instructions if you’re using a Linux VM for your workspace.

Please Take Note: The Windows install instructions may or may not work with the ‘bash shell’. It might be easier to install the DC/OS CLI in Powershell instead of using the GITBash shell.

Log in

$ dcos auth login

Generate an OAuth token in a browser and paste it into the CLI.

Create service

Install MinitTwit as a new Service.

# create service definition
$ cat > minitwit.json << EOF
{
  "id": "/minitwit",
  "instances": 1,
  "cpus": 1,
  "mem": 512,
  "container": {
    "docker": {
      "image": "mhausenblas/minitwit",
      "forcePullImage": false,
      "privileged": false,
      "portMappings": [
        {
          "hostPort": 80,
          "containerPort": 80,
          "protocol": "tcp"
        }
      ],
      "network": "BRIDGE"
    }
  },
  "acceptedResourceRoles": [
    "slave_public"
  ],
  "requirePorts": true
}
EOF

# create service
dcos marathon app add < minitwit.json

View services

$ dcos marathon app list

Locate service endpoint

Because this service is mapping to port 80 on the host, we can use the host’s IP.

$ dcos marathon app show minitwit | jq -r .tasks[0].host

Destroy service

# stop the service (could be started again later)
$ dcos marathon app stop minitwit

# remove the service (delete record and logs)
$ dcos marathon app remove minitwit

Whew! Lotsa stuff to mull over today!

This is where we close Day 2

Day 3 will focus on:

** I hope you enjoyed today’s session :)**