[ToDoList] Docker Compose

We learned in the previous section how Docker is a fantastic tool with which to build and deploy applications, however has some inconveniences with developing these apps such as having to rebuild the image every time there is a code change as well as any development database entries being wiped.

This section will offer some relief against these two issues by using Docker Compose to build our application and make live code changes, as well as using a separate container to host our development database in PostgreSQL - this will encompass three main tasks:

  • Back up and edit the Dockerfile we made in the previous section so our image can be built.
  • Create our docker-compose.yml in the app's root directory to create our web & postgresql database containers.
  • Back up and edit some our app's configuration (namely the database.yml & Gemfile) to use the new postgresql container.
After all this, we can set about using Docker Compose to continue developing our lovely web application!

  1. Preparing the Existing Docker Config
  2. Dockerphiles Don't Just Use Dockerfiles
  3. Configuring the Application Database
  4. Developing with Docker Compose

 

 

First of all, let's get the hallowed package from apt: 

$ sudo apt install docker-compose

Next we need to edit some of the configuration we set up in the previous section.
To avoid some inevitable confusion, let's first remove any existing images and containers we created in the section: 

$ docker ps -a                                                          ## list containers
$ docker rm to_do_list_web                                              ## plus any other containers in the list above
$ docker image ls                                                       ## list images
$ docker image rm to_do_list:latest to_do_list:0.0.1 to_do_list:0.0.2   ## plus any other to_do_list images
With our Docker bits cleaned up, we can start on our list!

First up is a quick edit of the Dockerfile we constructed in the previous section. Because we will be using a postgres database instead of the standard sqlite one, we need to remove the lines we had that built the database and ran the migrations - failing to do this would result in a failed build!
First we will back up our existing Dockerfile: 

$ cp Dockerfile Dockerfile.standard
and then remove the two RUN lines that do the database bits from the Dockerfile - when complete, it should look like: 
Dockerfile

FROM ruby:3.3
RUN apt-get update -qq \
    && apt-get install -y npm postgresql-client
RUN npm install --global yarn n
RUN n 16.4.0
ADD . /usr/src/to_do_list
WORKDIR /usr/src/to_do_list
RUN bundle install
EXPOSE 3000
CMD ["bundle", "exec", "rails", "s", "-b", "0.0.0.0"]
As a final point, we will back up this version of the Dockerfile too, so that we can easily switch between the standard and docker-compose versions when required: 
$ cp Dockerfile Dockerfile.compose
With our Dockerfile amended, let's move onto where the magic happens.

 

 

Unlike in the previous section, where we were triggering image and container builds with the corresponding configuration using Docker's command-line tools, docker-compose allows us to specify all of the configuration options in a separate config file: docker-compose.yml.

In this file, we will set up the two services we need in their own containers:

  • web - the application, which will be essentially the same as the container we've made previously.
  • db - the database, which will contain a persistent PostgreSQL database for us to use with the web container.
We will also set up a volume for the database container storage, to avoid any potential permissions errors with reading or writing to the database.
Each of our service definitions will contain the options to build and configure each container as we need them, using the following keys:
Key Description Used In
image / build Where the container receives its image:
  • The db container gets the postgres image straight from DockerHub - again we are being specific with the version number
    • .
  • The web container gets its image from a build of the app, where we specify the same location to build from as with the regular Docker command-line.
    • .
db / web respectively
volumes Where the local filesystem location will map to on the container, meaning any changes we make locally will be reflected on the container automatically! db, web
environment Any shell environment variables required are declared here, in this case database authentication and Rails environment. db, web
command The command to run when the container is brought online, essentially replacing the CMD value in the Dockerfile.
(Note that we have added an additional command to remove the stored server process ID before the rails server initialisation to get round a known issue). 		web
ports Where we were declaring port mappings using docker run before, instead we store them in config. web
depends_on This ensures that a container doesn't run without its dependencies first being online - in this case, we don't want web to run unless db is online. web

Using what we've learned above, we can construct our docker-compose.yml in our app's root directory: 

docker-compose.yml

version: "3.1"
services:
  db:
    image: postgres:16.2
    volumes:
      - postgres:/var/lib/postgresql/data
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: password
  web:
    build: .
    command: bash -c "rm -f /tmp/pids/server.pid && bundle exec rails s -b '0.0.0.0'"
    volumes:
      - .:/usr/src/to_do_list
    environment:
      RAILS_ENV: development
    ports:
      - "8008:3000"
    depends_on:
    - db
volumes:
  postgres:
One small thing we'll need to do off the back of this is to generate the local folder for the db volume we have configured above: 
$ mkdir tmp/db
$ chmod -R 777 tmp/db
With this complete, we can move onto the final step of our preparation.

 

 

The last thing we need to do before building our first image & containers with Compose is to configure the application's database connection to use the new database container.
Starting with the database configuration itself (in /config/database.yml), we will need to change any mention of sqlite to postgresql and point the app to our db container, provide some authentication and encoding options as well as amending the database names for each environment.

Let's first back up our existing database config: 

$ cp config/database.yml config/database.yml.standard
and make the required changes to the main file - /config/database.yml: 
database.yml

default: &default
  adapter: postgresql
  encoding: unicode
  host: db
  username: postgres
  password: password
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>

development:
  <<: *default
  database: to_do_list_development

test:
  <<: *default
  database: to_do_list_test

production:
  <<: *default
  database: to_do_list_production
  username: to_do_list
  password: <%= ENV['TO_DO_LIST_DATABASE_PASSWORD'] %>
For good measure, we'll also back up this version for ease of switching between our original and the new version: 
$ cp config/database.yml config/database.yml.compose

Finally, we will need to edit our Gemfile to remove the now unnecessary sqlite3 gem, and move the pg gem out of the production group and instead into the default list - first we'll back up our existing version Gemfile versions: 

$ cp Gemfile Gemfile.sqlite
$ cp Gemfile.lock Gemfile.locl.sqlite
and, for now, just sticking this at the bottom of the file will suffice here: 
gem 'pg'
before creating a backup of this version for quick switching: 
$ cp Gemfile Gemfile.pg
With this complete, we are ready to go!

 

 

With our setup complete, we are finally ready to go! Admittedly there was a bit of faff getting to this point, but we can now use a beautifully simple command to allow Docker Compose to do everything it needs to do including:

  • Pull the postgresql image if it isn't already available locally.
  • Build the web image from our app's root directory.
  • Generate the containers for web & db services with the required configuration for volumes, ports, etc.
  • Start all of the service containers.
To achieve all this, we need simply run: 
$ docker-compose up
This will display everything it does in the same vain as regular Docker, as we experienced in the last section, and will start each container when ready to go!
Checking our list of running containers in another terminal will confirm the magic we have just witnessed: 
$ docker ps
CONTAINER ID   IMAGE              COMMAND                  CREATED          STATUS          PORTS                                       NAMES
f3fd3c8a6d9a   to_do_list_web     "bash -c 'rm -f /tmp…"   10 seconds ago   Up 9 seconds    0.0.0.0:8008->3000/tcp, :::8008->3000/tcp   to_do_list_web_1
ab7c8dce1c06   postgres:16.2      "docker-entrypoint.s…"   2 hours ago      Up 10 seconds   5432/tcp                                    to_do_list_db_1

Clever as you are, I'm sure you're wondering how our database is going to work as we removed the create and migrate commands from the Dockerfile.
Because the db container needed to be built before we could run either, we have had to let this run as is. Now, however, with the containers built, we can use a docker-compose command to run this, specifying which service we need it to run on.

First we'll kill the running containers with a quick Ctrl + c, and then run the database commands as described: 

$ docker-compose run web rails db:create
$ docker-compose run web rails db:migrate
Both of these commands will generate temporary containers to run their functions and then gracefully die. We can now bring our containers back up: 
$ docker-compose up
and you'll notice that there is no building going on now - all we're doing is starting the containers! No need to re-create or migrate database stuff, it's all already there.
A quick form submission in your browser window will reveal full abilities to interact with our lovely new database container using our lovely new application container!

ToDoList database running in Docker

As far as general usage goes, Compose is even easier to use than vanilla Docker. As we've seen, a simple up function is all that is needed to start all of the app's service containers - stopping these is also laughably simple: 

$ docker-compose down
If you go ahead and make any code change on the app, you'll also notice this this is reflected immediately!!!

The one downside to Docker Compose in my experience is that, as we're not manually building new images every time we make a code change, we are not able to tag images in the same way as before.
However, because we are essentially working directly inside the container, the need for this is largely redundant and so we're only as worse off as when we were developing on local / virtual machines! If need be, we can just tag new images using the manual method we saw in the previous section.

The only time we should need to rebuild an image is either when you have a working version that you want to tag, or when a change is made to our app's dependencies (e.g. gems, yarn packages, etc).
In this case, all we need to do to trigger a rebuild of the image is simply to add a --build argument to the up command: 

$ docker-compose up --build
and, because it's obscenely clever, Docker only knows to rebuild the image you've made changes to, meaning all of your database entries are still intact!

 

 

This section, and its parent section, have only covered the basics of what is capable with Docker and docker-compose - there is an awful lot of incredibly clever stuff you can do with it (for example, creating entire new apps in Docker without having to install Ruby on your local machine - see here!)
For now though, we will leave it here and use our new skills to create the best app we can in a reliably re-buildable environment! Let's begin this by pushing our changes up to GitHub: 

$ git add -A
$ git commit -m "Docker-compose config for app"
$ git push origin dockerise
and this time, we can create and merge a Pull Request so this functionality is accessible from our main branch!

 
 

< [ToDoList] Dockerising the App | [ToDoList] ... >
Back

Comments

Post a Comment

Popular posts from this blog

[ToDoList] Basic Pages

Image
Well that was fun. However all we’ve really done is spooked someone called Bob, without actually creating anything remotely resembling a web application. Let’s rectify this and create some pages one would normally find on a website; a Home page and an About page. Home & Away About Helpful Links [Insert Beatles Reference]     Because we already learned a bit about MVC, we know that we will need a controller and at least one view to display these pages (we won’t need to worry about models until we’re handling real data). As before, we’ll need to add a route configuration so Rails knows where to go, too. Rails provides a fair bit of magic to streamline this process in the form of generators . However, because we’re learning, we’re going to do this manually for now and learn what this magic entails, so we can appreciate it more later. We already created a PagesController in the...
Read more

New Rails Apps with Docker Compose

Image
Following on from the Docker & docker-compose guides, it is possible to use docker-compose to build a Rails application that uses a PostgreSQL database without even installing Ruby on your local machine by instead using containers ! This becomes a very useful option to have if you work from many different workstations and don't want to ensure they are all equipped with the correct Ruby and Rails dependencies, or if you wish to use a version of Ruby / Rails not supported by your operating system (for example, Ubuntu 20.04 only allows provides up to Ruby 2.7 in its apt cache). In fact, Rails 7 added native support for Docker, such that it generates almost all of the default files we need for us! This comes with its problems and, I won't lie, updating this post to work with Rails 7 (it was previously written for Rails 6) has been a colossal pain and features a bit of hackery. However it is ultimately a good thing, and having ...
Read more

[ToDoList] Building the App

Image
The new Generator The Rails Development Server     So we have now learned some theory about Rails and set up our development environment with the required packages and version control prerequisites. I think that’s enough preamble, let’s get developing! The next few sections will involve creating a funky little web app that contains a To-Do list that we can add, remove and edit entries from. We saw a very small snippet of the extensive file structure that Rails uses in the first section. If you thought you’d be constructing the longest mkdir -p command in the observable universe to get all of this set up, you will be pleased to know (as I was) that our first utilisation of Rails magic will be to get Rails to generate all of this for us! Feel free to set up a projects directory on your dev machine, or you can just use your home dir if you’d prefer. Whatever you choose, change into this dir n...
Read more