0

I want to take a holistic approach backing up multiple machines running multiple Docker containers. Some might run, for example, Postgres databases. I want to back up this system, without having to have specific backup commands for different types of volumes.

It is fine to have a custom external script that sends e.g. signals to containers or runs Docker commands, but I strongly want to avoid anything specific to a certain image or type of image. In the example of Postgres, the documentation suggests running postgres-specific commands to backup databases, which goes against the design goals for the backup solution I am trying to create.

It is OK if I have to impose restrictions on the Docker images, as long as it is reasonably easy to implement by starting from existing Docker images and extending.

Any thoughts on how to solve this?

I just want to stress that I am not looking for a solution for how to back up Postgres databases under Docker, there are already many answers explaining how to do so. I am specifically looking for a way to back up any volume, without having to know what it is or having to run specific commands for its data.

(I considered whether this question belonged on SO or Serverfault, but I believe this is a problem to be solved by developers, hence it belongs here. Happy to move it if consensus is otherwise)

EDIT: To clarify, I want do something similar to what is explained in this question How to deal with persistent storage (e.g. databases) in docker but using the approach in the accepted answer is not going to work with Postgres (and I am sure other database containers) according to documentation.

Krumelur
  • 27,311
  • 6
  • 71
  • 108

1 Answers1

2

I'm skeptical that there is a custom solution, holistic, multi machine, multi container, application/container agnostic approach. From my point of view there is a lot of orchestration activities necessary in the first place. And I wonder if you wouldn't use something like Kubernetes anyways that - supposedly - comes with its own backup solution.

For single machine, multi container setup I suggest to store your container's data, configuration, and eventual build scripts within one directory tree (e.g. /docker/) and use a standard file based backup program to backup the root directory.

  • Use docker-compose to managed your containers. This lets you store the configuration and even build options in a file(s). I have an individual compose file for each service, but a single one would also work.
  • Have a subdirectory for each service. Mount bind-mount directories aka volumes of the container there. If you need to adapt the build process more thoroughly you can easily store scripts, sources, Dockerfiles, etc. in there as well.
  • Since containers are supposed to be ephemeral, all persistent data should be in bind-mount and therefore in the main docker directory.
fzgregor
  • 1,671
  • 11
  • 20
  • Thank you! I agree with and are usually doing pretty much all of what you write. The one problem that I haven't solved is how to deal with containers whose state cannot be backed up by just copying the data volume (e.g. databases) – Krumelur Sep 19 '17 at 20:32