The aim of this article is to introduce our new approach to simplify the on-premise installations with Docker and what it implies
Some companies are still not ready for SaaS yet… or might never be
Toucan Toco’s solutions are SaaS solutions. We run them, update them, back them up and monitor them with everything hosted on our side. However, with the growth of the business and the sensitive nature of some data, some clients asked us to host our solutions directly on their infrastructure to respect their data security policy.
This can range from “data are too sensitive to be hosted elsewhere” to “nothing gets out from their networks because it is the government”.
Two years ago we created our first on-premise package for the Toucan Toco backend stack to answer these needs.
We already talked about this “DevOps” journey on a previous blog post (A Devops Journey - From SAAS To On-Premise).
The result of this work was a bench of Ansible scripts, which allows you to auto-magically deploy a full Toucan Toco backend stack. All these scripts are now used by our team to deploy and update all our SaaS customers stacks on our infrastructure, with the latest weekly release.
However, even if our package works and our customers install it without any help from our support team, we knew we had room for improvement. E.g.: some mandatory requirements are not always simple to enable for our customers: like having a network access to some famous code repositories like Github or the official MongoDB place.
In these cases, we always find a workaround but it is not optimal:
- for our customers: they don’t want to change their policy for Toucan Toco and they are right!
- for us: it’s always time consuming to find custom solutions that we could not use elsewhere.
This is why we tried to figure out a way to install the Toucan Toco stack with less requirements.
What about creating a Toucan Toco Virtual Machine ?
Obviously it’s a solution, why did we not do it ? :)
VM is an “old” concept and a lot of our customers already have a VM management system.
So for each release, we would just need to create and send a VM with the whole Toucan Toco stack…
… and that’s not so easy for several reasons:
- We wouldn’t know which VM to support: VirtualBox, Xen, something else ? And we didn’t talk about the versions yet…
- It is a nightmare to easily allow our customers to configure it: like updating default password, connecting the app logs to their system, installing their own certificates…
We felt like it would be tricky to maintain and automate this.
But there may be another way…
What about a Toucan Toco Docker container?
We deliberately ignored Docker as an on-premise solution for a long time because it was not used by our customers (fortune 500 companies).
By pure logic we would rather focus on projects and features that could be used by all our customers and not only one or two.
However a change is gonna come (like Sam Cook told us). Docker is becoming a target for almost all our on-premise customers or prospects… Well done guys !
We had to be on this train and offer something cool with it.
The docker dilemma: several containers or one “master”?
One part of the Docker philosophy is to have one container for each service/process and then you orchestrate them with tools like Docker Compose or others.
But there are too many tools with different versions and potential breaking changes… What we want is to be agnostic about them. For example with Docker Compose you have 3 main versions, which require specific Docker versions (cf. the famous compatibility matrix). Moreover, “compose” files are not compatible between major versions.
Remember we try to reduce the requirements so we need to avoid to impose new technologies or versions. So the docker philosophy didn’t work with ours.
We wanted to support our application/stack and not the side technologies used by our customers. We had to focus on our product.
This is why we decided to create one container, which holds the full Toucan Toco backend stack as described as below:
Other software editors (like Discourse) chose exactly the same “one container approach” for the same reasons: simple, easy to launch and agnostic.
How to create the Toucan Toco Docker container?
When you start from scratch, creating the Docker image can take time and be very difficult.
But if you remember we already got our Ansible scripts, which allow us to deploy and configure automatically the full stack.
Because we use them daily for our SaaS infrastructure we were really confident about them.
So as a first try, the build of the Toucan Toco backend image could be a basic Ubuntu image where we would launch our shipping scripts with the right settings!
This would be a perfect quick win.
Dockerfile could be something like this:
FROM ubuntu:16.04 # Backend sources # ############### ADD ./ backend_source/ # Start to install the Backend stack # ################################## RUN cd backend_source/ && \ make fulldeploy stage=docker # launch our Ansible scripts # Expose only the HTTP port # ######################### EXPOSE 80 RUN chmod +x /data/toucan_scripts/entrypoint.sh ENTRYPOINT [ "bash", "-c", "/data/toucan_scripts/entrypoint.sh" ]
After some minimal changes and trials, it worked like a charm without rewriting everything.
All the “smartness” is in the Ansible scripts which are robust and easy to maintain.
We absolutely wanted to avoid to have some parts of the configuration in the
Dockerfile and other parts in the shipping. We try to keep things simple.
Moreover we are not tied up to Docker, we could have the same approach with VM or a future new technology as long as we are able to use our Ansible scripts. Of course we could reduce and optimize the size of the image but it’s only a first version and the beginning of a new “devops” journey ;).
Please note that we also created a “smart” entrypoint bash script, which lets the users configure the container as they want. Kind of like the security settings or the details about Redis, MongoDB and mail settings.
It’s actually the only part we will need to maintain in parallel of the Toucan Toco backend evolution.
Now you have the same configuration flexibility between the usual on-premise package and the backend Docker image.
Ok but what about your CICD with Docker?
Our CI/CD is totally agnostic, it only knows how to checkout a project and launch a command according to it.
Each time a new release is created, it triggers the generation of a package via a
make package command included on each managed repository. Then the result of the command is automatically uploaded to our download place. So to plug the creation and upload of the Docker image, we only needed to add a new target (
package-docker) and configured our CICD to launch it for each release.
That’s all, pretty simple isn’t it?
How to set up and run the Toucan Toco backend image?
Now you can run a full Toucan Toco backend with less than 5 commands:
# download the latest version of the Toucan Toco image $ curl -u "USER:PASSWORD" https://get-package.toucantoco.com/laputa/docker_backend-latest.tar -o /data/tmp/docker_backend-latest.tar # load the image in your Docker system $ docker load < /data/tmp/docker_backend-latest.tar # check the image is well imported $ docker image list REPOSITORY TAG IMAGE ID CREATED SIZE toucantoco/backend v43.4.4 02d4415908d8 2 days ago 3.2GB # launch a container with the Toucan Toco backend image $ docker run -d -p 80:80 toucantoco/backend:v43.4.4 # test your container with the usual monitoring curl $ curl -A "Plop 4rQD3KzCxWzTYaRyp0NSEfd6" http://127.0.0.1/status
As you can see, you only need Docker and bash commands to run your own Toucan Toco backend stack. There is nothing more to install and it’s absolutely straightforward.
Of course for a production use, we suggest you to check the available configurations, which are described in detail in our online documentation (to persist data, enable HTTPS…).
All you need is 5 minutes
Because we “capitalize” on our existing robust shipping scripts, we found a quick win to create a real Toucan Toco backend image without changing the tools or the way we work. We are now able to have two ways to install the Toucan Toco backend stack:
- first with the usual package but you need to follow the asked requirements
- second with the brand new dedicated Docker image but you need to have a Docker stack on your infrastructure
This new on-premise approach is also pretty interesting because you’re also able to test it without reserving hardwares or asking your whole IT team to open network flows. You can create your own PoC in only 5 minutes.
If you previously thought our installation was too complicated, let’s talk again about it, we believe we can challenge your view now.
Of course the journey is still long about Docker and the on-premise context, we already have plenty of ideas for the future about optimization, features and so on… But thanks to Docker, we hope to unlock a lot of new business opportunities where the requirements were to difficult to meet.
In the end we are happy that we managed to simplify the installation and update for all our customers.