Clean Disk Space

At Cleavr, we never want to hear our users say their deployment is failing. Making sure that they can deploy their web apps is our top-most priority. So much so, that we've enabled internal notifications for when deployments fail so that we can check right away to see if there is anything we can do to help our users and minimize their frustration.

Often times, it's due to a user error (trying to migrate without setting up a database for example) and sometimes it is a bug in Cleavr - oops, but we are humans after all :). But, sometimes it is neither and turns out to be the server’s own limitation - such as getting to a point where, during deployment, there is no space left to generate any new files. This disk space issue shows its head so frequently that the first thing we do during debugging is to verify that the server has enough space available and notify users if not.

One of the things we have done to help save disk storage is create a deployment hook to "Clean Node Artifacts" for each web app. This removes the large, but no longer needed, 'node_modules' folder for NodeJS apps. This hook is disabled by default in order to support rollbacks and to support PM2 restart. You can jump into the related discussion in Cleavr Forum.

We always knew that there was more we could do as there are more ways to save disk space; such as by clearing cache, deleting journal logs, deleting compressed log files, and so on. With the recently released Clean Disk Space feature, we are exactly doing that. Here are some of the behind-the-scenes actions about this new feature.

First, we clear the cache using apt-get clean. This operation clears the cache that has been created during the installation of packages. It removes the files that are no longer required but still reside in your system and takes up disk space. Since the files being removed with this are downloaded with each installation, there is no harm in performing this operation. You can run this time and again to free some disk space.

We clear journal logs that are older than 3 days. Journal is a component of systemd. Systemd is a Linux initialization system and service manager. Journal log is the centralized collection of logs from different components. These logs are often helpful to investigate what's going on in the system. Over time, these logs from different components of systemd takes a lot of space.

In Linux, log files from various services are always located in /var/log. These logs are generated by the system log daemon, syslogd, and certain applications. Cleavr provisions the server in a way that these logs are automatically rotated and after a certain time of rotation, the log files are compressed. With “Clean Disk Space” we clean the compressed log files from /var/log. We understand that you may not always want to delete these logs since you never know when we may need the past logs for debugging. So, you may disable cleaning the compressed logs if you wish to.

If you're using MySQL, you probably know about binary logs. The binary log is a set of log files that contain information about data modifications made to a MySQL server instance. Binary log files can be useful but most of the time they may not be useful if you're not doing replication. We purge the binary logs that are older than 3 days. You may disable this option as well if you wish to keep the binary logs.

You can pick and choose any of the above and run clean disk space action. If you are one of the “AUTOMATE EVERYTHING!” kind of people, you can also set a scheduled disk clean-up job to automatically run on a set period, right from within the Cleanup UI.

With the release of this feature, we expect we'll never have to look back to the failed deployments because of low disk space. And if you do, cleaning the junk is just a step away. Let us know what you think!

• Anish Ghimire