Docker containers can take up quite a bit of disk space, especially if used in a development environment. We create containers for Tomcat and they are upwards of 600MB. If you are developing and pushing code multiple times a day space gets chewed up fast. Our dev servers are in the cloud, and by default they don’t have tons of disk space. So using Docker with big images meant lots of regular maintenance on disk space.
I alleviated the problem by moving Docker’s directory structure to an attached drive instead of on the operating system drive. We are running Docker on Red Hat Enterprise Linux 7. In AWS I provisioned a larger drive and attached it to my RHEL instance, linking it to the /data directory. I want my Docker files in /data/docker. My first inclination was to make a symbolic link and move Docker, but I have found reports of this going awry with Docker. So reading some StackOverflow posts got me to this approach using /etc/sysconfig. Here are the steps for RHEL (it may be different for other Linux distributions):
Create a file at /etc/sysconfig/docker. Add this line to the file: DOCKER_OPTS=”-g /data/docker”
add EnvironmentFile=-/etc/sysconfig/docker under [Service]
Add $DOCKER_OPTS to ExecStart=/usr/bin/docker daemon $DOCKER_OPTS -H fd://
Save the file
Reload the daemon configuration: sudo systemctl daemon-reload
Restart Docker: sudo systemctl restart docker
After you restart Docker, check the /data/docker directory, it should have lots of content now. Any images you pull and create will now be stored there.
If you installed Docker via yum, this configuration change is lost when you perform a yum update where Docker gets updated.
I recently needed to add a Windows Service to a web app solution to handle some long-processing operations asynchronously. The web app is being deployed to Elastic Beanstalk (EB). You can log into the EC2 instance that EB creates and install the service manually, but this won’t last. EB can delete your instance at any time and create another one, and redeploy the app from it’s own git repository. So in order for the service to persist, it also has to be installed during the deployment. Fortunately for us, Amazon already supports this and there is a introductory article explaining how to do it and some sparse documentation.
Basically, you are adding a new directory called .ebextensions to your .Net project. In that directory you place YAML files with a .config extension. The YAML files contain the commands you can execute, as described in the docs. You can have more than one file, they execute in alphabetical order. The series of commands you create allow you to install the Windows Service when deploying a web application to EB.
Unfortunately, the procedure in the article only works on the initial install. If you need to push incremental updates, the script just doesn’t work because you can’t copy files over the versions currently in use by the Windows Service. Stopping the service wasn’t working for me because the account performing the deployment didn’t seem to have enough permissions to stop the service. But it did have enough permissions to uninstall and install the service.
The solution ended up being similar to the article, but with some variation. The correct steps for me turned out to be:
Fetch the zipped files from S3, unzipping to a temp location
Create the destination directory, ignoring errors (it already exists for updates)
Uninstall the existing Windows Service, also ignoring errors (the service doesn’t exist on the first install)
Copy the unzipped files in the temp directory to the destination directory
Install the Windows Service
Start the Windows Service
This translates to the following YAML file:
command: mkdir c:\\service\\serviceName\\Release
command: C:\\Windows\\Microsoft.NET\\Framework\\v4.0.30319\\installutil NameOfService.exe /u > “Remove.txt”
command: copy c:\\service\\temp\\Release c:\\service\\vserviceName\\Release
command: C:\\Windows\\Microsoft.NET\\Framework\\v4.0.30319\\installutil NameOfService.exe > “Install.txt”
commands: d-install-service The service uninstall/install commands are piping out their results to text files, so that if there is an error during execution I can RDP to the server and take a look at the specific error message. Also of note for troubleshooting, there is a directory created by the EB install process at c:\cfn. It contains a log directory, and in there is cfn-init.txt. If your install process fails, the errors can be found there.
I’m seeing more and more business units buying servers and services in the cloud in order to circumvent their internal IT. Obviously there is some additional risk here, but I think the main message is that traditional IT is not keeping up with the pace of change that the business wants.
This is an expansion of my thoughts on part of this article.
This table is just a quick comparison for the types of services that the major cloud providers offer as of the posting date. This is by no means a comprehensive comparison, as I am sure there are other providers and types of services I’m not listing. And I’m not being feature or platform specific either. For instance I realize that Google’s App Engine and Windows Azure don’t allow for the same development platforms. But they do provide a similar service at a high level. Also I’m sure the features for each category vary by provider.
As a consultant, I am always trying to see where new tools may fit so that I can help my clients. So I really need to know where the cloud fits into application development recommendations. Really, it’s just another platform that expands the options for a client. This is my 10,000 foot view of criteria for when Azure applies to solution (given what I know today):
Lower cost of maintenance. You don’t own the hardware, you don’t fix it, update it, etc.
Pay as you go. A short term app (say for a marketing campaign) is very cheap to run.
The app is abstracted from the infrastructure. You don’t have access to it, so now you ignore it.
Scalability is easy. Apps that are seasonal or have know traffic spikes benefit from this greatly. Also apps that become more or less popular are easier to scale up or down in a very short period of time.
The base cost for Azure is about ~ $1000/yr just to have one active app instance ($0.12/hr). Bandwidth and disk space are extra charges on top of the base cost, but seem inexpensive to me.
Since bandwidth and storage are cheap in Azure, so if you have a high need for either Azure is a good idea.
Reliability. Azure has Service Level Agreements (three nines or better) with penalties for Microsoft if they fail.
There is alerting/API integration with SCOM, so if you want to monitor an Azure app with your in-house tools you can.
Geo-location. You can have multiple instances of your Azure app in geographically disparate locations. Of course there are issues with data sharing, authentication, etc. But it’s significantly easier with Azure than other options available today.
You can use Azure as a storage-only option and split your app between self hosting and Azure, or vice-versa.
Having an app in the cloud may not be compliant with some regulatory guidelines
Cloud applications are web apps only. Not all solutions use web apps.
Identity and authentication are a challenge. The app is no longer in your domain, so you will need to look at federation or other options.
Most out-of-the-box apps are not hosted on Azure. Think CRM, ERP, financial, etc.
Apps that need access to the infrastructure aren’t going to work or need to be re-designed for the new paradigm.
It’s a new platform, even if it seems just like ASP.NET. This means you design differently, you develop a little differently as well. So there is a learning curve.
You are still responsible for data backups, I’m still not clear on how this works for the non-SQL storage options.
Overall, I think this is a significant new option for all sizes of companies. I’m not sure many companies are ready to give up the security of having mission-critical apps in-house, but I think for smaller companies Azure is a huge step forward. If you are already paying for hosting an app of some kind, give Azure a look.