Archive for the ‘Tools’ Category

Moving Docker to Another Drive on RHEL

Docker containers can take up quite a bit of disk space, especially if used in a development environment. We create containers for Tomcat and they are upwards of 600MB. If you are developing and pushing code multiple times a day space gets chewed up fast. Our dev servers are in the cloud, and by default they don’t have tons of disk space. So using Docker with big images meant lots of regular maintenance on disk space.

I alleviated the problem by moving Docker’s directory structure to an attached drive instead of on the operating system drive. We are running Docker on Red Hat Enterprise Linux 7. In AWS I provisioned a larger drive and attached it to my RHEL instance, linking it to the /data directory. I want my Docker files in /data/docker. My first inclination was to make a symbolic link and move Docker, but I have found reports of this going awry with Docker. So reading some StackOverflow posts got me to this approach using /etc/sysconfig. Here are the steps for RHEL (it may be different for other Linux distributions):

  1. Create a file at /etc/sysconfig/docker. Add this line to the file: DOCKER_OPTS=”-g /data/docker”
  2. Edit /lib/systemd/system/docker.service
    1. add EnvironmentFile=-/etc/sysconfig/docker under [Service]
    2. Add $DOCKER_OPTS to ExecStart=/usr/bin/docker daemon $DOCKER_OPTS -H fd://
    3. Save the file
  3. Reload the daemon configuration: sudo systemctl daemon-reload
  4. Restart Docker: sudo systemctl restart docker

After you restart Docker, check the /data/docker directory, it should have lots of content now. Any images you pull and create will now be stored there.

Update

If you installed Docker via yum, this configuration change is lost when you perform a yum update where Docker gets updated.

Posted on:
Posted in Cloud, Tools | Comments Off on Moving Docker to Another Drive on RHEL

Migrating from DasBlog to WordPress

Due to many circumstances, I have a need to upgrade my shared hosting from .Net 3.5 to .Net 4.0. Normally this is no big deal, but I was running my blog on DasBlog, which hasn’t been officially updated since 2009, and it won’t just recompile and run under .Net 4. While there are some branches of their code re-compiled for .Net 4.0, I decided that un-tested code was not for me. I then decided on WordPress for a platform, mostly to continue expanding my horizons outside of Microsoft technology, and also because my hosting provider supports it, and I have more MySQL instances available for database use than SQL Server. I found an excellent post on migrating from DasBlog to WordPress, and used it mostly for a template on what I did, but since I did do some things differently, I decided to document my process. So below are the steps and technologies I utilized to migrate my blog.

Create a Test Environment

For my purposes, the first thing I did was create a Windows 7 virtual machine to do the migration testing. My production site is using shared hosting, so I can’t run a .exe on the server to get the content, which one of the tools I wanted to use required. I installed IIS, PHP, MySQL, and WordPress. I also made a new instance of WordPress in a virtual directory that matched the name of the virtual directory on my production WordPress install.

Set Up DasBlog Locally

I downloaded both the app files and the content from my hosting provider, then placed them in a virtual directory under IIS in my test environment, inside a virtual directory named to match the directory from production. I changed the site name in the DasBlog site.config file to reference the new location so the app would work.

Extract the DasBlog Content

I used DasBlogML to extract the data from DasBlog on the local instance of DasBlog, which outputs a blogML XML file. Some articles imply you need to remove any non-breaking spaces (   ) from the resulting XML, but I didn’t and had no issues.

Import the Data

I added the BlogML Importer plug-in to both my local install of WordPress and my production instance. I used that plug-in from the test instance to import the data and examine the results. For me it worked without issues, bringing over all the posts, comments, and categories. I had about 160 posts to migrate.

Create A WordPress theme

I am still enamored with the current look and feel of my blog which I built for myself as a DasBlog theme. I decided to migrate that HTML layout into a new WordPress theme, instead of using one of the myriad of available themes. I started with the HTLM5 Reset WordPress theme as a base and added the HTML structure and CSS from my DasBlog theme. I ended up modifying quite a few files in the theme in order to get it to work, and getting accustomed to PHP along the way.

Migrate Data and new theme to Production

I zipped up the WordPress theme directory on my test machine, and uploaded it as a theme in my production WordPress instance. Then I migrated the data using the BlogML Importer plug-in. I also had to configure the widgets in the navigation area and manually migrate some data into them.

Redirection

This is where the real work for the migration begins. Since my blog has existed since 2003, I wanted to maintain my links in search engines as much as possible. I already lost this once when DotNetJunkies (where I began to blog originally) got closed down by Dr. Dobbs, and it killed my comment traffic going forward. The post by Andreas that I was following advocated writing your own Http Module in .Net to handle all the redirects. I initially thought that was the way to go, but I found out that my hosting provider has the URL Rewriting feature installed for IIS7, so I could use the config file route. I figured that would perform better than any code I would write and is likely less buggy, even though over time the code should be used less and less. It also let me handle some of the one-off pages I added on the site that looked like part of DasBlog but really were just static HTML files, as well as some other media content.

There are a number of type of pages that DasBlog uses that need redirect templates:

  • Permalink
  • Permalink by Guid
  • Categories
  • Dates
  • CommentView

DasBlogML also generated a CSV file matching up the DasBlog permalinks with the WordPress permalinks. I used that as a starting point, and also queried the blogML XML file for the guids and dates of each post, and then matched them up with the CSV file in Excel.  I then used simple formulas in Excel to generate the XML needed for a static re-write file. Because the number of rewrites is so numerous, I decided to store the mappings in an external file called “maps.config” and reference that from web.config. The basic structure (one example per type) of maps.config is:

  1. <rewriteMaps>
  2.   <rewriteMap name="StaticRedirect">
  3.     
  4.     <!– Permalink –>
  5.     <add key="/blog/2011/12/16/AndroidAVDProblem.aspx" value="/2011/12/16/android-avd-problem/"/>
  6.     <!– Category –>
  7.     <add key="/blog/CategoryView,category,Mobile.aspx" value="/category/mobile/"/>
  8.     <!– Permalink by guid –>
  9.     <add key="/blog/PermaLink,guid,d280262b-e81e-496b-abde-453b6381574f.aspx" value="/2011/12/16/android-avd-problem/"/>
  10.     <!– Comment View –>
  11.     <add key="/blog/CommentView,guid,d280262b-e81e-496b-abde-453b6381574f.aspx" value="/2011/12/16/android-avd-problem/"/>
  12.     <!– Track back –>
  13.     <add key="/blog/Trackback.aspx?guid=d280262b-e81e-496b-abde-453b6381574f" value="/2011/12/16/android-avd-problem/trackback/"/>
  14.     <!– Date –>
  15.     <add key= "/blog/default,data,2010-01-04.aspx " value= "/2011/12/"/>
  16.     
  17.   </rewriteMap>
  18. </rewriteMaps>

And in web.config of the DasBlog virtual directory I added the following section:

  1. <system.webServer>
  2.     <rewrite>
  3.       <rewriteMaps configSource="maps.config"/>
  4.            <rules>
  5.           <rule name="RedirectRule" stopProcessing="True">
  6.               <match url=".*" />
  7.               <conditions>
  8.                     <add input="{StaticRedirect:{URL}}" pattern="(.+)" />
  9.               </conditions>
  10.               <action type="Redirect" url="http://localhost/wp{C:1}" redirectType="Permanent" />
  11.           </rule>
  12.       </rules>
  13.     </rewrite>
  14. </system.webServer>

To get it to take effect I had to recycle the app pool of the DasBlog app. The important part of the rule is the action. By setting the redirectType to Permanent, it causes IIS to issue a 301 response for the redirect, which tells the caller this is a permanent change. So any search engine should re-write the link in their database and not start you all over again in search engine results. And any other links outside of your site will just redirect automatically.

At this point it seems the new blog is running fine and the redirects are all working. I left the DasBlog instance running, at least for a while. I’m keeping my eye on Google Analytics to see if any other hits show up on the old blog. I am not an expert as DasBlog, so if any other page types show up I’ll find out about it and fix it in the redirect file.

Posted on:
Posted in IIS, Strange Problems, Tools | Comments Off on Migrating from DasBlog to WordPress

Why Use var?

Ever since the var keyword was released in C#, I’ve struggled with using it. To me var felt like using variant or object in VB, even though logically I know it is not.

List<string> items = new List<string>();
vs.
var items = new List<string>();

Since I bought Resharper, it’s been nagging me to use var in my declarations, so I  have acquiesced. Today that paid off. I had to refactor something which caused a few type changes. It was so much easier to deal with because I used var instead of writing out the declaration the old way, there were just fewer changes. I’m now a complete convert to var, thanks Resharper.

Don’t Forget Your Place

It’s easy to get wrapped up in the technology you use, thinking everyone is using it because everyone you interact with is using it. But when you back up and look at the bigger picture (which you should do), you may not be where you think you are. Tiobe Software creates a monthly Programming Community Index rating the use of programming languages. Our beloved C# comes in a number 6, behind (Visual) Basic of all things! I’m not really surprised by the leading languages (Java, C, C++), and it shows that while Microsoft is strong in the marketplace, it’s development tools and technologies are not leading the pack. But when looking at the charts over time, two of those leading languages have been declining in use for many years. C and (Visual) Basic have been consistent for a long time, and C# has been growing. I think the programming landscape has been fragmenting, the dynamic languages like Ruby and Python are gaining more mainstream acceptance each year, and the popularity of new platforms like mobile phones is going to keep changing that landscape.

So does that mean my choice of Microsoft as a platform isn’t the best choice? Hard to say, but I believe it’s a solid, reliable choice. I certainly think that it’s getting harder and harder to be specialized in any one language. Employable developers are going to need many programming tools in their tool chest. The end result is that fewer developers will truly be an expert with their tools, but will be good enough to get some work done. I don’t believe this bodes well for software quality going forward. There is already too much to know and be good at, I certainly struggle with this problem myself. As a consultant, learning lots of different tools is a necessity, so I will continue to keep working towards expertise in the tools I know, and not letting my interests get too fragmented.

Posted on:
Posted in Programming, Reviews, Tools | Comments Off on Don’t Forget Your Place

Cloud Services for Developers

This table is just a quick comparison for the types of services that the major cloud providers offer as of the posting date. This is by no means a comprehensive comparison, as I am sure there are other providers and types of services I’m not listing. And I’m not being feature or platform specific either. For instance I realize that Google’s App Engine and Windows Azure don’t allow for the same development platforms. But they do provide a similar service at a high level. Also I’m sure the features for each category vary by provider.

  Microsoft Amazon Google Rackspace
App Platform Azure   App Engine Cloud Sites
Relational DB SQLAzure RDS   Cloud Sites
Blob Storage Azure S3 App Engine Cloud Files
Table Storage Azure SimpleDB App Engine  
Pure VM planned EC2   Cloud Servers
CDN Azure CDN Cloudfront   Cloud Files
Queue Azure SQS App Engine  
On-Premise planned   App Engine Open Stack
Posted on:
Posted in Azure, Cloud, Tools | Comments Off on Cloud Services for Developers

Staying with the Fire Hose

At the recent Ann Arbor Day of .Net, my colleague David Giard arranged an excellent panel discussion with Jim Holmes, Mike Eaton, Jay Harris, Patrick Steele, and Jason Follas, about keeping current with the stream of constantly evolving technology with which we work (aka the Fire Hose). I felt this was one of the best sessions of the day, the audience was very engaged with the panel. It definitely ended too soon. *Update* David Giard posted a video of the discussion on his Technology and Friends webcast.

One of the discussion points revolved around training. I think a couple points were pretty clear from the discussion:

  1. Companies don’t provide enough or the right training
  2. You own your career, it’s up to you to get the training you need.

One of the discussion points that I wanted to contribute, was that the existence of Day of .Net is partly driven as a reaction by the community to the two points above. We attend these conferences to learn (among other things) just enough about new technologies so we can effectively evaluate them and see if they fit into our day-to-day jobs or personal interests. Traditional training methods (books, classroom training, CTBs) seem to be falling further and further behind as the fire hose continues to flow faster and faster. We are at the conference to get some of the training we need, because it’s not available any other way.

Technical Books

For me, books have always been part of my learning, but more and more they are falling short. Technology is changing faster than publishers can respond. More often than not the book I need isn’t going to be published for a while. Even today (May 2010), look at the number of new developer technologies that are hot in my circles:

  • Silverlight 4
  • Windows Azure
  • Windows Phone 7
  • Android
  • SharePoint 2010
  • .Net 4.0

These are by no means niche technologies. You would be hard pressed to find many books published on them (maybe some around beta releases, those get stale fast and are often of dubious quality due to the rushed nature of the title). I completely understand the lag time, but I think the publishers need to step up and start paying authors as full-time employees so they can commit full time to a book and get it released sooner. Typically book authors are doing this on the side, I am sure that slows down time to market. There are some Microsoft technologies, like Commerce Server, that haven’t seen a new book since 2002. Arguably that’s a much more niche technology (and certainly not as popular as SharePoint for instance), but if a company as large as Microsoft is investing money to develop the product, they have users of the software so there must be some kind of market for those books.

I know lots of folks don’t work on the cutting edge (I spoke to a group of VB6 developers yesterday that didn’t even have Visual Studio installed), so this is less of a problem for them. As a consultant, this is a constant problem for me. Maybe I am am the minority and there isn’t as much market for these books so early? That’s hard for me to say. Either way, I know the publishers are losing my money, because I end up digging up blog posts and finding online articles to meet my need long before the book hits the shelves for me to purchase it.

Day of .Net Slides

I posted my slide deck from my talk (The Demise of Xcopy Deployment) at the terrific Ann Arbor Day of .Net 2010 on SlideShare. Thanks to everyone who attended both my talk and the event. A special thanks to the event organizers for doing a great job.

MSDeploy Resources

Blogs

The Microsoft Web Deployment Team Blog on IIS.net

Owais Shaikh’s Blog on IIS.net

Kristina Olson’s Blog

James Cook’s Blog

Kateryna Rohonyan’s Blog

Articles

How does Web Deployment with VS10 and MSDeploy Work?

Web Packaging: Creating web packages using MSBuild

Web Packaging: Installing Web Packages using Command Line

SQLCMD scripts in Web Deploy (MSDeploy) v1.1

ASP.NET Deployment Content Map

Visual Web Developer Team Blog

MS Deploy Basics

How To Use MSDeploy to Migrate Global Assembly Cache

How We Practice Continuous Integration And Deployment With MSDeploy

Getting MSDeploy working on our build/integration server

Using MSDeploy as a build task in TFS

Posted on:
Posted in IIS, MSDeploy, Tools | Comments Off on MSDeploy Resources

Setting up MSDeploy on a Development Server

It’s not enough to just install MSDeploy on a server to allow Visual Studio 2010 to connect to the server. If you don’t follow the steps below, you will probably get 403 errors when trying to connect.

  1. When adding the Web Server (IIS) role, be sure to install Management Service.
  2. In the Services snap-in, find the Web Management Service and set its startup to Automatic (it’s manual by default). Don’t start it yet.
  3. Open the Internet Information Services Manager, and select the server.
  4. Open the Management Service feature.
  5. Check Enable Remote Connections.
  6. Choose a port (or keep the default), IP, Certificate, choose any restrictions you might require.
  7. In the Actions part of the right pane, click Apply.
  8. Click the Start link in the right pane.

Now IIS is configured to accept incoming connections. Open a port on the firewall for the port (8172 by default) you selected in step 6 above.

Install MSDeploy. Make sure you start the Web Deployment Service after the install, and set the service startup to Automatic.

Now you can use the Publish feature of Visual Studio 2010 to push directly to your development server.

See also: http://msdn.microsoft.com/en-us/library/dd465337.aspx

Posted on:
Posted in IIS, MSDeploy, Tools, Visual Studio | Comments Off on Setting up MSDeploy on a Development Server

Creating VPC Networks using Windows 7 as Host

I just want to track these two great posts on how to set up a virtual private network for hosting one or more virtual machines where you may not have an actual network connection. This happens to me when I’m giving presentations (like my upcoming one at Day of .Net!) and the host and guest machines need to be able to talk to one another, even if the host machine is not on any network.

Windows Virtual PC: Network Between Host and VM Using Loopback Adapter

Windows 7 & Network Loopback Adapter Settings

Posted on:
Posted in Speaking, Strange Problems, Tools | Comments Off on Creating VPC Networks using Windows 7 as Host