Tuesday, September 8, 2009

Steps for Migration to New VPS at Viviotech

I just completed the migration from my old VPS to a shiny new one at Viviotech and I figured I'd actually document things this time. I've done this a couple of times before with Viviotech, but since this is a bigger move it's a bit more manual this time around. Since I'm being moved from a FreeVPS server to a Xen server the VM image itself can't just be ported over, which means moving files, mail, configuration, databases, etc. all manually. Honestly this is probably a good thing since it gives me a chance to do some serious cleanup.

Another big change with this move is there's a new Kloxo (formerly Lxadmin) administration console. Previously they were using CP+ but I still did all my configuration by hand. Kloxo is a bit more draconian about having control over config files for Apache, but I dig into that later in this post. Kloxo is a much nicer admin console than CP+ (and since there hasn't been a release of CP+ since 2007 I think it's probably dead ...) but Kloxo does have some different concepts to be aware of.

So, here's the steps I took. Even if you're not going through this yourself hopefully they'll be some interesting info for future reference, or you'll gain a few Linux tips if nothing else. And thanks to Tim, Mark, Jordan, and everyone else at Viviotech for answering my questions as I went through this. I'm continually impressed by the level of service those guys provide. Best in the business bar none.

Step 1: Create User Accounts
I have a couple of freeloading friends using my VPS for some things (just kidding--they know I don't mind), so I set up user accounts for them on my server. This way they have their own directory, I can set quotas if I feel like it, and I don't have to give them root access unless I need my Linux sysadmin friend to fix something for me, which is a fair trade for giving him some space on my server.

In Kloxo they deal with users a bit differently than I was used to. Kloxo has the concept of "Clients," and clients can either be "Resellers" or "Customers." A customer account is the equivalent of a regular user account, so when you create a new customer they'll get a home directory, a web site, an FTP account, a mail account, and you can set features on the account like what they can and can't run on the web site, as well as put any quotas you need in place. Customers can also create additional domains if you allow them to.

Resellers are a bit different in that resellers can have customers under their account. Since I don't need reseller capabilities I used the customer option for creating all the accounts I needed.

When you SSH into the server you'll see the usual stuff, meaning a home directory for each user, and under the home directory they'll have a directory for each domain that is associated with their account. This is a really quick process and seems much less klunky and error-prone than with CP+, so overall I think Kloxo is definitely a step up.

Step 2: Move Files
This is probably obvious but since I'm moving about two metric tons of files from the old server to the new one, there's no way FTPing everything down to my local machine and FTPing back up was feasible. Since both servers are in the same data center, moving directly from one to the other is much, much faster.

SCP is a great tool for this job. You could also just FTP between the two servers but FTP is a lot less elegant for mass file moves like this.

So I sshed into server 1, navigated to the various directories, and scped all the files over:

scp -r -P port_num source_dir user@server2:target_dir

Again, since both servers are in the same data center this is amazingly fast. I gotta wonder about my one friend who has 9GB of stuff in his home directory, but I won't ask. ;-) Even that much data didn't take more than a few minutes to transfer.

Repeat scp as necessary to move things from the home directories on the old server to the home directories on the new server.

Step 3: Database Backups
I use MySQL exclusively on this box so this step was pretty easy. SSH into server 1, log into MySQL, and backup all the databases:

mysqldump -u username -p --all-databases > filename.sql

SCP that file over to the new server and restore it to MySQL on the new server:

mysql -u username -p < filename.sql

I still can't get over how much easier that is than dealing with SQL Server.

You'll also have to set the appropriate permissions on the databases once they're loaded up on the new server, so don't forget that step. (I'll assume if you use MySQL you know how to do that, or it's easy enough to Scroogle.)

One final thing I still haven't solved is that my storedprocs in MySQL throw a "Error occurred attempting to prepare the Call Procedure" error when they're run from OpenBD. The permissions are fine, and I can even run the storedprocs from the MySQL command line tool. At first I thought it was the MySQL driver, but I tried three different drivers and no luck. I'll follow up once I get that figured out, but if anyone has ideas I'm all ears.

Step 4: Tomcat and Open BlueDragon Installation
The new server already had MySQL and Apache on it so I didn't have to install those myself, but since I use Tomcat and Open BlueDragon I needed to get those set up.

Getting these running in and of themselves was as dead simple as usual: download Tomcat, Java, and OpenBD, and you're off to the races. Hit things at {server_name}:8080 to make sure Tomcat's working on its own. During this migration I was reminded again of how much I love working with OpenBD. Since I have each of my OpenBD sites completely self-contained, moving them to the new server meant copying files. That's it. No messy installations and configurations, I just drop files in Tomcat's webapps directory and I'm done. Complain about Java as a language all you want, but one thing Java gets absolutely right is web app deployment.

So all this went smoothly, but configuring Tomcat to work with Apache in the new Kloxo world was a bit different than doing it by hand.

Step 5: Apache Configuration
When you create web sites from Kloxo it likes to put the root directory for the site under a client's home directory. This is OK, particularly for plain HTML sites, but since I run most of my sites from Tomcat I personally prefer to have everything living under Tomcat's webapps directory.

You have a couple of options here. Depending on what you're wanting to accomplish, what will and won't work given your situation, and your preferred way of doing things, you could turn the home directory for the domains (e.g. /home/mwoodward/mattwoodward.com) into a symlink to the appropriate directory under webapps. Follow symlinks is enabled for virtual hosts when you set them up via Kloxo so this will work fine.

I use AJP proxying with Apache and Tomcat, however, so I left the home directory alone and added the proxying information to the virtual host entry. More on that below.

As far as general Apache configuration goes, Kloxo will overwrite the virtual hosts sections of config files, but the basic stuff like enabling/disable modules, etc. Kloxo leaves alone. So you can set up new web sites easily from the Kloxo interface, but still do higher-level server configuration in the config files themselves, which is a nice compromise.

For the AJP proxying, mod_proxy_ajp was enabled by default so I was in good shape there. One thing I do notice is that compared to Ubuntu anyway, the Apache setup on CentOS is a bit odd. I have Apache 2.2.3 installed on CentOS, but this install seems to follow the "older" way of doing configuration. By that I mean there is still a monolithic httpd.conf, and it doesn't have the mods-available, mods-enabled, sites-available, and sites-enabled directories that I'm used to with Apache 2.2+ on Ubuntu.

With Kloxo the configuration files are in a bit of a different place than you may be used to. Most of the files are in the typical /etc/httpd directory, but the virtual host config files are handled a bit differently. The httpd.conf still handles all the top-level server configuration, and it also includes /etc/httpd/conf/kloxo/kloxo.conf.

The kloxo.conf file in turn includes several config files under /etc/httpd/conf/kloxo, most of which you won't have to mess with. One of the files that is included is virtualhost.conf, and virtualhost.conf enables NameVirtualHost on ports 80 and 443, and then includes individual virtual host config files for each web site. The individual virtual host config files are stored under /home/httpd/{domain}/conf/kloxo.{domain}

These individual virtual host files are where you can get into trouble with Kloxo overwriting things. If you want to take full control over the config files yourself and not let Kloxo modify the files, Tim from Viviotech gave me this pointer--you can change the attributes of the file to be immutable:

chattr +i /path/to/config/file.conf

Then when you need to edit it, just change that back before editing:

chattr -i /path/to/config/file.conf

And of course change it back to immutable when you're done so Kloxo can't overwrite anything. As long as Kloxo only messes with virtual host config files this shouldn't be necessary--being able to click a button in the admin panel to get someone set up is definitely pretty nice.

Having individual virutal host config files seems messy at first, but from the standpoint of each site being discrete from all the others it makes sense, and certainly makes Kloxo's job of managing the files a whole lot easier.

So if you don't want to touch the virtual host config files but need to add something like AJP proxying info to a virtual host, you CAN edit the site's virtual host information from Kloxo. Just go to the screen for the domain in Kloxo, scroll to the bottom, and under "Extras" there's an "add extra tags" section. Note that anything you put in this box will not REPLACE what's already in the virtual hosts configuration but will be inserted INTO the virtual hosts block for the host. And if you have typos, etc., be aware that Apache may not restart.

So for adding AJP proxying to my sites that needed it, I just went to the "add extra tags" page in Kloxo and added this:

ProxyPass / ajp://host.in.tomcat:8009/
ProxyPassReverse / ajp://host.in.tomcat:8009/

And that worked fine. The resultant virtual hosts conf file for the site still contains the old DocumenRoot line but that doesn't impact the AJP proxying.

For simple stuff this should be fine; I'm still debating whether or not I like giving up all this control. Maybe I just got too used to editing the files directly but adding the extra tags I need through Lxadmin is certainly working.

Also note that once you're proxying with AJP you may find that you need to tell Apache to restart a couple of times before it actually restarts. My assumption is that this is because AJP maintains open connections between Apache and Tomcat. I notice this same behavior on both my laptops as well so this was nothing new.

Step 6: Test Web Sites
Kloxo has a pretty slick way to test sites before you change DNS over. In the domain management you can click a "DNSless Preview" button that lets you see if the site is more or less working before you change DNS. Obviously you're going to see things like missing CSS, images, etc. depending on how the particular site is configured, but it's a nice way to get a basic idea if the files are in the right place and things are limping along.

If you get a white screen when trying the DNSless preview chances are you have a permissions problem, so make sure that the site's file directory is owned by username:apache, and if you have symlinks in the user's home directory, e.g. a public_html directory that points to the domain name's directory, chown that directory as well. And don't forget the -R flag. ;-)

Step 7: Email
Since the two VPSes involved with this migration have different mail servers installed (Postfix on the old server, Qmail on the new server), and I use IMAP, my mail couldn't be moved over directly.

Rather than trying to deal with mailbox file conversions I decided to create a new account in Thunderbird pointing to the new server and just drag my mail over. I'm not an email hoarder per se but I did have a lot of mail to move, so this was the simplest way to do this. If you have a massive amount of email accounts to migrate you'll probably want to look into other options. I Scroogled briefly and didn't find anything very promising but I'm sure it can be done.

Webmail is another big improvement with the move to my new VPS. On my old VPS, the CP+ webmail interface would corrupt my inbox. With Postfix that's kind of a big deal since Postfix stores all mail in a single file. I finally gave up on CP+ webmail after having to edit my mail file by hand to remove the corrupted bits. Qmail stores individual files for each message, which to me as an email server layman makes a lot more sense.

On the new VPS there are two choices for webmail access: Horde and RoundCube. Horde is actually a PHP application framework with the Horde Groupware Webmail being one of the flagship apps from the project. It's very, very nice from my use of it thus far (though truth be told I prefer Thunderbird to webmail).

I haven't had a chance to try RoundCube yet because for some reason it won't connect to my mail server. I had MySQL permission issues during the migration that caused some mail outages for me, and at this point RoundCube still isn't working. It looks nice from the screenshots on the web site though, and it's nice to have some choices. I'm just happy these won't corrupt my inbox every time I use them.

When it comes to sending mail from a CFML engine, the new VPS requires mail be sent by an authenticated user. So you do have to create an email account on each domain from which email will be sent, and then use that as the username and password when you send mail from CFMAIL. Bit of a hassle but I suppose this is better from the standpoint of keeping the spam relaying potential to a minimum.

Step 8: Update DNS
Last step's the easiest. When you're ready to flip things over to the new server, just put in a DNS change request on Viviotech's site, and you're done.

Step 9: Miscellany
I did run into a few file permission issues. In one case I was getting 403 errors on a particular domain and even though all the file permissions looked correct in a terminal, it wasn't working. I deleted the domain in Lxadmin and recreated it, and that solved the issue.

The sure-fire fix if you run into permission issues is to make sure all the files you want to serve via Apache are chowned to username:apache. Weird thing is I had some files being served fine without being in the apache group, but others didn't, so when in doubt chown to username:apache.

There is also apparently a bug in Kloxo that in some cases when it creates accounts, it will set the file permissions to 711 which means Apache can't read any files. So if the owner and group permissions are correct, set the permissions to 755 on the user directory itself and that should resolve it.

Also, don't forget to migrate other things from the old server to the new one like anything in your crontab, any custom scripts in /etc/init.d, or other assets that may not fall into anything discussed above.

Thanks Viviotech!
Just a couple more comments about Viviotech's support. I've used a lot of hosting companies over the years, and I still have not seen another hosting company even begin to approach Viviotech's level and responsiveness of support. Answers to questions are quick, personalized, and thorough, and other support requests like updating DNS happen extremely quickly.

They also don't hesitate to go out of their way to help with things that technically go beyond what they say they'll support on a VPS with root access. If you have a problem of any sort, they'll help you through it quickly and courteously, and the response is thorough so next time you run into the issue you'll be able to handle it yourself.

So with some hacking around and Viviotech jumping in to help, I'm up and running on the new VPS! Hope others found this helpful or at least mildly entertaining. ;-)

3 comments:

christianready said...

I'm trying to do a viviotech setup. So far, I cannot seem to run the mysql command to restore my database. I'm trying mysql -u username -p password mura < mura.sqlWhen I run this command, I only get a list of possible mysql arguments. Any idea what I may be doing wrong?

Matthew Woodward said...

Hey Christian--pretty sure you need to leave the actual password off, and it will prompt you for it. So it's just:mysql -u username -p mura < mura.sqlHit enter, it'll prompt for your password, and then it should work.Let me know if you need any more help!

christianready said...

Thanks Matt. I tried it that way as well but I get an 'access denied' error. Any ideas?