An ESXi host, a NAS and NFS

When you're an IT guy tinkering is part of your lifestyle.  There has to be at least a modicum of curiosity about how things work.   We drive department heads crazy because just keeping stuff working isn't good enough for us.  We want that extra Megabit of throughput or another free Gigabyte out of the SAN.

Sadly, most of us can't afford to set up an server farm in a spare bedroom just to satisfy our need to tinker; but with virtualization you can come very close.

That is of course the promise.  Being able to harness the same capabilities of a small enterprise with a lot less hardware is undeniably a good thing.  Letting us run wild in our own little enterprise is even better.

VirtualBox, VMWare Workstation and their kind is fine for taking a new OS out for a spin but they fall a bit short for giving you real world skills.

ESXi, however, is another story.  Maybe more than any other platform, it's probably the most useful and relevant virtualization lab platform you can experiment with.  Don't confuse this with your grandpa's ESXi, though. 

Starting with version 5, VMWare decided that ESXi is the one hypervisor to rule them all instead of just being ESX's little brother.  That means anything you do with ESXi translates to what you can do in the enterprise.

It's one thing to play with virtual servers in VMWare but it's quite another to play with the platform itself.   After all, the more you tinker with it the better it works right?  Well, at least till we blow something up...

So this time around I decided I wasn't satisfied just locking myself out of the VSphere Client because I forgot the password.  I wanted to get some external storage online but I didn't have a spare ISCSI array laying around.   So I decided to venture into the wonderful world of NFS.

In ESXi you've basically got 2 options for storage.

1. Local - meaning it's either physically attached to the host or on a dedicated backbone via ISCSI

2. NFS - Which is pretty much "other"

Local's easy, if your storage controller can see it so can VMWare.  ISCSi  adds a wrinkle but so long as your target's on the network it's not a big deal.

NFS, ah, that's a different story.  A lot of Sys Admins can go an entire career without having to deal with it.   Like SMB, NFS is designed to offer up access to files on a network.  Those shares are usually hosted on Unix servers but unlike SMB, NFS is designed to fool your local PC into thinking a network resource is local.  An impressive feat compared to the clunky "Map Network Drive" or CLI "Net Use" commands in Windows.

Ok, so we know what NFS is but why do we care about it for VMWare?  Simple, most NAS storage devices will have support for the protocol to allow UNIX clients to access their shares.  Set up NFS on your NAS and you've given ESXi another potential datastore to play with.

NFS has it's quirks but most NAS management interfaces make it a relatively painless process to set up.  Once that happens just be sure you have a solid network connection between your virtual host(s) and the NAS.

Follow along with the video as I set up an NFS share for ESXi 5.5, play with a VM that lives on it and even break it!

CLI-nging ON

Imagine if you had to do the following  just to start your car every morning...

1.    Pull the choke adjacent to the right fender while engaging the crank lever under the radiator at the front of the car, slowly turning it a quarter-turn clockwise to prime the carburetor with fuel.

2.     Get into the car. Insert the ignition key, turning the setting to either magneto or battery. Adjust the timing stalk upward to retard the timing, move the throttle stalk downward slightly for an idle setting, and pull back on the hand brake, which also places the car in neutral.

3.     Return to the front of the car. Use your left hand to crank the lever (if the engine backfires and the lever swings counterclockwise, the left arm is less likely to be broken). Give it a vigorous half-crank, and the engine should start.

Chances are, your great grandparents were very familiar with the procedure if they owned a Model T.  Mind you, all that bother just starts the car, you haven't even tried to drive anywhere yet.  A complicated procedure to be sure but it beat shoe leather, barely.

Times changed and eventually somebody figured out how to take all those manual processes and mechanically automate them.  Soon, starting your car was just a turn of a key and driving it was more about where you wanted to go than how to control a mass of cantankerous machinery.  It also made the act of driving more accessible to more people.

That's called progress....

So as I stare at my open PowerShell window with a line of gobbledygook that to my dismay is my only means of finding out how much space my email users are hogging I have to wonder...

[PS] C:\>Get-MailboxDatabase "Mailbox Database 1" | Get-MailboxStatistics | Sort totalitemsize -desc | Export-CSV C:\mailboxes.csv

What's up with this command line stuff? 

Go into just about any forum or social media discussion even remotely related to IT and the flame wars will start with the mere mention of the command line.

"Go play with Windows if you want to click buttons"

countered with...

"You know, nobody uses punch cards anymore"

On and on ad infinitum. 

I had just such an exchange the other day with an open source devotee.  He was one of those grizzled admin types who clings to the mantra that nothing real happens in IT without a CLI (Command Line Interface)

He went on and on about his numerous accomplishments with his mastery of cursor and alphabet.  He also bemoaned the lack of functionality in all those "buttons."

It brought me back to the story a friend told me of an old CPA who refused to use a computer with Windows on it because all those kinds of OS's were just "Game" machines.

Anyway, apparently support for his favorite open source R&D project was callously left out of the latest VMWARE Vsphere client.  There was no doubt that to him such an omission doomed VMWare's GUI.

I was quick to point out that the Vshpere client is the preferred method (or the Web client for 5.5) to manage VMWare.  Actually, it's one of the more complete GUI management interfaces available and there aren't a lot of "buttons" to it.

He didn't think so...

To my mind, if it isn't officially supported then you take your lumps and have no right to complain.  But it's pointless to argue with someone like that.  As far as I was concerned he could just go back to his cave...

So it should be no surprise that I hate the command line but only because it's frequently a requirement instead of an option and to me that's all kinds of wrong.  A throwback to carrying around a crank to start your car and the Charleston.

But I have to admit that my console loving friend does have a point.  Far too often Graphical User Interfaces (GUIs) let us down.  They only contain the bare minimum of functionality often forcing you to stare into the inky blackness of a console.

 After 30 years of interface development why is it that CLI's are still so prevalent even in operating systems called "Windows?"  Why, for instance, do I have to execute scripts to select multiples of anything instead of just selecting what I want as a group and clicking "OK"

What my troglodytic friend doesn't seem to understand with all his mastery of the blinking cursor is that he's being forced to do more work than he needs to.  System Administration is frequently bogged down by syntax.  I shouldn't need to be a closet code monkey just to efficiently manage an enterprise.

Microsoft, the entity that made the whole concept of a GUI interface acceptable to most of the corporate world seems to be at odds with its own history. (BTW, Yes I know MAC was first with GUI's)

With every new version of windows it seems simple functions are being relegated to scripts and command lines.  Only the most rudimentary controls are left behind.  Spend any time managing Windows Server 2008 or 2012 and it starts to feel like you've entered some twisted Twilight Zone version of Linux.

I'm not saying the command line should be abolished.  Just like you occasionally need to roll up your sleeves and get a little dirty to get your car started in the morning; there's times you need to drop to that nasty old CLI.

That's ok, I have a right to get a little dirty if I need to BUT it shouldn't be a requirement.  That's my problem with the CLI. 

There's an assumption among the CLI elitists that if you're not comfortable with the command line then you must not know what you're doing.

I can assure you, I know exactly what I'm doing.  I'd just rather do it than get bogged down in a layer of abstraction to get to it. 

Yes, I said it, the CLI is an abstraction almost to the point of a DIStraction.  Even more so than a GUI, the convoluted syntax and cryptic commands do more to separate you from the task at hand than any click ever could.

Is your DNS server more secure and better optimized than mine because you configured it with your arsenal of scripts?  Or is it really an indication of a lazy GUI development team that's given you inadequate tools.

Microsoft introduced the concept of a modular GUI interface called the MMC (Microsoft Management Console) back in the NT 4 days.  It was designed to be customizable and more flexible than the standalone applets that accompanied every service.  It was a step in the right direction but unfortunately began the trend of minimizing GUI functionality by neutering the available commands within the MMC applets.

I submit that both the GUI and CLI devotees are right.  I should never have to go to a command line or call up a script to perform management functions.  I should, however, have a robust CLI that doesn't require me to understand the intricacies of of the Microsoft Foundation Classes.  Interfaces should be complete or at least offer the option to be that way regardless of their presentation.. 

What's wrong about the CLI guys is all the arrogance.  So what if you can dash out 30 line scripts from memory or poke holes through VMWARE to support the latest R&D cloud project. 

Don't assume that because I choose not to waste my time on hand cranked cars to get where I'm going that I don't know how to get there. 

I just expect...no...I demand better and at this stage in the game it's just developer laziness that I don't have it.

VMWare 5.5, Vconverter and VSphere Client tips

Let's face it, when you think virtual servers, Microsoft isn't the first name to jump into your head.  VMWare shows up a whole lot more than Hyper-V and chances are if you're in a large enough organization it's the one you've got to manage.

The latest incarnation is version 5.5 which is primarily an answer to a laundry list of bugs that started with the 5.0 platform. 

For most people, the first time they get to play with virtualization is in a sandbox like VirtualBox or VMWare Workstation.  But what if you want to go beyond just taking Ubuntu out for a spin on your Windows desktop?

That's where ESX comes in and even in a large deployment you're going to find out that your most important tools are deceptively simple.  That being the VSphere Client and VConverter.

The whole concept of Virtualization revolves around the Host.  It's the container (hardware) that all your virtual machines live on.  While VMWare's VCenter suite has a number of tools and features to help you manage the virtual enterprise nothing really ever strays that far from those two tools.  Know them and you've got a handle on 80% of what VMWare is all about.

The VSphere client is your portal into managing a single VMWare host and while you can learn a lot by creating a Virtual Machine from scratch on it most people take a shortcut.  That shortcut is provided by VConverter which lets you take an image of your physical PC and migrate it to and ESX Host.  It's called Physical to Virtual Migration or P2V for short.

As VMWare has matured so have the tools.  VConverter now allows you to use backups from programs like Acronis True Image and Symantec to create a new virtual machine.  It also allows the import of Microsoft Virtual PC and HYPER-V images and a few other formats as well.  In fact, VConverter is so versatile that its VM's can often be used in other virtualization platforms like VirtualBox. 

That's the good but of course there's always a few bad apples in the bunch.  The most annoying of which is the fact that VConverter still needs Windows.   Speaking of windows, migrating a Windows installation often requires having full administrative privileges on the machine running VConverter as well as the disabling of UAC (User Account Control) on the Windows P2V target.  Those aren't exactly best security practices in my book.

That there isn't a native Linux version of the VConverter utility only exacerbates nagging problems with migrating Linux deployments to the VMWare platform.   

That doesn't mean you can't move Linux PC's to ESX or even Workstation.  The option is there, it just doesn't always work and requires a lot of configuration of the Linux host just to find out it doesn't.

Still, getting familiar with the tools is a worthwhile endeavor and can only add to your arsenal of IT skills. 

It's said a picture is worth a 1000 words so a video should be worth at least twice that, so I've provided one below...

Enjoy!

Virtualization on the cheap with VirtualBox

Recently, I've been working on virtualization projects and one of them just didn't rise to the level of a VMWare rollout.  Even ESXi was a bit much for the deployment so instead we went with virtualbox which was much cheaper (free) and easier to administer.

Follow along with the following videos as I show you how I "kick it old school" with DOS and Windows for Workgroups in the latest VirtualBox platform.  Why such an old OS? Hey, anyone can load up Ubuntu or XP but a 20 year old OS takes some real effort and forces you to get up to your elbows in the virtualization platform.

Links: 

VirtualBox
Windows 3.1 Driver Pack

Hyper-V on Windows Server 2012

This is the latest video series on Server 2012 and specifically Hyper-V.  If you've ever wondered about the Microsoft flavor of Virtualization I've gone ahead and took the plunge for you....

Enjoy this tutorial and overview.

Semi-Competent Tutorials for the mildly interested.... ;-)

Part 1...


Part 2

The Pitfalls of Virtualization - Part 1 A little history

At the start let me quiet your fears, I'm not going to bash virtualization or the cloud, they're great options.  I'd even go so far as to say they're fast becoming the de facto standard for dealing with your data.

First a little history because contrary to popular belief virtualization was not present at the Big Bang.

When virtualization was in its infancy the promise was great but the future not so certain.  There was more chance of your office PC running a Linux distro than I.T. trusting their infrastructure to a server that didn't have a physical off button.   Virtualization was immature and more often than not when something bad happened there was little chance of recovery.  Worse, compatibility problems with operating systems left many deployments relegated to a corner running a few instances of UBUNTU.  I.T. was still suffering from the implosion of the tech bubble and didn't need another reason to worry about their jobs. 

Sometime around 2007 virtualization became acceptable.  Corporate bean counters liked the idea of doing more with less.  Hardware and storage costs were falling and Virtualization gained credibility when large companies coming up on hardware refresh cycles decided to make the move. 

VMware, Microsoft and Citrix responded with their own offerings promising ease of administration, lower energy costs  and better use of hardware.  It wasn't uncommon, for example, for the average windows server to only be using 40% of its capacity at any time.  Virtualization promised to fix that.

How times have changed.  Now you're hard pressed to find server hardware dedicated to something other than virtualization in any organization of size.   Gone are the days of hovering around the lobby waiting for your RAID controller to show up so you can get your new Exchange server up and running.  Now, a few clicks, an ISO image and you can have a new server online in minutes. 

Unfortunately, virtualization is a victim of its own success.  The bean counters have become addicted to the whole concept of more with less resulting in more downward pressure on  I.T. budgets. 

As a result, more often than not when I run into an organization heavily dependent on virtualization the hardware is at least 5 years old and probably repurposed from something else.  I recently walked into a multimillion dollar company, for example,  that was relying on second hand hardware sourced from EBay to run their virtual server farm!

I'm all for recycling but no server is immortal regardless of the operating system it's running and after awhile hardware will start to fail.  With the rapid pace of Moore's law it's not uncommon to find parts availability for servers relegated to the secondary market within 2 years.   Server hardware still tends to be proprietary and unlike your home computer isn't available at Newegg.  If it's obsolete you roll the dice and hope that hot deal on EBay isn't for something worse than what you already have.

The promise of cheap or free frequently guts reason, however, forcing I.T. departments into less than best practices.

Continued in Part 2

The Pitfalls of Virtualization - Part 2 Virtual Realities!

So it seems that virtualization's benefits can be quickly negated by an overly zealous accounting department.

Still the benefits are considerable. 

The aforementioned leveraging of hardware resources, reduced power consumption and the ability to allocate resources on the fly are undeniable benefits.  We're not quite at plug and play, however, and virtualized environments introduce their own caveats. 

Take hardware compatibility for example.  It's actually more of an issue with virtual environments than physical.  Remember we're dealing with layers of abstraction between your operating system and the hardware. Since virtualization vendors know their product can end up on everything from a re-purposed desktop to server class hardware they know better than to t try to support every configuration.  That means you're likely to be on your own if your chosen platform isn't on their compatibility list.

If your chosen virtual platform doesn't know how to talk to your SAN adapter, for example,  you're at a standstill if it's not on the compatibility list.  Nothing like trolling forums for support while your Fortune 500 company waits.   The same can be said for physical servers but a virtual host usually serves more than one virtual machine which just added an unwanted exponent to your headache. 

It takes some time to figure out the nuances of managing a virtualized environment as well.  Keeping in mind that everything you're seeing is largely an artificial construct and not necessarily reality has found more than one administrator scratching his head.

Ignore that fact at your own peril as It's far too easy to over commit a virtual resource and suddenly find alarms because you've overtaxed your processor and evaporated your storage.   Oh yeah, and all those angry voicemails on your phone.

That brings up another annoyance, licensing.  

While VMWARE, for example, will allow you to have a fully functional virtual host ready to accept as many virtual machines as you can throw at it for free, scaling that up to enterprise level can be an exercise in futility. 

Just like Microsoft, figuring out what you need is never straightforward and usually involves engaging a consultant unless you like to pay for things you don't need.  I've yet to walk into a VMware shop that had the right licensing mostly because the IT director decided to just wing it.  Unfortunately that route usually means the loss of much of the functionality virtualization offers. 

Just for fun, I went online searching for licensing packs for VMware and found a dozen vendors selling 100 concurrent user licenses for $25000.  They all had the exact same description which told me nothing about the product aside from how much better my life would be should I make the purchase.  It makes me miss the days of shrink-wrapped software.  Back then, I didn't need a 5 figure consultant just to figure out how to spend my money!

It seems the more user friendly things get the more money I have to pay someone to explain it to me.

We wrap it up in Part 3

The Pitfalls of Virtualization - Part 3 The Cloud

So if you really don't want to deal with the pitfalls of your own virtual infrastructure you have the option to use someone else's.  

Yes, I'm talking about the cloud which promises unlimited potential so long as your internet connection is working.

Bean counters like the cloud too.  After all to them it's almost free.  No hardware costs, no support overhead and virtually no downtime just a monthly invoice.

That's the promise at least...

Far beyond simple cloud storage from services like Dropbox, software as a service and hosted services via the cloud offered cost savings over the traditional model of keeping it all onsite. 

The highest profile players in the space currently are Microsoft (of course) and Google.  Both are more than happy to rent you their infrastructure for a "nominal" fee. 

The early days of this kind of service tended to over promise and under deliver.  Outages, bankruptcy, vague Service Level Agreements (SLA's) and questionable security hindered adoption.  

Imagine a law firm storing its confidential client files with a cloud provider who suddenly goes out of business. 

A good System Admin knows better than to put all their eggs in one basket but the question of who owned the data in the cloud still remained.  Could they trust that the data would be returned or destroyed if the unlucky provider went under? 

Around the same time Software as a Service(SAS) vendors came along promising universal access to business applications via the cloud.  Data protection showed up via something called software escrow.  Software Escrow promised your data would be safe with a third party should something go wrong.  

Salesforce and Google docs were the first examples but because of the vagaries of their SLA's most businesses decided to stick with their local office suites from vendors like Microsoft. 

Speaking of Microsoft...

Seeing an opportunity to appease the bean counters in the face of resistance to  their ever increasing software licensing costs they came up with Office 365 and Windows Azure.  Moving responsibility for messaging and data to Microsoft's cloud not only reduced infrastructure costs but in some cases headcount.  Why have a legion of IT professionals when any problem could be solved with a phone call?

There's nothing wrong with the logic but sometimes the execution can leave something to be desired.  Salesforce seems to have an outage at least once a year  and Microsoft Office 365 users have found themselves relying on smartphone messaging when hosted exchange servers go MIA.  Lest we forget the troubles with Google's cloud services. 

The hidden costs of cloud services have to come into play at some point.  Nothing's free as many a surprised supervisor has found when faced with a bill for his users going over their mailbox limit.  The purported cost savings in the server room can quickly be offset by the subscription model employed by cloud providers.

Just because I.T. services have moved out of the office and into the datacenter doesn't mean you don't have to pay for them. 

Cloud providers usually have tiered SLA offerings which means how fast they deal with an issue is directly related to how much you're paying.  If it's a system wide issue the SLA goes out the window.

Of course nothing's perfect and highlighting the flaws is no condemnation.  For the most part cloud services have lived up to their claims.  Like anything else the wise IT Pro knows not to rely on anything  exclusively.   Google docs offline? Work with a local copy.   Hosted Exchange services down?  Chances are you have more than one email account available to you elsewhere.

At this point the bloom is off the rose.  Virtualization is approaching the ubiquitous and there's no turning back.  Chances are at least some of the applications you work with every day have at least some portion living in the cloud.

That's not a bad thing just know that it's not the only thing.  As the old saying goes don't put all your eggs in one basket.