Skip to main content
Wazo, LLC Company Logo

Wazo, LLC Network Administrators Blog

Go Search
Home
  

Wazo, LLC Network Administrators Blog > Categories
Dell IT Assistant Unable to Update Your Linux Servers?
If like me you manage a lot of Dell servers, IT Assistant can be a godsend (if somewhat clunky). As I installed and started to play with the features, I noticed I was unable to deploy updates to my Linux machines. The tasks would immediately fail with the following error:
 
File deploy command failed to execute with error message: The server's host key is not cached in the registry. The system may not be the same.
 
As I investigated, I noticed that most of the OpenManage processes were running as SYSTEM (or the LOCAL SYSTEM account). Therefore, if I used plink to cache the Linux server SSH keys in the registry of the Administrator, the IT Assistant process can't find them. Ah ha!
 
Luckily, there are two methods to solve this. The first method I'll provide will allow you to cache the SSH keys in the registry of the LOCAL SYSTEM account, the second (and recommended) method will make the OpenManage IT Assistant processes run under the Administrator account.


 
Method 1: (Dangerous)
 
To cache the keys in the registry of the Local System account, you'll need to download and install the PSTools from Microsoft, located here. We're after psexec.exe.
 
After you download and extract the tools, open a cmd prompt, navigate to the pstools extraction folder and run the following command:
 
psexec.exe -s cmd.exe
 
To verify it worked, type whoami and you should see nt authority\system listed.
 
WARNING: You can do SERIOUS, IRREVERSIBLE damage to your system when logged in as this account. Do not delete or create any files.
 
Navigate to the folder where you installed the OpenManage IT Assistant - it's usually in C:\Program Files\Dell\SysMgt\ITAssistant\bin - and use plink.exe to cache each SSH host key in the registry.
 
Example:
 
C:\Program Files\Dell\SysMgt\ITAssistant\bin>plink.exe root@192.168.1.1
 
The server's host key is not cached in the registry. You have no guarantee that the server is the computer you think it is.
 
The server's rsa2 key fingerprint is: ssh-rsa 2048 ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff
 
If you trust this host, enter "y" to add the key to PuTTY's cache and carry on connecting. If you want to carry on connecting just once, without adding the key to the cache, enter "n". If you do not trust this host, press Return to abandon the connection.
 
Store key in cache? (y/n) y

Once you're done adding all your SSH host keys, type exit to leave the LOCAL SYSTEM user's shell.
 

 
Method 2: (Preferred)
 
Using this method, we will simply change the service account that one of the IT Assistant processes runs under. Once we force the service to run as the Administrator account, we can then cache the SSH host keys into the registry of the Administrator rather than the LOCAL SYSTEM account.
 
To do this, start by closing any browser sessions you have open to the IT Assistant interface. Then, open the Administrative Tools - Services MMC and find the DSM IT Assistant Connection Service and double-click on it.
 
Select the This Account radio button and type in Administrator - then verify the password twice. Click apply, ok, and then right-click on the service and restart it.
 
Now, simply follow the plink.exe example above to cache all your Linux server SSH host keys in to the registry and you're done.
 

 
That's it! Good luck and happy updating!
 
 
Dell DRAC5 and Firefox on Linux
I manage a lot of Dell servers and I absolutely love the Dell Remote Access cards, affectionately known as DRAC. The problem is, I have a lot of DRAC5 cards which use an XPI plugin for Firefox, but the plugin won't seem to install automatically on Linux.
 
So, here's how to manually install the plugin for remote video and remote disk mounting capabilities on Ubuntu 7.10. This is likely applicable to other Linux distributions as well.
 

 
  1. Log into your DRAC5 card as normal.
  2. Paste this link into the address bar, overwriting what is currently there: https://YOUR_DRAC_IP/plugins/ (be sure to include the trailing slash!)
  3. Download the .xpi files from both the vkvm and vm directories.
  4. Extract the .xpi files to temporary folders.
  5. Open a Terminal, and navigate to /usr/lib/firefox/plugins
  6. Copy the .so files from each extracted folder into this directory.
  7. Copy the "videoviewer" folder and all subfolders into the plugins directory. (cp -R)
  8. Navigate to the videoviewer folder and make both .sh scripts executable by everyone, along with the "videoviewer" file.
  9. Navigate to /usr/lib/firefox/components
  10. Copy the .xpt files from the extracted directories to this folder.

Now, navigate to your DRAC5 in Firefox and bask in your remote server management capabilities!

Linux and Exchange 2007
In my quest to migrate at least one of my machines to Linux full-time, there were a couple of "must-haves" that I simply, well, must have. The main one being a real email client for managing my work email.
 
The client I work for, Alpha Theory, uses an Exchange 2007 server, so my Linux Email Client options are limited at best. The Evolution-Exchange connector won't work with Exchange 2007, so I'm left with POP3 and IMAP. I decided to go the IMAP route so all my mail will remain on the server, still accessible via Outlook Web Access and my desktop machine's Outllok client.
 
Since I don't want to open IMAP to the Internet via the firewall, I'll connect to it via our corporate VPN connection.
 


First, make sure the IMAP service is running and configured to start Automatically. The second thing we need to do is enable Exchange 2007 IMAP plain-text authentication. To do this, open the Exchange Management Shell and type the following commands:
 
  1. Set-IMAPSettings -LoginType PlainTextLogin
  2. restart-service -service msExchangeIMAP4

Now, simply open Evolution (or your favorite Linux email client with IMAP support) and configure it as you normally would for any other IMAP server.

Keep in mind, you'll likely need to use the INTERNAL FQDN or IP address of your Exchange 2007 server, rather than the external FQDN or IP address if you are connection over a VPN. I used the private internal IP address of our Exchange 2007 server during the configuration of Evolution with no problem whatsosever.

Installing Linux over the Internet from Windows
In my post yesterday titled OpenSUSE 10.3 and Microsoft PPTP VPN I noted I installed OpenSUSE 10.3 on my desktop computer in a dual-boot configuration with Windows XP. What I didn't mention is that I tend to do these projects late at night (what geek doesn't?) and I unfortunately found out I was not only out of blank DVDs, but blank CDs as well. After trying unsuccessfully for several hours to make various install images work off my USB key, I decided to heck with it and I'd just configure a PXE server on my laptop and PXE boot my desktop with a Linux install image.
 
It was while searching for the OpenSUSE 10.3 PXE boot image that I cam across this gem, Unetbootin (You Net Boot Install?). You run this program from within Windows and it adds an entry to your Windows boot menu which allows you to perform an Internet installation of Ubuntu, OpenSUSE, Debian, Fedora, Mandriva, or Arch Linux.
 
Check it out, good stuff.
OpenSUSE 10.3 and Microsoft PPTP VPN
I recently decided it was time to assemble a desktop computer again as I am starting to reach the limitations of my Thinkpad T60's screen and horsepower and in the process, set up a nice dual-boot WinXP/Linux machine.
 
I definitely wanted a RAID-0 configuration for speed, however I cannot use Linux LVM to create the array because of the dual-boot with WinXP requirement, so I'm stuck using the onboard nVidia RAID (nvraid) implementation. This would severely limit the distributions I am able to use. Fedora Core 7 would install but then not boot, Ubuntu 7.10 wouldn't see the array at all unless I loaded dmraid but then gParted wiped the partition table destroying everything in the process, so I was left with OpenSUSE which I heard had great nvraid support ... and I wasn't disappointed. I chose OpenSUSE 10.3 and I was pleasantly reminded why I always end up back with Suse, everything I need to do seems to work pretty much out of the box.
 
The one issue I ran into was creating a Point-to-Point Tunneling Protocol (Wiki: PPtP) connection to our work Microsoft VPN server. I followed all the HOW-TO guides I could find for vpnc and pptpconfig but nothing seemed to work. I eventually heard about network-manager-pptp for Ubuntu and ran across this entry in the Novell Bugzilla database with a link to a compiled network-manager-pptp rpm for Suse 10.2. I'm running 10.3, but I figured 'Why not?' I can always uninstall the rpm if it doesn't work.
 
So, without further ado, here's how I pulled it all together:
 

  1. Download and install the network-manager-pptp rpm from this link.
  2. Install it as root using rpm -ivh NetworkManager-pptp-0.6.3.cvs20060819-16.1.i586.rpm
  3. Log out and back on to restart network-manager.
  4. Click on the network-manager icon, choose VPN Connections, Configure VPN, + Add
  5. On the "Choose which type ..." screen, hit the drop-down and choose PPTP Tunnel.
  6. Connection Tab:
    1. For Type, make sure to choose "Windows VPN (PPTP)"
    2. For Gateway, put your VPN Endpoint IP Address
  7. Authentication Tab:
    1. Check Refuse EAP
    2. Check Refuse CHAP
  8. Compression & Encryption Tab:
    1. Check Require MPPE encryption
    2. Uncheck Require 128 bit MPPE encryption
  9. Optional Routing Tab:
    1. I only want to send traffic for servers on the VPN across the VPN connection, so check Only use VPN connection for these addresses and input your subnet.
  10. That's it! Just click on the network-manager icon, choose VPN Connections and then the entry you just created.
Combining/Parsing JBoss Weblogs using Stone Steps Webalizer on Windows
Recently I was asked by a client who has a mixed Windows/Linux infrastructure if there were any solutions for providing basic web statistics. Naturally The Webalizer by Mr. Unix came to mind. It's free, simple and has low resource usage. I've recently been using an updated version of The Webalizer by Stone Steps called, appropriately enough, Stone Steps Webalizer - those crazy Canadians and their wild imaginations. Feel free to click the link to see a summary of what the wonderful folks at Stone Steps have added to enhance The Webalizer.
 
The issue that immediately came to mind, was how to output a logfile from JBoss in a format that The Webalizer could parse and how to combine the log files from each of our load balanced JBoss servers into one file that the Webalizer can process.
 
So, let's get JBoss to output a logfile in the Apache Combined, or rather, the Combined Log File format, because we want to know information like the browser in use (User Agent) and Referrer (where they came from).
 

 
JBoss makes this pretty simple, open the following file in your favorite text editor and add the line I've noted below.
 
 /server/default /deploy/jbossweb-tomcat55.sar/server.xml
  1. Find the org.apache.catalina.authenticator.SingleSignOn Valve section and add a new Valve immediately above it that contains the following information:
    1. <Valve className="org.apache.catalina.valves.AccessLogValve" directory="${jboss.server.home.dir}/log" prefix="access-server##." suffix=".log" pattern="combined" resolveHosts="false" />
  2. This will create a file in your JBoss home directory/log subfolder, called access-server##.log in the Apache Combinded Log File format. Make sure to replace the ## symbols with a unique identifier for each server.
  3. Perform a restart of your JBoss server to complete the process.
  4. Done!


Now, we've got a log file that Webalizer will parse, so how do we combine multiple log files into a single logfile for Webalizer to parse? On Windows, this is pretty simple. I'm sure anyone with basic shell scripting experience could whip something up in a few seconds to do the same thing on Linux as well.



The first step is to get the log files over to the Windows machine for processing. I do this with an hourly cronjob that simply copies the files to a Windows (CIFS) share I have mapped across our management subnet. This could also be accomplished by using FTP, NFS, or SCP. That is beyond the scope of this article, so I'll leave that portion up to you. For reference, this is how simple the cronjob script is that runs on each of my JBoss servers:

  1. #!/bin/sh
    cp /jboss-4.0.5.GA/server/default/log/access-server* /mnt/win/logs

Yep, two lines, very simple eh?


Ok, so now we've gotten the files over to our Windows machine and they are ready for parsing. I am also not going to detail the steps for configuring Webalizer, as you should have a functional copy before attempting these steps. (Stone Steps Webalizer FAQ)

I simply created a batch file that does a Windows copy, but instructs Windows to combine the files. We don't need to worry about sorting, because Webalizer will use the date/time stamp on each entry to calculate the stats correctly!

So, click on Start -> Run and type notepad combine-logfiles-for-webalizer and run.bat. The simply add the following entry updating for your environment as necessary:

    1. @echo off
      c:
      cd\webalizer\logs
      copy access-server01*.log /a +access-server02*.log /a web-access.log /a
      cd ..
      webalizer.exe webalizer.conf
      cd logs
      del /q web-access.log
      del /q access-server01*.log
      del /q access-server02*.log

Just make sure in your webalizer,conf file, you've told it to look for web-access.log, then run webalizer or webalizer -c webalizer.conf and go have a look at your beautiful new, combined stats!

After you verify everything is working correctly, create a Windows Scheduled Task to run combine-logfiles-for-webalizer and run.bat once an hour or so to keep your stats up to date. I personally have my JBoss cron job configure to run at 30 minutes past the hour and then I kick off Webalizer right on the hour to combine the log files and update the stats.


Note: If your stats output isn't displaying correctly, make sure a copy of the webalizer.css file lives in the output directory or webroot. In addition, if webalizer is running on Windows, copy lucon.ttf and tahomabd.ttf to the webalizer folder and add the following two lines to your Webalizer.conf file:

    1. GraphFontNormal     c:\webalizer\lucon.ttf
    2. GraphFontBold          c:\webalizer\tahomabd.ttf

 

Happy Stats Processing!


 

 

CentOS 5 and Windows 2003 R2 Active Directory Integration
Several weeks ago I was tasked with creating single sign-on capabilities for a web application infrastructure (http://www.alphatheory.com/) that consisted of several Windows and Linux (CentOS 5 64-bit) servers. The client wanted the ability to create a user account in a single location for an employee or contract programmer, utilize that username/password combination for SSH access to the Linux servers, Exchange server account authentication, Sharepoint Service 3.0 Portal sign-on as well as Terminal Services and the CVS repository authentication. The client had a Windows 2003 R2 Active Directory domain I created earlier for Exchange/Sharepoint, so instead of reinventing the wheel, I decided to utilize the new features of Windows 2003 R2; more specifically the new Identity Management for Unix component of the Active Directory Services. (Ref: MS Technet)
 
In researching this, I came across several excellent links that I will include at the bottom of this post. Please feel free to visit them for more detailed information.
 

 
Windows 2003 R2 Configuration:
 
The first thing necessary is to install the Identity Management for Unix services. To do this, simply go into the Control Panel -> Add/Remove Windows Components -> Active Directory Services and check the Identity Management for Unix box.
 
 
Identity Management for Unix Screenshot
 
After this is installed, you will have a new tab on all your user properties called Unix Attributes. This is where you choose the NIS (Ref: Wikipedia) domain, the UID, login shell, home directory and primary group.
 
Unix Attributes
 
Go ahead and open the properties for the domain Administrator account and configure his/her Unix Attributes. We'll use this account later on to join our server to the Active Directory domain and for verification we can communicate with the domain via LDAP. (Ref: Wikipedia)
 
At this point, you should also go ahead and create an Active Directory account for the Winbind service. This account doesn't need any special privlidges or Unix Attributes, a standard Active Directory user account will work. Make sure to specify the password never expires and the user cannot change it to avoid issues with the Domain Security policy. I suggest creating a STRONG password for this account. Here's a link that will create a random, 24 character password for you: PC Tools Password generator. Make a note of the username/password for this account, as we'll need it later.
 
That's all you need to configure on the Windows Active Directory controller. Yes, really. Let's move on to the CentOS 5 configuration.
 

 
CentOS 5 Configuration:
 
I am assuming you have CentOS 5 installed and are able to SSH into the server to perform remote management. This will obviously work from the local console as well.
 
First, verify you have the Samba packages installed by typing the following:
  1. yum list samba*

You're looking for something similar to the following output:

Yum Samba Output via Putty

If you do not see the following, you can install them by using the following command:

  1. yum install samba samba-common samba-client

Now, let's verify we have an NTP daemon installed as Kerberos is dependent on proper synchronization of time across all servers.

  1. yum install ntpd
  2. chkconfig ntpd on

Let's edit the ntpd.conf file to utilize the NTP pool like all good administrators should.

/etc/ntp.conf:

    1. Remove all lines that have server 0.redhat.pool.ntp.org, 1.redhat, 2.redhat, etc.
    2. Add a single line of server us.pool.ntp.org
      1. Of course, use the public NTP pool that is appropriate for your country/locale.

Save and Exit

Start the NTP Daemon by typing service ntpd start

Let's make sure our CentOS machine can find the Windows Active Directory (aka: AD) server via DNS.

/etc/resolv.conf:

  1. search your-ad-domain.local
    nameserver ip.address.of.ad.domain.dns.server

Save and Exit

Now, let's make sure if we have a DNS failure, we can still find our Active Directory domain controller via the hosts file. We'll add a line to the bottom of the file pointing to the Windows 2003 Active Directory domain controller.

/etc/hosts:

ip.address.of.ad.domain.controller    hostname-of-ad-server.your-ad-domain.local    hostname-of-ad-server

Save and Exit

Now, let's edit the Kerberos configuration file - we'll need to add/change a few lines in here for our Active Directory domain.

/etc/krb5.conf:

Note: Use the FQDN of your domain (example: mydomain.local) rather than the NetBIOS name unless otherwise noted.

[libdefaults]
 default_realm = YOUR-AD-DOMAIN-NAME.LOCAL
 dns_lookup_realm = true
 dns_lookup_kdc = true
 ticket_lifetime = 24h
 forwardable = yes

[domain_realm]
 .your-ad-domain = YOUR-AD-DOMAIN.LOCAL
 your-ad-domain = YOUR-AD-DOMAIN.LOCAL

Save and Exit

Do not edit anything else, leave everything at their default values or risk the wrath of the evil Linux gremlins when things go wrong later.

The next step is to edit the Lightweight Directory Access Protocol configuration file, ldap.conf.

/etc/ldap.conf:

Delete everything in this file and add the following:

host ip.address.of.ad.domain.dns.server
base dc=your-ad-domain,dc=local
uri
ldap://hostname-of-ad-server.your-ad-domain.local/
binddn your-winbind-ad-account@your-ad-domain.local
bindpw strong-winbind-account-password
scope sub
ssl no
nss_base_passwd dc=your-ad-domain,dc=local?sub
nss_base_shadow dc=your-ad-domain,dc=localsub
nss_base_group dc=your-ad-domain,dc=local?sub?&(objectCategory=group)(gidnumber=*)
nss_map_objectclass posixAccount user
nss_map_objectclass shadowAccount user
nss_map_objectclass posixGroup group
nss_map_attribute gecos cn
nss_map_attribute homeDirectory unixHomeDirectory
nss_map_attribute uniqueMember member

Save and Exit

Now, let's edit the nsswitch.conf file so our server knows where to look to authenticate accounts.

/etc/nsswitch.conf:

Add the following ines, or edit to match if they are present.

passwd:     files winbind ldap
shadow:     files winbind ldap
group:      files winbind ldap
protocols:  files winbind
rpc:        files winbind
netgroup:   files winbind
automount:  files winbind
hosts:      files dns

Save and Exit

Now, we need to edit the Samba file so it knows where to look for the proper information. Instead of editing the existing file, I decided to start from scratch since I am not worrying about home directories and such. I just want authentication here.

/etc/samba/smb.conf:

workgroup = AD-NETBIOS-NAME
security = ads
realm = YOUR-AD-DOMAIN.LOCAL
use kerberos keytab = true
password server = hostname-of-ad-server.your-ad-domain.local

Save and Exit

User access is controlled on each server via a file called system_operators, let's edit this file and allow the domain administrator access. Just add a single username per line for any user you'd like to allow access to the server.

/etc/security/system_operators:

root
administrator
Some_Other_User

Save and Exit

Now, let's get the required services started and attempt to communicate with the AD server.

  1. chkconfig smb on
  2. chkconfig winbind on
  3. service smb start
  4. service winbind start

Ok, we should have all the Linux configuration files ready to go, name resolution taken care of and our Windows server prepped. Let's test this thing!

Let's initialize the Kerberos system by typing the following:

  1. kinit administrator@YOUR_AD_DOMAIN.LOCAL
    1. Capitalization is important here. Use it!
    2. You should not receive any output. You should simply be returned to your shell prompt after typing the administrator's password.
  2. getent passwd administrator
    1. This command will test account enumeration against AD.
    2. Administrator:*:10001:20000:Administrator:/home/Administrator:/bin/bash
      1. Something similar to the above should be returned. Don't worry if it's slightly different.
  3. net ads join
    1. Ignore anything other than the following:
      1.  Joined YOUR-CENTOS-SERVER to realm 'YOUR-AD-DOMAIN.LOCAL'

You're done!


Reboot your server for good measure (although it shouldn't be necessary) and feel free to log in with any AD accounts you've specified in your system_operators file. A home directory should be automatically created for the users upon initial login.

For futher reading or more detailed information, check out these links: