udev and cloning a linux vm: Network not working…

Have you ever stumbled upon a cloned Linux system, in my case CentOS 6.5, where eth0 does not exist and eth1 isn’t started automatically?

When VMware clones a VM it gives its network card a new MAC address, ensuring that you don’t end up with several VMs with the same MAC. If your distro uses udev and it discoveres the new NIC, it gives it a different UUID, thus creating eth1 in the process, since it can’t match the MAC addresses and UUIDs of the NICs. This might break all sorts of scripts or configs.

Here is how to fix it:

  • First we need to remove the discovered and assigned UUIDs from udev:

rm -f /etc/udev/rules.d/70-persistent-net.rules

  • Secondly we need to edit the networking script for eth0:

vi /etc/sysconfig/networking/devices/ifcfg-eth0

Here you should change the old MAC address to the new one the VM got after cloning.

  • Reboot.

Thats it. eth0 should work as it used to on the parent VM.

 

thanks to William: http://www.envision-systems.com.au/blog/2012/09/21/fix-eth0-network-interface-when-cloning-redhat-centos-or-scientific-virtual-machines-using-oracle-virtualbox-or-vmware/

Advertisements

HowTo: Use/Migrate an existing local OS X user profile for use with an ActiveDirectory User

So, we’ve all been there: A user is using his Mac with a local account. At some point IT needs to manage all Computers and Passwords, and thus this Mac together with it’s user needs to be ActiveDirectory managed. But of course: No setting, no file, nothing should change, because the user is king (and maybe the company’s boss that hates being upset, and even a changed background or shortcut-location upsets him….). Here’s how to do it:

  • Create a new local user with admin rights.
  • Logout of existing User and into the new admin user.
  • Delete the user you want to migrate. When the system asks, don’t delete or archive the user folder, just leave it where it is.
  • In a terminal issue the following command “sudo mv /Users/oldusername /Users/newusername” where newusername is the shortname of the AD User. This is critical!
  • If not already happened bind the Mac to the AD.
  • Use “chown” in terminal to change the owner of the users directory to the new domain user. Use the shortname, no need to write the FQDN of the AD.
  • Use “directory utility” to change the settings and check the box to create a “mobile account at login”, and check the second box, too.
  • Now logout, maybe reboot. (Sometimes it is needed, sometimes not, depending on how quickly the Mac gets the new AD binding.
  • Login using the new users shortname. It should ask for a mobile profile, create one!
  • You might need to update the keychain password.

Thats it: Enjoy your migrated user folder and settings. You shouldn’t notice any difference besides a new password 😉

One note: The new user is a standard user without administrative rights. If you need to give him/her or the Administrator-Group admin rights, you can to this in the “Directory Utility” as well. Single users won’t work, use groups like this: DOMAINNAME\groupname .

All the best.

vSphere/VMware: failed to connect virtual device ethernet0

failed to connect virtual device ethernet0

That message said hello for every single VM after there was a major breakdown in a data center. The breakdown was seen as a welcome opportunity to upgrade everything from 4.1 to 5.1. And since everything was broken anyway (although the VMs continued to run, yeah VMware ;-)) no one bothered going the proper path but just evacuated some ESXi-Hosts, re-installed them with 5.1, created a new vCenter and tried to import the VMs.

What was happening?

The GUI gave no hint as to what was wrong. But in the ESXi host logfiles something gave away what was going on: “vShield filters cannot be found for ethernet0”. Now, that is a clue, indeed!

The old 4.1 was running with everything filtered through vShield, whereas it was decided to not use vShield in this setup for 5.1 anymore. But in every single vmx-file for every VM there had to be removed the following two lines in order for everything to work as it should:

ethernet0.filter0.param1 = “uuid=5006f477-a2df-b018-b331-b2b61f1b95f9.000”
ethernet0.filter0.name = “vshield-dvfilter-module”

So, people, beware of vShield when moving VMs from one cluster to another.

All the best,

Thomas

vShield Manager 4.1 spikes in CPU usage, acting up…

Since a couple of weeks we let vSphere monitor our VMs in addition to the nagios monitoring we have had from the beginning.

The vShield Manager VM started to show odd behavior: CPU spikes to 100% every 10 to 30 minutes, then returning to normal. Operation worked as normal, nothing showed in the log files (although: you don’t really see the system log files in the interface).

Somewhere I read something about the mySQL database having problems, and a reboot would fix that. So, reboot… After all, what could go wrong, right? The single firewall VMs are working independent from the Manager, so nobody should even notice… so far so good.

vShield Manager VM did not boot up properly again. You could not login via the web interface. On the console you’d see only the message “System startup is not complete. Please logout and log back in after a few minutes.”

After some hours still the same. On the console (you don’t get a full bash, just a stripped down management shell that does not much) not much was showing. On thing made us curious though: “show filesystems” did say something about mount file could not be read.

A bad foreboding struck my colleague and me: Filesystem corrupt? Without a bash there is not much you can do. So we downloaded a live linux cd, rebooted the VM into that livelinux, and saw it in an instance: The filesystem simply was full. 0 bytes free. Ok, that definitely is a reason for mySQL and tomcat not to start, and definitely a reason why nothing shows up in the log files.

Now with a bash at hand, we investigated, and our first thought where the log files. But we found only 20 MB worth of log files in /var/log. Everything rotated, gzipped and deleted as supposed. So what was going on?

Turns out vmware forgot about the tomcat logfiles. Tomcat saves its own logfiles into its own directory. And that directory was full of catalina-logfiles, going back to the very day the system was installed.

Simple fix: Delete the old junk no one needs, and the vShield Manager is rebooting like a charm.

So, vmware, I’d guess you have fixed that in a newer version already, although I have not checked yet. If not: Shame on you. Leaving admins out with no proper bash or other good way to determine what is going on. That is a shortfall on your side!

P.S.: No hard feelings though, vmware. I still love you 🙂

bridging ethernet interfaces under OS X

So, one of our domain printers was acting up and the printer department wanted me to sniff all the packets to and from the printer. So far so good, usually you could use a hub (still got one around? neither do I), or for example a switch with a monitoring or promiscuous port (not available? Welcome to the club!). Ok, now I could start arp spoofing the printer, but was too lazy to do that, besides: we have intrusion prevention up and running…

Take a good look at your machine. Is it a portable laptop like mine? Does it have any flavor of Unix or Linux on it? Well, so does mine, in this case Apple OS X (10.8.2). There’s got to be a way to bridge two ethernet interfaces, connect myself to the network and let the printer talk to the world through my laptop? You betcha!

Here’s what I did:

  • grab one of those USB or Thunderbolt Ethernet Adapters
  • ifconfig // to find out which interface is which
  • sudo ifconfig en3 up // bring up the USB/Thunderbolt adapter, or it won’t work!
  • sudo ifconfig bridge0 create
  • sudo ifconfig bridge0 up
  • sudo ifconfig bridge0 addm en0 addm en3
  • sudo sysctl -w net.inet.ip.forwarding=1

It works! Now we can sniff away with tcpdump or wireshark for example, and are sure that no packet is going to escape us…

And to undo what we just did:

  • sudo ifconfig bridge0 down
  • sudo ifconfig bridge0 deletem en0 deletem en3
  • sudo ifconfig bridge0 destroy
  • sudo sysctl -w net.inet.ip.forwarding=0

Many thanks to Chrissy from netnerds, who set me on the right track with her post about NAT on OS X 🙂

Site-to-Site VPN, or: IPSec via IPv6

Hey there,

more and more IPv6 addresses are assigned, and since we are using IPSec-tunnels to encrypt the traffic between our branch-offices, I was wondering ‘how far has the support for IPSec via IPv6 come’?

So, I checked it out, using our Astaro (now Sophos) Firewall at work and my M0n0wall at home.

First of all, you of course need IPv6 activated on both ends and need an active connection. Wether you get that native from your provider or, for example, through https://www.sixxs.net is up to you. If you see fe80:… addresses: These are the link local addresses and do not work for us here.

Setup on the Astaro (Sophos UTM):

  • Go to ‘Site-to-Site VPN’ -> ‘IPSec’, create a ‘Remote Gateway’. We use a Preshared Key for our test setup now, in a real setup you might want to use RSA or a certificate. For the gateway you use the IPv6 WAN address of the m0n0wall. Oh, and don’t forget to add the remote networks. (This can be the whole /48 for example, no need to use several /64).
  • Then go to ‘Connections’ and create a new connection, using our just created gateway. I use TrippleDES for a policy here.
  • If you hit ‘automatic firewall rules’ your remote network gets full access to your local network. If this is unwanted, don’t do it! You can create the rules you like under ‘Network Security’ -> ‘Firewall’

All done here!

Setup on the M0n0wall:

  • Go to ‘VPN’ -> ‘IPSec’ and click the + symbol to create a new tunnel
  • For the interface chose ‘WAN’, unless you are routing internal or something (the interface should have the same IP that you chose for the remote gateway on the Astaro).
  • Enter your local subnet, I chose my /48 here.
  • Enter remote gateway (again, WAN IPv6 from the astaro)
  • Phase 1: Use 3DES, MD5, DH Keygroup 5, Lifetime 7800, PreShared Key
  • Phase 2: 3DES, MD5, Lifetime 3600

These are the values taken from the pre-existing definition for 3DES on the Astaro. You could change that, but do it on both sides.

Now just create rules what traffic you want to allow through the tunnel and which not. Remember: Both sides must fit in order for traffic to go through.

All save, all encrypted, all IPv6.

Voilà: Enjoy your Site-to-Site IPv6 tunnel.