esxtop not working in OS X terminal

Also, as I did some troubleshooting lately and came across this issue, here is how to resolv the problem with the OS X terminal and esxtop:

Simply change the setting of the terminal emulation from xterm-256-color to just xterm. voila it works!

Thanks to Punching Clouds: http://www.punchingclouds.com/2013/01/30/esxtop-data-display-issues-with-osx-terminal-application/

Advertisements

vShield Manager 4.1 spikes in CPU usage, acting up…

Since a couple of weeks we let vSphere monitor our VMs in addition to the nagios monitoring we have had from the beginning.

The vShield Manager VM started to show odd behavior: CPU spikes to 100% every 10 to 30 minutes, then returning to normal. Operation worked as normal, nothing showed in the log files (although: you don’t really see the system log files in the interface).

Somewhere I read something about the mySQL database having problems, and a reboot would fix that. So, reboot… After all, what could go wrong, right? The single firewall VMs are working independent from the Manager, so nobody should even notice… so far so good.

vShield Manager VM did not boot up properly again. You could not login via the web interface. On the console you’d see only the message “System startup is not complete. Please logout and log back in after a few minutes.”

After some hours still the same. On the console (you don’t get a full bash, just a stripped down management shell that does not much) not much was showing. On thing made us curious though: “show filesystems” did say something about mount file could not be read.

A bad foreboding struck my colleague and me: Filesystem corrupt? Without a bash there is not much you can do. So we downloaded a live linux cd, rebooted the VM into that livelinux, and saw it in an instance: The filesystem simply was full. 0 bytes free. Ok, that definitely is a reason for mySQL and tomcat not to start, and definitely a reason why nothing shows up in the log files.

Now with a bash at hand, we investigated, and our first thought where the log files. But we found only 20 MB worth of log files in /var/log. Everything rotated, gzipped and deleted as supposed. So what was going on?

Turns out vmware forgot about the tomcat logfiles. Tomcat saves its own logfiles into its own directory. And that directory was full of catalina-logfiles, going back to the very day the system was installed.

Simple fix: Delete the old junk no one needs, and the vShield Manager is rebooting like a charm.

So, vmware, I’d guess you have fixed that in a newer version already, although I have not checked yet. If not: Shame on you. Leaving admins out with no proper bash or other good way to determine what is going on. That is a shortfall on your side!

P.S.: No hard feelings though, vmware. I still love you 🙂

bridging ethernet interfaces under OS X

So, one of our domain printers was acting up and the printer department wanted me to sniff all the packets to and from the printer. So far so good, usually you could use a hub (still got one around? neither do I), or for example a switch with a monitoring or promiscuous port (not available? Welcome to the club!). Ok, now I could start arp spoofing the printer, but was too lazy to do that, besides: we have intrusion prevention up and running…

Take a good look at your machine. Is it a portable laptop like mine? Does it have any flavor of Unix or Linux on it? Well, so does mine, in this case Apple OS X (10.8.2). There’s got to be a way to bridge two ethernet interfaces, connect myself to the network and let the printer talk to the world through my laptop? You betcha!

Here’s what I did:

  • grab one of those USB or Thunderbolt Ethernet Adapters
  • ifconfig // to find out which interface is which
  • sudo ifconfig en3 up // bring up the USB/Thunderbolt adapter, or it won’t work!
  • sudo ifconfig bridge0 create
  • sudo ifconfig bridge0 up
  • sudo ifconfig bridge0 addm en0 addm en3
  • sudo sysctl -w net.inet.ip.forwarding=1

It works! Now we can sniff away with tcpdump or wireshark for example, and are sure that no packet is going to escape us…

And to undo what we just did:

  • sudo ifconfig bridge0 down
  • sudo ifconfig bridge0 deletem en0 deletem en3
  • sudo ifconfig bridge0 destroy
  • sudo sysctl -w net.inet.ip.forwarding=0

Many thanks to Chrissy from netnerds, who set me on the right track with her post about NAT on OS X 🙂

managing iptables on linux

hey there,

as you manage more and more linux servers, you might stumble upon iptables firewalls. Especially with newer distros that happens more and more often, for example Fedora 17 and CentOS 6.3 have their iptables switched on by default.

If you now start to install services and server software, you might want to disable your firewall to test if the server responds, right? wrong!

The firewall is there for a reason, and should not just be switched off. One little trick I picked up is to always have a little shell script on hand that contains all the rules you want on your server and blocks everything else.

That way you can easily add an open port or close one, can rearrange the rules to your like and then run the script and have the firewall in place exactly as you want it to be.

This is especially helpful in case you mess up your rules or something goes wrong. You are up and running with your set of rules again in no time.

So, here goes a little basic firewall file on one of my test servers that runs a media wiki and for test purposes webmin on port 4444. It even takes into account ipv6 (same rules as for ipv4) and dumps the rules to the screen for you to see whats in place now.

all the best,

maybe

#!/bin/bash

#flush all rules
iptables -F
ip6tables -F

#allow 22 (ssh)
iptables -A INPUT -p tcp –dport 22 -j ACCEPT
ip6tables -A INPUT -p tcp –dport 22 -j ACCEPT

#allow 80 (http, the wiki)
iptables -A INPUT -p tcp –dport 80 -j ACCEPT
ip6tables -A INPUT -p tcp –dport 80 -j ACCEPT

#allow 4444 (webmin)
ip6tables -A INPUT -p tcp –dport 4444 -j ACCEPT
iptables -A INPUT -p tcp –dport 4444 -j ACCEPT

#set default policies for INPUT, FORWARD and OUTPUT chains
iptables -P INPUT DROP
ip6tables -P INPUT DROP
iptables -P FORWARD DROP
ip6tables -P FORWARD DROP
iptables -P OUTPUT ACCEPT
ip6tables -P OUTPUT ACCEPT

#set access for localhost
iptables -A INPUT -i -lo -j ACCEPT
ip6tables -A INPUT -i -lo -j ACCEPT

#accept packets belonging to established connections
iptables -A INPUT -m state –state ESTABLISHED,RELATED -j ACCEPT
ip6tables -A INPUT -m state –state ESTABLISHED,RELATED -j ACCEPT

#save settings
/sbin/service iptables save
/sbin/service ip6tables save

#list rules
iptables -L -v
ip6tables -L -v