IP LAYER MONITORING IN VMWARE VSPHERE – 2

2 posts earlier, I talked about NetFlow in VMware 5.x and how to enable it in vSphere dvSwitch. I have also shown how you can send IP traffic flow information to a NetFlow collector. Nowadays, there are lots of commercial NetFlow collectors available; however, in this post I will introduce a simple, open-source NetFlow collector which you can use in your VMware environment to analyze IP traffic. This pretty piece of software is: ‘nfdump

As it’s shown, Nfdump has 2 major elements: ‘nfcapd‘ which is a daemon to gather and store relevant packets and ‘nfdump‘ which collects packets from all the daemons and interprets them. Apparently, nfcapd and nfdump could run on different machines and there could be multiple daemons but in case of VMware vSphere, it depends solely on the number of dvSwitches. If there is only one distributed switch, all the IP traffic flow information from all portgroups in that dvSwitch will be forwarded to one nfcapd. For test purposes, I also deployed both nfdump and nfcapd on a single linux machine but in cases that traffic is high, it maybe a good idea to deploy them on two different machines. Of course nfdump should have access to the storage in that case.

After installation, first you need to run daemon and specify a port and directory to store ip traffic information. Apparently, nfcapd will store information in binary. The command is simple, something like this:

  • nfcapd -w -D -l /var/netflow/dvswitch -p 23456

Then, daemon will run and listen to the specified port: 23456. If you have configured dvSwitch correctly (by specifying ip address of linux machine and 23456 as port) and activated monitoring on some portgroups in vCenter, this daemon will generate a couple of files in that directory.
Now, whenever you want to view the captured ip traffic flows, you should run nfdump. Since there are lots of files in that directory, you can interpret the whole directory using -R option with this command:

  • nfdump -R /var/netflow/dvswitch/

Filtering in nfdump is also possible, pretty much the same as tcpdump and you can view traffics of interest. You can find more information on nfdump website.

To view NetFlow captured traffic visually, you can mix nfsen with nfdump. It uses information that is dumped by daemon and utilizing rrdtool, it visualizes traffic flow. Installation is not difficult and you can see more information on their website. I’m really satisfied by this beautiful combination of nfdump and nfsen and if you intend to use NetFlow for monitoring, I recommend trying them. Good Luck!

Advertisements

IP Layer Monitoring in VMware vSphere – 1

Following my last post about administration and monitoring tasks in VMware 5, I will talk about another promising feature of VMware vSphere 5.x: supporting NetFlow. NetFlow is a network protocol developed by Cisco for collecting IP traffic information. NetFlow has become an industry standard for traffic monitoring.
As I wrote earlier, cloud/network engineers would like to have the same capabilities in virtualization as they have in physical networks and nowadays NetFlow is turning out to be the new trend in producing networking devices such as switches. In the same way switches support NetFlow, VMware implemented NetFlow that can be enabled on vSwitches, specifically very useful in Distributed switches. Good to mention that from version 5.1 VMware also supports newer version of NetFlow which is IPFIX.  You can find more information about NetFlow by itself on the internet.
Configuring NetFlow in VMware vSphere is a 2 step process:

  1. Configure NetFlow properties on the dvSwitch.

    Hints:
    – Port is a UDP port which NetFlow collector will listen on. In NFDUMP, it is 23456! by default.
    – Of course, IP session between dvSwitch and NetFlow collector should be established in a proper way. I mean dvSwitch should see NetFlow collector.

  2. Enable NetFlow on the specific dvPort group.netflow

That’s it. In the next post, I will show how you can use a free, simple NetFlow Analyzer (nfdump, nfsen) to gather and display information about IP traffic flows in your dvSwitch.

Deploying IDS in VMware vSphere

As a network or cloud administrator in VMware environment, we would like to have the same capabilities we’ve got in a physical network. One of the most important tasks is network traffic monitoring and inspection control. Let’s say you want to install a network Intrusion Detection System (like SNORT) to monitor the traffic of a specific Virtual Data Center in vCloud environment that is translated to monitoring a specific VLAN or port group in VMware vSphere. Fortunately, VMware 5.x provides these features but apparently implementing these features is beyond VMware vCloud Director operations and it’s part of infrastructure administration tasks introduced in vSphere 5.x.
Since normally there is a port group in Distributed Virtual Switch defined by vCloud Director for each virtual data center, let’s talk about port groups in VDS. You may have noticed that when you want to create a port group in a distributed switch, you can define some security policy and one of the policies is enabling ‘Promiscuous Mode’. This is exactly equivalent to enabling promiscuous mode in a physical switch. So, as shown in the following picture, a port group can be edited to enable this mode (in vSphere Web client).

promisc

The only concern is that promiscuous mode should be defined on a port group or the whole distributed switch and not on a particular port. Doing this will cause all the traffic to be forwarded to all of the VM’s in that port group! and apparently it’s a security risk because we would like to forward the traffic to only one specific VM (port) which is our IDS. A work-around here would be to define a new port group with the same VLAN ID of the port group/VLAN we would like to monitor with the exact same configuration, then enable promiscuous mode for this newly defined port group and place the IDS VM in this port group. Because VLAN ID is the same, only IDS VM would see all the traffic. That’s an easy trick! BUT I don’t know how this trick works in some vCloud port groups that use VCDNI-backed port groups instead of VLAN-backed network pools because as I understood, VCDNI is kind of encapsulation introduced by vCloud Director and I’m not sure if a port group that is created inside vCenter can decapsulate packets. I didn’t find enough information, so I will test this out and report it in this blog.

Another approach is to use Port Mirroring feature of a VDS. Using this method it’s possible to specify source ports which need to be monitored and destination port/ports where IDS is located.

This solution is explained in detail in the following link:

vSphere 5.1 – VDS Feature Enhancements – Port Mirroring

Sticky sessions in vShield Edge Gateway Load Balancer

One of the features of edge gateways in VMware vCloud Director is the capability of implementing load balancer for HTTP, HTTPS and TCP-based applications in a virtual data center. For web applications (in specific HTTP), session management is an important matter. If web developers don’t implement session management in application level (using database, … to store sessions) and rely on Cookies, load balancer could be an issue. In these cases, network administrators are asked to configure load balancer with sticky session. Simply it means that if a client is forwarded to a web server for the first time (especially login page), it should stick to that specific server in later web requests. If it doesn’t happen, user may be forced to login again that would be frustrating!

By the way, when it comes to configuring vShield Edge Gateway to do load balancing, there is no obvious option to choose Sticky Session but it’s possible to do this by specifying proper value for Cookie name in the Virtual Server. As it’s shown in the picture, the procedure is as follows. I assume that you already know how to implement Load Balancer by creating Pool Servers and Virtual Server. See this link fore more information on how to create Load Balancer.

 

lb_vcns

  1. Right Click on the Edge Gateway and choose ‘Configure Services’
  2. Select ‘Load Balancer’ tab
  3. Go to ‘Virtual Servers’ section
  4. Edit selected Virtual Server
  5. Choose ‘Cookie’ as Persistence Method instead of default ‘None’
  6. Type proper value as Cookie Name; i.e, ‘ASP.NET_SessionId’ for .NET application, ‘PHPSESSID’ for PHP, … (ask your developer)

Software Defined Networking

Last week I attended a seminar about SDN (Software Defined Networking) and SDDC (Software Defined Data Center) and I met some high profile people from high profile companies. It seems this topic will be hot in coming years and many manufacturers and providers are coming in to this road. The good news is that there are some standards like OpenFlow managed and maintained by Open Networking Foundation (ONF) and OpenStack that will help in orchestration and inter-operatability to the benefit of customers.
Although, there are some different ideas about the approaches to SDN; for example VMware likes to implement SDN in an all-software solution (NSX) , while Cisco (and other device manufacturers) apparently prefers hardware implemented devices which support SDN. For this latter one, imagine that you have a SDN-enabled switch with some API’s that you can program it to perform in your desired way. That’s cool! Maybe, somethings like load balancing or geofencing can be implemented on the fly by using these APIs in a networking appliance!
For someone with hardware background like myself, this hardware approach seems more attractive and I’m thrilled how it goes. As a matter of fact, a while ago I was thinking: if we can have a tiny device doing a lot of things that can be programmed by developers (I meant Smartphone), why we don’t do the same with more advanced equipments like networking devices?  And now it’s coming to the reality! Smart switch or router! combining them with Virtualization and Cloud and on-demand services, customers can implement interesting functionalities which are more cost effective and agile. HP networking was talking about HP SDN App Store! You see! I’m not an advocate of HP but as a result of this, maybe we see a revolution in networking area!

Related articles