AWS Solutions Architect Certification

On the new exciting journey that I started in public cloud, today I earned AWS Solutions Architect (Associate Level) certificate. It was a bit more challenging than what I expected but it was fun! For those who want to pass the exam, in my opinion despite what’s said about the focus of exam on VPC, RDS, high availability and scalability; the truth is that you should get familiar with almost all of the services and update yourself with new ones. For example, to my surprise I didn’t have any direct question about RDS but instead 5 or 6 questions about SQS and SWF and 1 question about Kinesis! I suppose questions are randomly selected and others may have different experiences but it’s a good idea to know basics of all the services. Of course, VPC, security, high availability and scalability are super important and you must be fluent in them but all I say is that they are not enough for passing the exam. Also, expect more scenario sort of questions which include different concepts rather than direct one sentence questions that you may find in internet.

So, if you are preparing for this exam, work harder and good luck!

command line package manager in Mac and windows

Linux administrators enjoy a lot from command line tools to install, upgrade or remove packages. No need to say how much ‘apt-get‘ and ‘yum‘ are handy! If you are looking for something similar in Windows and Mac, here we go:

  • Chocolatey for windows : Chocolatey is a package manager for Windows (like apt-get or yum but for Windows). It was designed to be a decentralized framework for quickly installing applications and tools that you need
  •  Homebrew for Mac: Homebrew installs the stuff you need that Apple didn’t.

Top vBlog voting

Time goes fast! It was not a long time ago when I was listed in Virtualization and Cloud portal (vLaunchpad) and it’s my pleasure that I’m a candidate now. So, if you like this blog, please go and cast your vote here. Please remember that you should choose 10 items to proceed.You can also find winners for last year in vLaunchpad main page.

eJabberd Puppet Module for Debian

PuppetForge is a great repository to find and utilize required modules; however, sometimes you need to make some changes to satisfy your own requirements. In my case, I needed an ‘eJabberd’ module for Debian based machines. I found this useful module by Lee Boynton that worked very well in CentOS but apparently not in Debian/Ubuntu. Those who are familiar with eJabberd installation, know that it’s a bit tricky when it needs to use mySql as its storage and requires specific drivers, schema. I modified Lee’s module slightly and the proper mysql manifest is as follows. I have tested this in Debian Squeeze as well as Wheezy:

# Installs the native erlang mysql driver
class ejabberd::mysql(
    $lib_dir = $ejabberd::params::lib_dir
) inherits ejabberd::params {
    if !defined(Package['git']) {
        package { 'git':
            ensure => installed,
        }
    }
    if !defined(Package['erlang-rebar']) {
	case $::osfamily {
	    'redhat': {
	        package { 'erlang-rebar':
        	    ensure => installed,
        	}
	    }
            'debian': {
                package { 'erlang':
                    ensure => installed,
                }
                file { "/home/debs":
                        ensure => directory
                }

                file { "/home/debs/rebar_2.0.0-5_amd64.deb":
                    owner   => root,
                    group   => root,
                    mode    => 644,
                    ensure  => present,
                    source  => "puppet:///modules/ejabberd/rebar_2.0.0-5_amd64.deb"  
                }

                package { 'erlang-rebar':
                        provider => dpkg,
                        ensure => installed,
                        source => "/home/debs/rebar_2.0.0-5_amd64.deb"
                }

            }
        }
    }


    vcsrepo { '/usr/local/src/mysql':
        ensure      => latest,
        provider    => git,
        source      => 'https://github.com/processone/mysql.git',
        require     => Package['git'],

        # use first version which is compatible with ejabberd 2.1.x
        revision    => '967f3a0bb7'
    }

    exec { 'compile-mysql':
        command     => '/usr/bin/rebar compile',
        creates     => '/usr/local/src/mysql/ebin/mysql.beam',
        cwd         => '/usr/local/src/mysql',
        environment => 'HOME=/root',
        require     => [
            Package['erlang-rebar'],
            Vcsrepo['/usr/local/src/mysql'],
        ]
    }

    file { "${lib_dir}/ebin/mysql.beam":
        ensure  => present,
        source  => '/usr/local/src/mysql/ebin/mysql.beam',
        require => Exec['compile-mysql'],
    }
    file { "${lib_dir}/ebin/mysql_auth.beam":
        ensure  => present,
        source  => '/usr/local/src/mysql/ebin/mysql_auth.beam',
        require => Exec['compile-mysql'],
    }
    file { "${lib_dir}/ebin/mysql_conn.beam":
        ensure  => present,
        source  => '/usr/local/src/mysql/ebin/mysql_conn.beam',
        require => Exec['compile-mysql'],
    }
    file { "${lib_dir}/ebin/mysql_recv.beam":
        ensure  => present,
        source  => '/usr/local/src/mysql/ebin/mysql_recv.beam',
        require => Exec['compile-mysql'],
    }
}

HAProxy Load Balancing IIS with Sticky Session and SSL

HAProxy is a very good candidate for load balancing in a web cluster with high availability, even for Windows IIS servers! In its newer versions (1.5.x), HAProxy supports native SSL which makes it suitable for even enterprise level web applications with high traffic. It also supports sticky session which is useful when no session management is implemented. I know that the best option is to use centralized session management out of the box, but considering the fact that this central session manager will be point of failure (at least in IIS) and needs care, sticky session can be a good choice for some small to medium environments with short aged session applications.

Here, I will show how to configure HAProxy 1.5.x to support backend IIS servers with SSL (https) and sticky sessions.

– If you have IIS certificate, export it and use ‘openssl’ in Linux to convert it to appropriate format and put it in a protected directory.

– For SSL termination (HAProxy sends certificate to the users and takes over https protocol between user and load balancer), configurations is as follows:

  • frontend https-in
    bind *:443 ssl crt /etc/ssl/private/company.com.pem
    reqadd X-Forwarded-Proto:\ https
    default_backend application-backend

– To deploy sticky session, specify ’round robin’ as balancing policy and configure backend cluster part as follows. the key line is ‘cookie SERVERID insert indirect’:

  • backend application-backend
    balance roundrobin
    option httpclose
    option forwardfor
    cookie SERVERID insert indirect nocache
    server WEB-001 192.168.x.1:80 cookie A check
    server WEB-002192.168.x.2:80 cookie B check
    server WEB-003 192.168.x.3:80 cookie C check

To have more information about different policies and different session behaviours, read here.

vCloud Director and vCloud Automation Center (vCAC)

There has been many discussions about the future of vCloud Director and if VMware intends to deprecate it. There is not much information from VMware about its vision but recently there is a great emphasis on vCloud Automation Center for cloud management that is now embedded in vRealize Suite. It seems a great product, especially for enterprises considering that it integrates management of different infrastructures (even physical and public cloud) into a single platform; but I still have some questions when I look at both vCloud Director and vCAC in regards to service providers who offer services to completely different, critical isolated organizations. I have read this blog post by VMware that explains VMware vision of Cloud management and transition plan, but still unsatisfied!

By the way, it seems that sooner or later vCAC will be dominant solution for Cloud management and VMware will add features to satisfy different requirements rapidly. So, it’s a good time to get familiar with this product. I saw this post that is a very good practical introduction and hand out that explains and clarifies basic concepts in vCAC. There is also another simple, more detailed manual here. Of course, more information can be found in VMware documentation.

p.s – From business perspective, it’s a smart move from VMware to focus more on enterprises rather than service providers because cloud service provider market is and will be dominated by other players.

Mixing 802.1Q and 802.1ad in Linux

When it comes to networking, Linux kernel is really superior over Windows. Some will ask why? Apart from performance point of view, there are some great features in Linux that can not be deployed in Windows easily. To give an example, let’s think about 2 important features: support for VLAN and trunking (802.1q) and NIC teaming or Link aggregation (802.1ad).

As far as I know, Windows kernel doesn’t support 802.1q and it all depends on NIC driver and for 802.1ad Windows support starts from Windows 2012 which means it’s too young! and who knows how it works! but both are prolonged features in Linux kernel.

And these features are really useful; for example when one single computer needs to be part of different VLAN’s it needs to be connected to a trunk port on the switch; therefore should understand VLAN tags and decapsulate packets. This single computer can even act as a router between different VLAN segments. Connecting to different VLANs means more traffic, so it’s not a bad idea to double (as an example) its bandwidth by aggregating (bonding) two NIC’s to improve performance. I’m providing 2 links to show how to implement 802.1q and 802.1ad in a single Linux machine with 2 or more NIC’s:

And to have an idea about combining these 2 features, see:

NAT in Fenced vApps (vCloud Director)

An interesting feature in vCloud Director networking is the capability of creating a fenced vApp. Basically, it’s like having an extra  (in case you have one for Organization network which means routed) vShield router and firewall on the edge of vApp.

One of the coolest applications for fenced vApps is when you want to have identical machines (same IP and MAC) in your vDC; it means when you want to do a fast clone without customizing guest OS by changing IP’s and names, … In this case vApps are completely isolated while they can have connection to External networks or perhaps internet! See here for a how-to about creating fenced vApp.

After you created a fenced vApp, you will notice that the IP addresses in the vApp are in the same subnet with Organization Network (see the picture above), although a NAT gateway is operating between the vApp and Organization network. So when you want to do a DNAT (Destination NAT), there are 2 places you should configure. In the picture above, suppose you want to give access to a VM with IP 192.168.0.45 in Fenced vApp from External Network. Assume that Edge 1 got IP 192.168.0.3 (specified while fencing). First, you need to create appropriate rules in Edge Gateway of Organization Network, Edge 2 (if there is any) to NAT and open ports for the IP address of Edge 1 (192.168.0.3)

fenced1

Next step, you need to do NAT and open ports from Edge 1 to specific VM but this configuration is not in Edge Gateways of vDC (unlike Edge 2) but can be found in Networking Tab of the vApp itself.
Click on the vApp, go to Networking tab,

fenced2

right click on the selected network and choose ‘Configure Services’. there, you can define appropriate NAT and firewall rules.

fenced3

 

VMWorld 2014

As you may know, VMworld 2014 is going on with some big announcements. I didn’t have a chance to take part but fortunately we can find more information from a couple of active bloggers writing about new features, products and visions in VMware. So far, the interesting things to me are:

  • VMware Integration with OpenStack : Apparently, VMware is less flexible for developers comparing to public clouds and VMware is trying to mitigate this gap by OpenStack.
  • EVO: Rail – VMware converged Hardware : While there are some other smaller companies (VMware partners like Nutanix) who provide VMware appliances, I guess VMware sees a demand for this and wants to positively increase competition in converged hardware based on VMware. It’s also getting closer to a Software Defined Data Center. In EVO:RAIL there is a software layer which facilitates deployment of VMware ESXi’s and vCenter and managing VM’s. The interesting thing in EVO:RAIL is the use of VSAN. It’s more suitable for small to medium deployments I would say.

Key authentication with SSH Secure Shell

Non-commercial version of SSH Secure Shell (can be obtained here) from SSH Communications Security is a decent ssh client that I have used for many years in my experiments and academic works. It lacks PKI and PKCS functionality, but still safe for experiments! However; when it comes to public key authentication, it needs some tweaks to work. Here are the steps required to enable key authentication over a Linux host; given that Linux host settings allow public/private key authentication:

  1. Connect to the host using SSH Secure Shell (by password)
  2. In Secure Shell client, go to: Edit -> Settings -> User Authentication -> Keys and click on ‘Generate New’
    ssh1
  3. When generation is done, it will ask you to upload the public key to the host. Let it upload to ‘.ssh ‘ as destination folder.
    ssh2
  4. It assumes that the host has the appropriate SSH server for this client (the company has SSH server too) but since standard Linux servers use OpenSSH as SSH server, uploading the public key to the host is not enough and needs some modifications that follows.
  5. In Linux host, you will see that a public key (KeyAuthTest.pub in this case) is uploaded in ‘.ssh’ directory. For this to work, there are 2 ways:
    • Edit ‘KeyAuthTest.pub’ manually! and give it the right format. Remove these lines (or something like this) in the beginning:
      —- BEGIN SSH2 PUBLIC KEY —-
      Comment: “[3072-bit rsa, yyyy@xxxx, Thu Oct 04 2012 21:33:49]”
      And this at the end:
      —- END SSH2 PUBLIC KEY —-
      Also, you need to remove all the carriage returns (CR) in this file. Then add ‘ssh-rsa’ in the beginning of the file. The file would be something like:
      ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+…
      Finally, in shell append this file to the ‘authorized_keys’ file :
      cat ~/.ssh/KeyAuthTest.pub >> ~/.ssh/authorized_keys
    • Second approach: convert the key to proper OpenSSH format automatically and append it to the file:
      ssh-keygen -i -f ~/.ssh/KeyAuthTest.pub  >>  ~/.ssh/authorized_keys

Now, you will be able to connect to the host, using this public key.