After deploying Serengeti Management Server using OVF, it would be possible to install Big Data Extensions plugin in vCenter Server. Follow the instructions for doing this. After a Logout and Login (necessary), you will see Big Data Extensions plugin in vSphere Web Client Home, Inventories.
Next step is connecting Big Data Extensions plugin to Serengeti Management Server. For a successful connection, 2 conditions should be met: First, Serengeti Management Server should be reached by vCenter in terms of networking. Ping is not blocked by default, so vCenter should be able to ping Serengeti Server. Secondly, SSO Lookup Service URL should be correct. If it’s not correct, you will get an error like: “Connection failed! Check the server has enabled SSO.” If you are not sure if SSO URL is entered and works properly, try to re-enter SSO Lookup service URL manually:
Login to Serengeti server and issue these commands:
– sudo /opt/serengeti/sbin/EnableSSOAuth https://VC-FQDN:7444/lookupservice/sdk
– sudo service tomcat restart
In my case, I first got the following error:
“com.vmware.vim.vmomi.core.exception.CertificateValidationException: Server certificate assertion not verified and thumbprint not matched
Return code is: SslHandShakeFailed
Please check if sso lookup service https url is correct, and sso service work normally.”
The issue was that I entered URL by IP and certificate was generated using FQDN. So, I recommend using FQDN of vCenter and not IP address.
Also, check the time of Serengeti server to by synced with vCenter.
To install VMware vSphere Big Data Extensions 1.1, if you satisfy the requirements mentioned in vmware document, go ahead with installation by deploying Big Data Extensions OVA as documented. But attention that:
- Better to create a specific Resource Pool for your Big Data Cluster and specify the total amount of resources you want to assign and apply possible limits.
- Create a port group dedicated to Big Data Extensions as a communication link between management servers and working VMs.
- When deploying Big Data Extensions Management server (OVA), ‘setup networks’ asks you to assign a destination port group. Note that: Management Network will use this network to communicate with vCenter server. So, if you use VLAN tags, the port group should be in the same VLAN (use same VLAN id) with vCenter network. If vCenter can not see Big Data Management server and vice versa, integration will not be made properly.
- In ‘Customize template’ step, there are 2 important settings: SSO service and Management Server IP address. So, from right-pane open ‘VC SSO Lookup Service URL’ and ‘Management Server Networks Settings’. Enter appropriate values. For SSO Lookup Service URL, use vCenter server with the same format (if you didn’t change defaults), I mean port 7444/lookupservice/sdk. Use FQDN of vCenter and not IP address or certificate will not be accepted and you will see errors for connecting Big Data Extensions plugin to Serengeti server in the future.
Nowadays Big Data is everywhere. Many are talking about it and they are enthusiastic to deploy a Big Data instance in their environments. Installation and deployment can be difficult though. The fact is that there is no official mature Big Data standard and lots of open source standards are being developed, sometimes independently. Even if we accept Apache Hadoop as the dominant standard of Big Data, implementing Hadoop is a big challenge for IT departments. For example, according to this article: In addition to the technical challenges of deploying large-scale Hadoop systems and applications, another issue Manor cited is that IT operations often work in silos, with separate teams handling systems administration, database administration, storage, networking, security and application development. That approach can lead to problems in managing Hadoop clusters.
And it’s exactly where Virtualization, Cloud and SDN can help: integrating multiple administration tasks in a unified control center. And VMware did this beautifully by putting together all required Hadoop components in a package to create Clusters and control and scale the Hadoop Clusters by using VMware vSphere Big Data Extensions. Hadoop clusters which are created by vSphere Big Data Extensions are scalable, elastic and flexible. You can easily separate compute and data nodes or increase the number of working machines and so on. vSphere Big Data Extensions utilizes the open source project Serengeti that was initiated by VMware to implement Hadoop on a virtual platform. Serengeti or better to say VMware vSphere Big Data Extensions deploys HDFS, MapReduce, Pig, Hive and HBase on vSphere infrastructure.
You can find general installation instructions here, but there are some implementation tips which will help in vSphere Big Data Extensions installation. In my upcoming posts I will show the required steps and important considerations during installation.