tag:blogger.com,1999:blog-28074687681886345842026-04-13T12:24:35.999+05:30Gaining knowledge, is the first step to wisdom. Sharing it, is the first step to humanity.Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.comBlogger83125tag:blogger.com,1999:blog-2807468768188634584.post-33470582941084647862017-06-18T17:17:00.001+05:302017-06-18T17:17:18.736+05:30

We are going to install R software package on Linux and for that we are going to use SLES11 SP3 and R 3.3.3. A fresh install of SLES will not have any development packages and hence, it is assumed that the SDK repo has been enabled to resolve the dependencies. Java should be installed as a pre-requisite dependency package and it's installation is not covered in this tutorial. However, you can

Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.com6Pune, Maharashtra, India18.5204303 73.85674369999992518.2795358 73.534020199999929 18.7613248 74.17946719999992tag:blogger.com,1999:blog-2807468768188634584.post-13076188862061669562017-06-17T22:33:00.000+05:302017-06-23T09:27:23.024+05:30

In my previous post I have demonstrated the installation of multi-node Cloudera cluster. Here I will demonstrate how to kerberize a Cloudera cluster. Introduction to Kerberos Kerberos is a network authentication protocol that allows both users and machines to identify themselves on a network, defining and limiting access to services that are configured by the administrator. Kerberos uses

Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.com3Pune, Maharashtra, India18.5204303 73.85674369999992518.2795358 73.534020199999929 18.7613248 74.17946719999992tag:blogger.com,1999:blog-2807468768188634584.post-35535698425644300432017-05-01T10:31:00.001+05:302017-06-18T07:57:50.084+05:30

Here we are going to setup a multi-node fully distributed Cloudera Hadoop cluster configured with "MySQL" as external database. We will also configure our cluster to authenticate using Kerberos and authorize using OpenLDAP as additional security implementations. ENVIRONMENT SETUP & CONFIGURATION Operating System: CentOS-6.8 Cloudera Manager Version: 5.9.1 CDH Version: 5.9.1 We will

Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.com2Pune, Maharashtra, India18.5204303 73.85674369999992518.2795358 73.534020199999929 18.7613248 74.17946719999992tag:blogger.com,1999:blog-2807468768188634584.post-29412989037131326872017-01-05T10:07:00.001+05:302017-01-05T10:10:13.513+05:30

"Ransomeware is more about manipulating vulnerabilities in human psychology than the adversary's technological sophistication." --- James Scott, Sr. Fellow, Institute for Critical Infrastructure Technology According to Yahoo's latest revelation, two years back half a billion Yahoo user accounts' security was compromised. Twitter experienced an outage due to massive DDoS attack on Dyn.

Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.com0Hyderabad, Telangana, India17.385044 78.48667116.9002155 77.841224 17.8698725 79.132118tag:blogger.com,1999:blog-2807468768188634584.post-43451477372096085502016-07-24T22:39:00.000+05:302017-04-24T06:11:53.859+05:30

Various approaches have been taken to meet the increased demand for highly reliable infrastructures, providing "five 9s" standard of availability. The concept of high-availability has long been providing highly reliable solution to serve critical systems with single point of failures and handle their increased system load in the most efficient ways. NameNode was a single point of failure in

Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.com1Hyderabad, Telangana, India17.385044 78.48667116.9002155 77.841224 17.8698725 79.132118tag:blogger.com,1999:blog-2807468768188634584.post-27083392786449618962016-05-22T09:48:00.000+05:302016-05-24T04:16:33.638+05:30

LXD container on a single host is just like "chroot on steroids". LXD's main goal is to provide an experience that is similar to virtual machines and hypervisors excluding the hardware virtualization technique. My previous port "Layman's Guide for LXD - Canonicals OpenSource Container Hypervisor [Part-I]" provides an introduction to containers and LXD. This article will be based mostly on

Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.com3Hyderabad, Telangana, India17.385044 78.48667116.9002155 77.841224 17.8698725 79.132118tag:blogger.com,1999:blog-2807468768188634584.post-46142647655503674892016-05-07T09:10:00.000+05:302016-05-07T09:33:12.565+05:30

Introduction Certainly, "container" is the new buzz word among techies. Everyone is talking about Docker, LXC and LXD. The continous need to reduce costs, optimize performance, as well as maintain the data availability combined with data integrity has been the most prominent need for most of the organizations leading to convergence of various virtualization concepts to develop an efficient

Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.com2Hyderabad, Telangana, India17.385044 78.48667116.9002155 77.841224 17.8698725 79.132118tag:blogger.com,1999:blog-2807468768188634584.post-12711843424810873882016-04-22T01:00:00.000+05:302016-04-22T03:25:25.332+05:30

Here, I will explain behind the scenes of job execution process in Hadoop MapReduce or MRv1 (MapReduce version 1), from the time user fires a job to the time when the job is executed on the slave nodes. MapReduce is a "programming model/software framework" designed to process large amount of data in parallel by dividing the job into a number of independent data local tasks. The term data

Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.com71Hyderabad, Telangana, India17.385044 78.48667116.9002155 77.841224 17.8698725 79.132118tag:blogger.com,1999:blog-2807468768188634584.post-4789103532166925572015-07-13T00:00:00.002+05:302015-07-13T00:00:52.127+05:30

This is a quickstart guide to install Globus Toolkit 6.0 using yum on CentOS-6.6. The steps mentioned in “GT 6 Quickstart Guide” which is the official documentation of Globus Toolkit were followed during the installation process.The GT 6.0 release provides both source and binary RPM packages for CentOS which can be downloaded from here. We have two two servers out of which one will act as a

Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.com2Hyderabad, Telangana, India17.385044 78.48667116.9002155 77.841224 17.8698725 79.132118tag:blogger.com,1999:blog-2807468768188634584.post-75211575501237296142015-01-12T01:25:00.001+05:302015-01-14T17:18:16.637+05:30

After configuring an automatic failover HA with ZooKeeper and NFS, we will now configure an automatic failover HA with ZooKeeper and QJM. We will use the Quorum Journal Manager to share edit logs between active and standby namenodes. Any namespace modification done by active namenode is recorded by the journal nodes. These journal node daemons can be run alongside any other daemons like

Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.com9Hyderabad, Telangana, India17.385044 78.48667116.9002155 77.841224 17.8698725 79.132118tag:blogger.com,1999:blog-2807468768188634584.post-23307393029648576902015-01-11T20:27:00.001+05:302016-07-14T08:20:56.123+05:30

In my previous post we had configured a manual failover hadoop cluster. Now we being clear with the hadoop high availability features, we will slowly proceed towards configuring automatic failover cluster. Prior to that we need 'zookeeper'. What is ZooKeeper? An excerpt from Apache ZooKeeper website - ZooKeeper is a centralized service for maintaining configuration information, naming,

Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.com2Hyderabad, Telangana, India17.385044 78.48667116.9002155 77.841224 17.8698725 79.132118tag:blogger.com,1999:blog-2807468768188634584.post-78822521323504614892015-01-11T20:00:00.001+05:302015-01-12T01:29:28.154+05:30

Since manual failover mechanism was unable to automatically trigger a failover in cases of namenode failure, automatic failover mechanism made sure to provide a hot backup during a failover. This was overcome by zookeeper. We have already covered zookeeper installation & configuration in my previous post. To configure an automatic failover ha cluster we need more than one odd number of

Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.com16Hyderabad, Telangana, India17.385044 78.48667116.9002155 77.841224 17.8698725 79.132118tag:blogger.com,1999:blog-2807468768188634584.post-64821666117199895502015-01-11T19:45:00.002+05:302015-01-11T20:37:59.653+05:30

In my last post we had configured Hadoop Federation Cluster in a fully distributed mode. Next we will go for a fully distributed manual failover hadoop HA cluster in this post. I will skip the hadoop and java installation part as we have already gone through those a couple of times in my previous posts. For further learning we will use the hardware configuration mentioned in the below table.

Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.com10Hyderabad, Telangana, India17.385044 78.48667116.9002155 77.841224 17.8698725 79.132118tag:blogger.com,1999:blog-2807468768188634584.post-75196052768872274592015-01-11T19:40:00.000+05:302015-01-11T19:40:52.081+05:30

This post will assist you in installing Oracle Java 8 in RHEL 5, CentOS 5 and Ubuntu 14.04. Java Archive Download Download the latest Java SE Development Kit 8 from it's official download page. Java Installation Using Alternatives RHEL/CentOS Extract the tarball and we will install it in /opt directory. root:~# cd /opt root:~# tar -xzvf jdk-8u25-linux-x64.tar.gz root:~#

Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.com0Hyderabad, Telangana, India17.385044 78.48667116.9002155 77.841224 17.8698725 79.132118tag:blogger.com,1999:blog-2807468768188634584.post-78834711874261238132015-01-04T00:00:00.000+05:302015-01-11T20:42:53.994+05:30

Federation Concepts “HDFS Federation improves the existing HDFS architecture through a clear separation of namespace and storage, enabling generic block storage layer. It enables support for multiple namespaces in the cluster to improve scalability and isolation. Federation also opens up the architecture, expanding the applicability of HDFS cluster to new implementations and use cases. I

Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.com0Hyderabad, Telangana, India17.385044 78.48667116.9002155 77.841224 17.8698725 79.132118tag:blogger.com,1999:blog-2807468768188634584.post-5175975535533093042014-06-05T19:30:00.000+05:302015-01-22T21:36:47.889+05:30

In this tutorial I will describe the steps for setting up a multi-node i.e. five nodes Hadoop cluster running on Ubuntu. I have created this setup inside a VMware ESXi box. There are five virtual machines all having the same common configurations like 20GB HDD, 512MB RAM, etc. For this I will be using Ubuntu 14.04 LTS as operating system and Hadoop 1.2.1. All the machines will be

Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.com3Hyderabad, Andhra Pradesh, India17.385044 78.48667116.9002155 77.841224 17.8698725 79.132118tag:blogger.com,1999:blog-2807468768188634584.post-38889643517894290422014-06-05T19:00:00.000+05:302014-06-06T01:27:36.965+05:30

In this tutorial I will demonstrate how to install and run Single-Node Hadoop Cluster in Ubuntu 14.04. JAVA INSTALLATION As a requirement java needs to be installed. user@hadoop-lab:~$ sudo apt-get install openjdk-7-jdk HADOOP USER & GROUP CREATION Create a dedicated user account and group for hadoop. user@hadoop-lab:~$ sudo groupadd hadoop user@hadoop-lab:~$ sudo useradd -m -d

Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.com3Hyderabad, Andhra Pradesh, India17.385044 78.48667116.9002155 77.841224 17.8698725 79.132118tag:blogger.com,1999:blog-2807468768188634584.post-866114032620824362014-05-28T19:13:00.000+05:302014-06-06T08:53:51.671+05:30

This tutorial demonstrates how to setup an Apache Hadoop 1.2.1 Cluster using Oracle Solaris 11.1 Virtualization Technology or Zones. I am running this setup inside Oracle VM VirtualBox 4.3.10 on Ubuntu 12.04 and the guest machine is running Oracle Solaris 11. Namenode will run inside Global Zone whereas we will be configuring 4 guest zones with almost same configuration for separate “Secondary

Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.com2Hyderabad, Andhra Pradesh, India17.385044 78.48667116.9002155 77.841224 17.8698725 79.132118tag:blogger.com,1999:blog-2807468768188634584.post-65423674600705910902014-05-28T19:08:00.002+05:302014-05-28T19:52:42.725+05:30

A repository is a location where the clients publish and retrieve packages. There are two ways to obtain copy of Oracle Solaris 11.1 IPS (Image Packaging System) repository image, one method is to download the repository image from the Oracle Solaris 11 Website, create a local repository and the second method is to retrieve the repository directly from the internet. Here we dont have internet

Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.com1Hyderabad, Andhra Pradesh, India17.385044 78.48667117.385044 78.486671 17.385044 78.486671tag:blogger.com,1999:blog-2807468768188634584.post-62247482296836176122013-04-07T13:38:00.000+05:302013-04-07T13:38:07.109+05:30

Linux kernel 3.8.6 has recently released on April 05, 2013 with lots of bugfixes and improvements. The changelog contains all the improvements and bugfixes made in the kernel. Below mentioned are some important fixes and improvements. * ipv6: fix bad free of addrconf_init_net * net: ethernet: cpsw: fix erroneous condition in error check

Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.com1Hyderabad, Andhra Pradesh, India17.385044 78.48667116.9002075 77.841224 17.8698805 79.132118tag:blogger.com,1999:blog-2807468768188634584.post-80584450171278225962013-04-01T13:23:00.002+05:302013-04-01T13:23:49.550+05:30

Linux kernel 3.8.5 has recently released on March 28, 2013 with lots of bugfixes and improvements. The changelog contains all the improvements and bugfixes made in the kernel. Below mentioned are some important fixes and improvements. * drm/radeon: fix backend map setup on 1 RB trinity boards * drm/radeon: fix S/R on VM systems (cayman/TN/SI) * ARM: tegra: fix register

Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.com0Hyderabad, Andhra Pradesh, India17.385044 78.48667117.385044 78.486671 17.385044 78.486671tag:blogger.com,1999:blog-2807468768188634584.post-31176319008487046232013-04-01T13:23:00.001+05:302013-04-01T13:23:33.368+05:30

Linux kernel 3.8.4 has recently released on March 20, 2013 with lots of bugfixes and improvements. The changelog contains all the improvements and bugfixes made in the kernel. Below mentioned are some important fixes and improvements. * ALSA: seq: Fix missing error handling in snd_seq_timer_open() * tty: serial: fix typo "SERIAL_S3C2412" * ext3: Fix format string issues

Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.com1Hyderabad, Andhra Pradesh, India17.385044 78.48667117.385044 78.486671 17.385044 78.486671tag:blogger.com,1999:blog-2807468768188634584.post-16218083259054531542013-04-01T13:23:00.000+05:302013-04-11T16:56:15.194+05:30

The operating system is said to be stable when both the software and hardware of a machine or a system go parallel hand in hand. Your favorite operating system or the best hardware that you opt for alone cannot outperform to provide the best performance. Hence, if we choose for the best hardware we have to tune the operating system that we install to run efficiently on that hardware.

Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.com0Hyderabad, Andhra Pradesh, India17.385044 78.48667116.9002075 77.841224 17.8698805 79.132118tag:blogger.com,1999:blog-2807468768188634584.post-87185521941107774942013-03-12T08:07:00.000+05:302013-04-11T16:35:06.369+05:30

There are several articles around the web to improve the performance of Ubuntu Linux. I have jotted down some common tweaks that I too have tried on my system and due to which I have found quite a significant change in the performance of my system. It has been possible only because Linux kernel and Ubuntu are flexible enough to let you make the modifications as you want on the fly. I

Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.com2Hyderabad, Andhra Pradesh, India17.385044 78.48667116.9002075 77.841224 17.8698805 79.132118tag:blogger.com,1999:blog-2807468768188634584.post-10886182839113781432013-03-01T01:00:00.000+05:302013-04-01T13:21:23.075+05:30

Linux kernel 3.7.6 has recently released on February 04, 2013 with lots of bugfixes and improvements. The changelog contains all the improvements and bugfixes made in the kernel. Below mentioned are some important fixes and improvements. drm/i915: fix FORCEWAKE posting reads ALSA: hda - Fix non-snoop page handling ALSA: hda - fix inverted internal mic on Acer

Baban Gaigolehttp://www.blogger.com/profile/10440029644921138328noreply@blogger.com0Hyderabad, Andhra Pradesh, India17.385044 78.486671-35.729976 -4.1305164999999988 70.500064000000009 161.1038585