TechMusings (BigData,Hadoop,Pig,Hive,DataScience,IoT,EAI,SOA,J2EE)

Installing R in Ubuntu 14 LTS sudo apt-get install r-base Let's verify that installation was  correct. Start an R session. ~$ R R version 3.0.2 (2013-09-25) -- "Frisbee Sailing" Copyright (C) 2013 The R Foundation for Statistical Computing Platform: x86_64-pc-linux-gnu (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details.   Natural language support but running in an English locale R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R. > OOPS!!!.. This installed an old one... i.e /etc/apt/sources.list had a old reference to R  ...

1. Download and configure JDK1.8 2. Download and configure Hadoop 2.6 3. Download and configure Spark for Hadoop2.6     Start master and access the url via http://localhost:8080.     Start slave pass the spark://localhost:<port> seen in the master 4.Download and configure Kafka (http://kafka.apache.org/documentation.html#quickstart) https://www.apache.org/dyn/closer.cgi?path=/kafka/0.10.0.0/kafka_2.11-0.10.0.0.tgz 5. Start producer/broker..test using a message and check if spark gets the message. a) a uses ZooKeeper so you need to first start a ZooKeeper server if you don't already have one. You can use the convenience script packaged with kafka to get a quick-and-dirty single-node ZooKeeper instance. > bin/zookeeper-server-start.sh config/zookeeper.properties   b) Now start the Kafka server: > bin/kafka-server-start.sh config/server.properties [2016-07-13 04:54:07,557] INFO [Kafka Server 0], started (kafka.server.Kafka...