Learn Python ( Install python, Pycharm and Integration with Notepad++)

Comments

Post a Comment

Popular posts from this blog

Zeppelin and Anaconda

Set Anaconda As Default Python Interpreter In Zeppelin Click  anonymous  in top right corner. Click  Interpreter . Scroll down to the  python  interpreter. Click  Edit . Locate  zeppelin.python . Set value to  /home/hadoop/anaconda/bin/python Now find the  spark  interpreter. Locate  zeppelin.pyspark.python . Set value to  /home/hadoop/anaconda/bin/python https://dziganto.github.io/zeppelin/spark/zeppelinhub/emr/anaconda/tensorflow/shiro/s3/theano/bootstrap%20script/EMR-From-Scratch/ This works..tested.. conda install -c calex sklearn-pandas P.S:  zeppelin 0.7.3 don't support spark 2.3. spark 2.3 is supported by zeppelin 0.8 which will be released soon zeppelin 0.7.3 doesn't support python 3.6.. now testing with spark 2.1 and python 3.5...so far so good.. Check:::::: zeppelin home/conf should have zeppelin-env.sh. Here you can specify spark home and zeppelin port Starting Apach...
Read more

The auxService:mapreduce_shuffle does not exist

Error and Probable Fix: 1. Error : org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The auxService:mapreduce_shuffle does not exist Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1 2015-05-02 13:57:39,982 Stage-1 map = 0%,  reduce = 0% 2015-05-02 13:57:49,511 Stage-1 map = 100%,  reduce = 100% Ended Job = job_1430542224940_0001 with errors Error during job, obtaining debugging information... Examining task ID: task_1430542224940_0001_m_000000 (and more) from job job_1430542224940_0001 Task with the most failures(4): ----- Task ID:   task_1430542224940_0001_m_000000 URL:   http://0.0.0.0:8088/taskdetails.jsp?jobid=job_1430542224940_0001&tipid=task_1430542224940_0001_m_000000 ----- Diagnostic Messages for this Task: Container launch failed for container_1430542224940_0001_01_000005 : org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The auxService:mapreduce_shuffle does not exist at sun...
Read more

Running Spark Standalone mode..an error ...Failed to connect to master localhost:8081 akka.actor.ActorNotFound: Actor not found for: ActorSelection

Spark Refresher [pts/0][16:01:21:hadoop@localhost ] ~>cd spark-1.5.0-bin-hadoop2.4 [pts/0][16:01:27:hadoop@localhost ] ~/spark-1.5.0-bin-hadoop2.4>ls bin          data       examples  logs          python     RELEASE  work CHANGES.txt  derby.log  lib       metastore_db  R          sbin conf         ec2        LICENSE   NOTICE        README.md  scala [pts/0][16:01:28:hadoop@localhost ] ~/spark-1.5.0-bin-hadoop2.4>jps 29286 Jps [pts/0][16:01:41:hadoop@localhost ] ~/spark-1.5.0-bin-hadoop2.4>cd sbin/ ===========optional if you need UI view and connecting via windows to a lunix box========== [pts/0][16:03:11:hadoop@localhost ] ~/spark-1.5.0-bin-hadoop2.4/sbin>vncserver xauth:  timeout in locking authority file /u01/hadoop/.Xauthority New 'localhost:1 ...
Read more