Package your Python Code whl

Comments

Post a Comment

Popular posts from this blog

The auxService:mapreduce_shuffle does not exist

Error and Probable Fix: 1. Error : org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The auxService:mapreduce_shuffle does not exist Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1 2015-05-02 13:57:39,982 Stage-1 map = 0%,  reduce = 0% 2015-05-02 13:57:49,511 Stage-1 map = 100%,  reduce = 100% Ended Job = job_1430542224940_0001 with errors Error during job, obtaining debugging information... Examining task ID: task_1430542224940_0001_m_000000 (and more) from job job_1430542224940_0001 Task with the most failures(4): ----- Task ID:   task_1430542224940_0001_m_000000 URL:   http://0.0.0.0:8088/taskdetails.jsp?jobid=job_1430542224940_0001&tipid=task_1430542224940_0001_m_000000 ----- Diagnostic Messages for this Task: Container launch failed for container_1430542224940_0001_01_000005 : org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The auxService:mapreduce_shuffle does not exist at sun...
Read more

Zeppelin and Anaconda

Set Anaconda As Default Python Interpreter In Zeppelin Click  anonymous  in top right corner. Click  Interpreter . Scroll down to the  python  interpreter. Click  Edit . Locate  zeppelin.python . Set value to  /home/hadoop/anaconda/bin/python Now find the  spark  interpreter. Locate  zeppelin.pyspark.python . Set value to  /home/hadoop/anaconda/bin/python https://dziganto.github.io/zeppelin/spark/zeppelinhub/emr/anaconda/tensorflow/shiro/s3/theano/bootstrap%20script/EMR-From-Scratch/ This works..tested.. conda install -c calex sklearn-pandas P.S:  zeppelin 0.7.3 don't support spark 2.3. spark 2.3 is supported by zeppelin 0.8 which will be released soon zeppelin 0.7.3 doesn't support python 3.6.. now testing with spark 2.1 and python 3.5...so far so good.. Check:::::: zeppelin home/conf should have zeppelin-env.sh. Here you can specify spark home and zeppelin port Starting Apach...
Read more

Statistics Learning for Data Science-1 of N

Cited from http://www.maxvalue.com/tip104.htm Geometric Mean https://www.mathsisfun.com/definitions/geometric-mean.html A special type of average where we multiply the numbers together and then take a square root (for two numbers), cube root (for three numbers) etc. Example: The Geometric Mean of 2 and 18 = √(2 × 18) = 6 The Geometric Mean is useful when we want to compare things with very different properties. Harmonic Mean The harmonic mean is a better "average" when the numbers are defined in relation to some unit. The harmonic mean formula is: In petroleum engineering, the harmonic mean is sometimes the better "average" for vertical permeability                    with horizontally-layered bedding.  Cited from https://statistics.laerd.com/statistical-guides/descriptive-inferential-statistics.php Descriptive and Inferential Statistics Descriptive statistics is the term given...
Read more