Package your Python Code whl

Comments

Post a Comment

Popular posts from this blog

The auxService:mapreduce_shuffle does not exist

Error and Probable Fix: 1. Error : org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The auxService:mapreduce_shuffle does not exist Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1 2015-05-02 13:57:39,982 Stage-1 map = 0%,  reduce = 0% 2015-05-02 13:57:49,511 Stage-1 map = 100%,  reduce = 100% Ended Job = job_1430542224940_0001 with errors Error during job, obtaining debugging information... Examining task ID: task_1430542224940_0001_m_000000 (and more) from job job_1430542224940_0001 Task with the most failures(4): ----- Task ID:   task_1430542224940_0001_m_000000 URL:   http://0.0.0.0:8088/taskdetails.jsp?jobid=job_1430542224940_0001&tipid=task_1430542224940_0001_m_000000 ----- Diagnostic Messages for this Task: Container launch failed for container_1430542224940_0001_01_000005 : org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The auxService:mapreduce_shuffle does not exist at sun...
Read more

Zeppelin and Anaconda

Set Anaconda As Default Python Interpreter In Zeppelin Click  anonymous  in top right corner. Click  Interpreter . Scroll down to the  python  interpreter. Click  Edit . Locate  zeppelin.python . Set value to  /home/hadoop/anaconda/bin/python Now find the  spark  interpreter. Locate  zeppelin.pyspark.python . Set value to  /home/hadoop/anaconda/bin/python https://dziganto.github.io/zeppelin/spark/zeppelinhub/emr/anaconda/tensorflow/shiro/s3/theano/bootstrap%20script/EMR-From-Scratch/ This works..tested.. conda install -c calex sklearn-pandas P.S:  zeppelin 0.7.3 don't support spark 2.3. spark 2.3 is supported by zeppelin 0.8 which will be released soon zeppelin 0.7.3 doesn't support python 3.6.. now testing with spark 2.1 and python 3.5...so far so good.. Check:::::: zeppelin home/conf should have zeppelin-env.sh. Here you can specify spark home and zeppelin port Starting Apach...
Read more

Logistic Regression using German Credit Data

Image
Logistic Regression using German Credit Data In [1]: import numpy as np In [2]: import pandas as pd In [3]: from sklearn.model_selection import train_test_split from sklearn.metrics import accuracy_score , classification_report The German Credit Data contains data on 20 variables and the classification whether an applicant is considered a Good or a Bad credit risk for 1000 loan applicants. In [9]: credit_dat = pd . read_csv ( "C:\Work\Datasets\germancreditdata.csv" ) In [10]: print ( credit_dat . head ()) Creditability Account Balance Duration of Credit (month) \ 0 1 1 18 1 1 1 9 2 1 2 12 3 1 ...
Read more