Proxy Issues



----Conda Activating Virtual Env-

<CondaPATH>\Scripts>activate py35

Crosscheck .condarc file is not there..or backit up..

Then in the env..set the proxy
(py35) <CondaPATH>>set http_proxy=http://userid:password@proxy.foo.com:8080

Note if password has # replace it with %23 . @ is replaced with %40


---To install a package using conda in Anaconda---------
(py35) <CondaPATH>>conda install -c conda-forge <pkgname>

(py35) <CondaPATH>>conda install -c conda-forge tweepy

---To install a package using pip in Anaconda---------

(py35) <CondaPATH> python -m pip install <pkgName>

(py35) <CondaPATH> python -m pip install vaderSentiment


---------------Common Proxy Errors---------------------------

CondaHTTPError: HTTP 000 CONNECTION FAILED for url <https://repo.anaconda.com/pkgs/free/noarch/repodata.json.bz2>       Elapsed: -                                                                                                                                                                                                                                      An HTTP error occurred when trying to retrieve this URL.                                                                HTTP errors are often intermittent, and a simple retry will get you on your way.                                                                                                                                                                If your current network has https://www.anaconda.com blocked, please file                                               a support request with your network engineering team.                                                                                                                                                                                           ProxyError(MaxRetryError("HTTPSConnectionPool(host='repo.anaconda.com', port=443): Max retries exceeded with url: /pkgs/free/noarch/repodata.json.bz2 (Caused by ProxyError('Cannot connect to proxy.', error('Tunnel connection failed: 403 Forbidden',)))",),)

----------
Ypu may check this if behind corporate network for ssl issue


Windows

 .condarc file at the C:\Users\username folder.

-----------  .condarc file------------
channels:
  - r
  - defaults

show_channel_urls: true
allow_other_channels: true

proxy_servers:
  http: http://INDIA\username:pass@proxy.foo.com:8080
  https: http://INDIA\username:pass@proxy.foo.com:8080

ssl_verify: false


-------------
Windows Command Prompt

set http_proxy=http://userid:Password@proxy.foo.com:8080



---------------Activating Conda Virtual Env in Windows..................

<Anaconda_DIR>\Scripts\activate <envname>

The virtual envs created in Anaconda are in <Anaconda_DIR>\envs  directory


-------------Always crosscheck if .condarc file has updated pwd .....................

------------Without .condarc use below for proxy-----------------------------

Crosscheck .condarc file is not there..or backit up..

Then in the env..set the proxy
(py35) <CondaPATH>>set http_proxy=http://userid:password@proxy.foo.com:8080

Note if password has # replace it with %23 . @ is replaced with %40


Comments

Post a Comment

Popular posts from this blog

ScoreCard Model using R

Example R Notebook for ScoreCard Model One often receives mailers on pre-approved loan based on creditworthiness. When one applies for credit, often it’s approved, sometimes it isn’t... Have you ever wondered how exactly the decision to approve a loan is done?  Was the bank able to clearly articulate the rationale of their decision to decline? To ensure fairness and transparency, there are compliance rules which mandates Banks to have such a mechanism. The Federal Trade Commission (FTC), the nation’s(US) consumer protection agency, enforces the Equal Credit Opportunity Act (ECOA), which prohibits credit discrimination on the basis of race, color, religion, national origin, sex, marital status,age, or because you get public assistance. In BFS, often underwriter needs to decide the creditworthiness of an applicant. Among other things, a scorecard is typically in place and long been used. Now with data deluge and better processing power many Analytical models are also used.
Read more

The auxService:mapreduce_shuffle does not exist

Error and Probable Fix: 1. Error : org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The auxService:mapreduce_shuffle does not exist Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1 2015-05-02 13:57:39,982 Stage-1 map = 0%,  reduce = 0% 2015-05-02 13:57:49,511 Stage-1 map = 100%,  reduce = 100% Ended Job = job_1430542224940_0001 with errors Error during job, obtaining debugging information... Examining task ID: task_1430542224940_0001_m_000000 (and more) from job job_1430542224940_0001 Task with the most failures(4): ----- Task ID:   task_1430542224940_0001_m_000000 URL:   http://0.0.0.0:8088/taskdetails.jsp?jobid=job_1430542224940_0001&tipid=task_1430542224940_0001_m_000000 ----- Diagnostic Messages for this Task: Container launch failed for container_1430542224940_0001_01_000005 : org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The auxService:mapreduce_shuffle does not exist at sun.reflect.NativeCons
Read more

Zeppelin and Anaconda

Set Anaconda As Default Python Interpreter In Zeppelin Click  anonymous  in top right corner. Click  Interpreter . Scroll down to the  python  interpreter. Click  Edit . Locate  zeppelin.python . Set value to  /home/hadoop/anaconda/bin/python Now find the  spark  interpreter. Locate  zeppelin.pyspark.python . Set value to  /home/hadoop/anaconda/bin/python https://dziganto.github.io/zeppelin/spark/zeppelinhub/emr/anaconda/tensorflow/shiro/s3/theano/bootstrap%20script/EMR-From-Scratch/ This works..tested.. conda install -c calex sklearn-pandas P.S:  zeppelin 0.7.3 don't support spark 2.3. spark 2.3 is supported by zeppelin 0.8 which will be released soon zeppelin 0.7.3 doesn't support python 3.6.. now testing with spark 2.1 and python 3.5...so far so good.. Check:::::: zeppelin home/conf should have zeppelin-env.sh. Here you can specify spark home and zeppelin port Starting Apache Zeppelin from the Command Line On all unix like platforms
Read more