Using Different Versions of Python - Compatibility/Dependencies?

[WWGD]

I have done web scraping with BeautifulSoup in both Python 2 and Python 3. There isn't much difference between the two. The kind of code you will need to write for web scraping will be pretty much the same in both versions.

https://pypi.org/project/beautifulsoup4/

Do you use some special package for ML, like Matplotlib , Graphlab, etc?

 

[PeterDonis]

Do you use some special package for ML, like Matplotlib , Graphlab, etc?

I haven't had a need for ML in the web scraping I've done, so I don't have any real experience with these libraries.

 

[FactChecker]

It will depend on what particular task you are trying to do (if you're invoking OS commands and parsing the output, then Perl, which is much closer to shell in this respect, will probably be more concise, yes). It will also depend on your own personal preferences, as well as who else will have to deal with your code.

Exactly. But when the work is heavily loaded toward a type of work, personal preference is not so important as how well the language has been designed for those tasks. A programmer must be adaptable. I didn't reprogram the Python code due to personal preference. I hate reprogramming working code. I only did it after it became obvious that the results would be so much better and easier to maintain. I handed the code off to others who had no experience in either language (but they were smart programmers) and they had no trouble modifying it as they desired.

 

[scottdave]

My first exposure to it was version 3.5, I think. I have come across code examples which were written for 2.x One thing I found is that the print statements work different in 2. and 3.
In 2.x, it is a statement. You would type

print "Hello, World"

for example.
In 3.x it is a function. The argument of the function must be enclosed in parenthesis:

print("Hello, World")

 

[jedishrfu]

Here's a summary of the major differences between Python 2 and Python 3:

https://www.geeksforgeeks.org/important-differences-between-python-2-x-and-python-3-x-with-examples/
I ran into several while converting some C code to python2 and then discovering I had to do it for 3 instead. The / vs // operator split was the most troubling conversion as the code was indexing into a custom database via seeks and record sizes. They used / everywhere with the understanding that it returned an integer when the operand datatypes were integers but 3 changed that notion and "broke" the code. The fix was to change to // wherever the / was used with integer operands. Duh

 

[jedishrfu]

My first exposure to it was version 3.5, I think. I have come across code examples which were written for 2.x One thing I found is that the print statements work different in 2. and 3.
In 2.x, it is a statement. You would type

print "Hello, World"

for example.
In 3.x it is a function. The argument of the function must be enclosed in parenthesis:

print("Hello, World")

Yes, I think the designers of Python changed from supporting print as a feature of the language to making it a function call instead as is done in many other languages like C/C++. (BASIC had a print statement that was a feature of the language).

 

[WWGD]

If I saw a job that required version 2, that would tell me that they have a lot of legacy code that they need to maintain. Call me silly but I wouldn't want to be a janitor.

As for my IDE, I've been coding everything in Jupyter.

But there are some ML packs designed for 2.7 that are not yet available AFAIK, for 3.x . Specifically, Graphlab , with SFrames . I learned these in a class years back and prefer to build on what I know, so I use both Anaconda and Idle. Graphlab has a lot of nice stuff to it, easy to use.

 

[Anand Sivaram]

Hi All,
I am trying to learn versions 2,3 of Python ( long story). Just curious as to the "Dependency/Compatibility" issues between and within versions 2 and 3. Are packages/libraries in 3.x available for 3.x' where x'>x ? How about for version 2? Specifically, at this point, I am trying to do some Web Scraping. If I learn to use it with one version, how likely am I of being able to do the same or similar within an other

Many people have given the answer to use python3. My suggestion is also to start with python3 when someone starting fresh. I would not bother much about minor versions like 3.6 vs 3.7 etc.
Broadly I do the following
1) Use pip and virtualenv, create a local setup completely - does not touch the python given by the OS
2) I mainly use Debian GNU/Linux so that almost every package is available to install from Debian itself, just do apt-get install. You could get similar results with other distributions like Ubuntu, Mint, MX Linux, Fedora etc. too.

 

[WWGD]

Many people have given the answer to use python3. My suggestion is also to start with python3 when someone starting fresh. I would not bother much about minor versions like 3.6 vs 3.7 etc.
Broadly I do the following
1) Use pip and virtualenv, create a local setup completely - does not touch the python given by the OS
2) I mainly use Debian GNU/Linux so that almost every package is available to install from Debian itself, just do apt-get install. You could get similar results with other distributions like Ubuntu, Mint, MX Linux, Fedora etc. too.

I agree,but some of the package s for ML I have learnt, specifically Graphlab with SFrames , which are very effective are , last I checked, available only for 2.7 . If they became available for 3, I would fully drop 2.

 

[StoneTemplePython]

I agree,but some of the package s for ML I have learnt, specifically Graphlab with SFrames , which are very effective are , last I checked, available only for 2.7 . If they became available for 3, I would fully drop 2.

Dato/Grahlab was bought by Apple a few years ago. The fact that they still aren't on python 3.x is a bit ominous. Maybe it's time to get on SKLearn instead, and use Python 3.x.

I understand that there are good legacy packages. But they are basically legacy packages. If you're new-ish to the language you'd be wise to not entangle yourself in any legacy issues. I did a system cleanup recently and no longer have 2.x as part of my conda / ipython setup.

The decision is yours of course but it's also a bad idea for a Newbie (reference thread title) to override advice from more experienced hands.

There's probably something about Fleetwood Mac and how once youre in 2.x you can never break the chain.

 

[WWGD]

Unfortunately I am in a sort of no-man's-land of having spent enough on Graphlab, which allows me to do ML without much h effort. But don't worry, I do listen to those who have been at it longer than I .

 

[Dr Transport]

In a discussion of this very subject yesterday at work, two comments were made that kind of puts this question to rest.

Python 3.x (x < 6) were essentially no better than python 2.7 because they were fixing the errors introduced with the updates. So, 3.6 is the best version of python out there.

Support for python 2.7 and earlier will end at the end of 2019, so why would anyone want to write new python codes in a version that will not be supported any longer, or more succinctly, why would anyone write in python 2.7 in 2019 and beyond?

 

[FactChecker]

so why would anyone want to write new python codes in a version that will not be supported any longer, or more succinctly, why would anyone write in python 2.7 in 2019 and beyond?

If some essential tools are not compatible with the new version, then there is no alternative to programming in the older version. I have run into that in another language.

 

[l0st]

As for IDE, supposedly the best one is PyCharm (community is free). But most people nowadays probably use VS Code.

 

[Lish Lash]

Python 3's incompatibility with the extensive archives of Python 2 code floating around the web is a slow-motion disaster. Very little effort has been made to go back and tag open source code and docs with version numbers, and you can't trust anything to work out of the box anymore. As mentioned above, Perl, for all its crufty sins, still does the job year after year. Python is in such a sad state that AOSP compiles its Python 2.7 interpreter straight from source code to insure compatibility.

 

[PeterDonis]

Python 3.x (x < 6) were essentially no better than python 2.7 because they were fixing the errors introduced with the updates.

I think that's too extreme. I would say that Python 3.3 had reached the point where the errors introduced in the 2 to 3 change had been fixed and the functionality was at least equivalent to 2.7. By 3.6 some genuinely new functionality (such as the async and await keywords) had been added.

 

[Dr Transport]

I think that's too extreme. I would say that Python 3.3 had reached the point where the errors introduced in the 2 to 3 change had been fixed and the functionality was at least equivalent to 2.7. By 3.6 some genuinely new functionality (such as the async and await keywords) had been added.

Just quoting my co-worker who is working the issue at work currently. I know that there was a step-back when 3+ was rolled out, that was to be expected, there usually is a little set of issues.

 

[Paul Colby]

Choosing a language for a project is important and nontrivial. For web scraping (never heard of it so I watched half a youtube vid making me essentially an expert) one might want to look at the design patterns Python 3 supports which Python 2 doesn't support easily. Decorators come to mind. Some of the f-string stuff is nice. My 1.2 cents.

 

[flywire]

If you are learning then just learn Python 3. If you need to write Python 2 just write Python 3 and fix the errors as you find them in your tests. I used that approach years ago, I didn't know basic so I wrote fortran and it didn't take long to get productive.

 

[WWGD]

If you are learning then just learn Python 3. If you need to write Python 2 just write Python 3 and fix the errors as you find them in your tests. I used that approach years ago, I didn't know basic so I wrote fortran and it didn't take long to get productive.

But the problem is I learned just a bit of Python 2.7 with ML SFrames and this is not available in 3

 

[Paul Colby]

Perhaps it's worth asking what ML SFrame'esk lib is in python 3? Sometimes names change to protect the innocent. Other times newer options have sprung up. Data mining the web is a going concern. Seems odd no python 3 options are included.

P.S. are you using pip? Have you looked at virtual environments? Both are essential IMO for python development. (not that I've done much, but I know people who have)

 

[Paul Colby]

Might be useful.

https://realpython.com/python-virtual-environments-a-primer/

 

[WWGD]

Any suggestion on web scraping? I am kind of worried about associated legal issues. Should I stick to using APIs?

 

[Paul Colby]

I lied, I only watched 1/4 of the youtube video. I really have nothing to suggest.

 

[PeterDonis]

Any suggestion on web scraping? I am kind of worried about associated legal issues.

If you're just scraping sites that are publicly visible anyway, I don't see why there would be any legal issues; everything on the page is there for the public to see. The site might throttle you if it detects that you're requesting too much data in too short a time, but that's a technical issue, not a legal issue; you just need to scrape at a slow enough rate, the same way the web crawlers for Google and other search engines do.