I felt it was a grave mistake for Python to direct the community effort into backwards incomparable Python 3, rather than on focusing on the speed (and better profiling instrumentation) of Python 2 (perhaps adding a standard JIT compiler). As a result, had our company started now, we would have written our Twisted server in C or Go instead of Python like we did.
Looking back Python 3 came too late, didn't offer much in terms of an incentive to justify switching to it, divided the community and in the process also left other things people worry about by the wayside -- performance, packaging, better concurrency handling (no not via Tulip or Twisted).
I think, like you point out, many before they decided to sink the time into upgrading to Python 3 to get better unicode support for example, would also take the time to evaluate and go with something else altogether -- Go for example.
The point was that once people are looking at restructuring the code base and re-writing parts of it, it also becomes a point of evaluation other technologies and frameworks. So there is a high chance many will not switch Python 2 -> Python 3 but Python 2 -> Go or something else.
Then well ok, the answer to "What did Python 3 bring to the Python community?" becomes "Yeah it ended up driving a lot of people away", which I am sure wasn't the intended purpose of Python 3.0
The time cost of 2->3 will be less but if I know I will get better memory footprint, and faster response times I might be willing to invest some more time into it.
Or another way to look at it, once one sits down and rolls their sleeves looking to re-factor/renew/refresh the code base there is a good chance they will look around them and see what else is out there.
Or something else, yes. If the cost is potentially going to be big enough to convert for a large code base (keeping in mind testing might be a huge part of it - having to potentially re-test everything), and for very little benefit (well, unicode possibly, but it's not like Python 3's really any faster, and the GIL's still there), it makes you evaluate if it's worth enduring a bit more transition pain to convert to something else completely which would bring the possibility of new features / better speed.
It's also quite an issue for COTS software - sometimes to jump forwards and clean up APIs they have to do a huge refactor, sometime even a re-write of the API. If this is troublesome enough, it's worth taking the time to look into the competition.
Because Go is at least as fast as PyPy or Cython, have a similar coding feeling to Python, have language level concurrency support, and has a clear/consistent development roadmap moving forward.
PyPy is not really a good example here. It is still highly experimental, takes long time to compile, it's not compatible with quite a few libraries in normal Python, and still has all relative downsides of dynamic typing. To be frank, I don't get the point of using PyPy instead of Go.
Previously most Python libraries already use C to optimize critical components. If you can use just one language to get similar performance, why do you bother using two then?
The issue is not about writing clean new programs in Go, but rather re-writing whole code bases from Python into Go, instead of trying out Python implementations that offer better performance than CPython.
Clean and efficient concurrency primitives, greater memory efficiency, and the ability to finely control memory layout, combined with just enough "high level" in the language to write complex systems. For that, you give up some (but not too much) of the rapidity of dynamic typing, and you don't have the marvelous ecosystem of libraries Python has built up. (yet)
PyPy supports greenlets and a Stackless Python-style CSP with tasklets and channels. See http://doc.pypy.org/en/latest/stackless.html . And it has for years. The original Stackless author is also one of the PyPy developers.
Or do you mean some other style of concurrency primitives?
Cython is really impressive, but it produces incredibly huge C files, which then compile into enormous binaries. So it's not terribly suitable when code size matters.
Yeah, so refactored PHP makes a PHP app 20% faster.
But you might actually not like to hear Node's V8 engine is between 50 and 150 times faster than PHP in synthetic benchmarks, and about 20 times faster in more diversified applications (as long as they don't sit an eternity blocking waiting on a DB call which is the way people did web apps 10 years ago). That's 2000% faster than PHP, just so we use the same units here.
I'm very happy PHP is getting faster and all, but just don't juxtapose it next to JS engines (which the article talks about), let alone something more serious like the JVM, because the "bright future" of PHP starts looking quite dim in comparison.
It depends, for things like Web Sockets Node.js is really good. But for other things, HHVM (Facebook's HipHop JIT) and the new PHP 5.7 dev with JIT are really fast.
PyPy is good. I wish there was more moment and more _pull_ for it from the main Python team. I feel the big push for the last 3-4 years should have been for PyPy (or performance and tooling in general).
PyPy is good. I wish there was more moment and more _pull_ for it from the main Python team. I feel the big push for the last 3-4 years should have been for PyPy (or performance and tooling in general).
EDIT:
To expand, I remember feeling so excited during the PyCon-s in Chicago and then Atlanta. Unladen Swallow was making some progress but PyPy was fantastic, great speed improvements. Everyone looked in awe at you guys, and your presentations. We talked about STM and all the great things.
And then...Guido announced in a rather annoyed tone how PyPy will never be a part of main Python distro and that's that. "Stop dreaming folks". And that made me pretty disappointed.
Then the drumbeat for Python 3.0 started. Unicode, unicode, unicode... it got louder and louder.
The wall of shame for "incompatible" libraries that were not being ported to Python 3.0. Lots of how to migrate and to Python 3 talks. Lots of oh look at the great view iterators.
Things that would make me get off my seat and start porting to Python 3 would be: greenlet integration for low memory, concurrency green threads (gevent and eventlet). Great CPU speed improvement with included PyPy. Notice what is not on the list -- unicode, view iterators and renaming Queue module to queue. Sorry if I am being too sarcastic here, it is because I am bitter about it.
It is also important to realize that Python 2.0 is just very good that it becomes hard for Python 3.0 to beat it. There were not big hairy warts that just bothered me about it. So it isn't that Python 3.0 is bad, it is not, it is just not better enough Python 2.0.
I think you're a little hard on Guido. CPython had amassed quite a few C extensions by that point. PyPy changes the low-level API in numerous significant ways. If Guido were to merge CPython with PyPy it would break all of those extensions. These extensions are in no small part the reason why Python is so popular. If he were to break all of this work, he'd piss off quite a few people. Many of the same people that helped him make Python what it is today.
Now I agree with you having PyPy's performance improvements in the main implementation would be awesome, and PyPy itself is quite impressive. If they could get both NumPy running on PyPy and support Python 3's changes, it would keep Python a strong competitor for the foreseeable future.
And to your last point, I entirely agree Python 2.0 had so many strengths that it's really hard to improve upon it. Though I think that it's unicode support was the big hairy wart.
Perhaps you are right. My bitterness was coming through. It no t just that he said but how he said it.
In a way Python 3.0 also broke compatibility, for what I think, are not very good reasons. Some of that code was from people/companies who have adopted and used Python and made it popular. They are now living on a dead-end maintenance only branch.
Read it, still not convinced. Have you read what I wrote though? I think I was pretty clear that I knew about changes and knew about other things (iterator views) etc.
None of those things make me want to get up roll up my sleeves and say "I can't wait to dig in and make my code compatible with 3.4". If I woke up tomorrow and someone would have magically done that for me and tested it, yeah great. I would buy them a beer. But that is about it. I have 0 incentive today to upgrade.
Code works very well now just using unicode support from Python 2 and print as a statement + other warts.
Yes we have: May 2 16:58:35 db2 kernel: [11106.666665] pypy[5672]: segfault at 18 ip 00000000010d9f39 sp 00007fff73204fa8 error 4 in pypy[400000+11dd000]
So for how long will the python community pretend that performance isn't a problem? (Inb4 "all the intensive parts of my app are written in C", addendum: "my app isn't CPU bound", "we have libraries written in C")
The interpreters (esp CPython) need some hardcore engineering (like how javascript got V8). Languages that have the same theoretical performance limits of Python are now blowing it out of the water.
The stories about teams that were able to go down from N servers to 1 server by switching from Python to Go/Scala/node.js will hurt it in the long run.
> So for how long will the python community pretend that performance isn't a problem?
Which python community are you talking about? Cython, numba, numpy, pypy?
> The stories about teams that were able to go down from N servers to 1 server by switching from Python to Go/Scala/node.js will hurt it in the long run.
Have you followed anything that is going on or just link-bait articles? Nearly everyone that re-implements something in another language uses different paradigms during the re-implementation, because they've learned something from the original implementation. Besides, those articles normally don't discuss any of the opportunity cost of re-invention or the long-term maintenance costs. I'm not saying they can't improve those things, but until you have the TCO over a few years, it's not an accurate portrayal.
The large amount of excellent work being done on PyPy would seem to indicate that the Python community does not, in fact, pretend that performance isn't a problem.
I don't think it is denial per se, but rather mostly inertia. Python has been a victim of CPython's success, in the sense that there is now a huge body of work that is tied to the CPython API. This is a problem for IronPython and Jython, and continues to be a problem for PyPy (though they are making strides to solve it), and it will be a problem for newer projects like Pyston. Numba has sort of skirted the issue by integrating with CPython, but the jury is still out there.
>>The stories about teams that were able to go down from N servers to 1 server by switching from Python to Go/Scala/node.js will hurt it in the long run.
concurrency designs and architecture play a greater role than the language , most of the times.
Bottle-necks are often in bad-sync patterns , data-structures or overall designs.
Language plays the role for semantic verification,formal models (for model-comparison...) ,type-system designs , proof systems etc .GIL is just one!
Can't compare language vs another.
After which the buck passes on to underlying implementation but cpython in our discussion is only thrown as a reference implementation and for production grade interpreters we might have to look outside like one by enthought.
However language's built-in patterns to help advertise concurrency is another question altogether.
Even scalable web-whatever is often IO-bound... of my several apps in production, there's only one where CPU is a serious concern, and I run that one on PyPy. cPython is fine for the others.
I'll go so far as to say that we haven't done enough with making things fast on multicore processors except with IO-bound tasks. The current architecture of multicore processors stinks for achieving anything but the "embarrassingly parallel."
In the browser environment, people usually use quite a few inline anonymous functions. Javascript is great at this since you can define full-blown anonymous functions anywhere.
Python, on the other hand, is definitely one of the worst at this. You can only create one-line, single-expression anonymous function with its lambda keyword. Of all the languages you want to convert to JS so that it runs on the browser, Python is probably the last.
I really don't understand this obsession with anonymous functions. If you're doing anything longer than one line, there is absolutely no reason why you should be sticking the entire function definition as a parameter.
I've had great usage from Javascript where I have separate functions with proper names and formatting, and then pass a reference to them into the code that asks for them.
Can someone please enlighten me about this whole anonymous functions thing that Javascript seems to have popularized? What are some of the pro's of using it over regular named/defined functions that sit by themselves and are reusable.
I suggest you read SICP and gain an appreciation for higher-order functions. Once you do, you will probably have answered your own question.
http://mitpress.mit.edu/sicp/
In python, it would be proper to say that ALL functions are lambdas except that some/most of those lambdas are assigned to variables via syntactic sugar (this is made very plain in sicp which is the main reason for suggesting it).The only thing different about python vs most other languages is guido refusing to implement multi-line lambdas.
As to any complaint, lambdas exist in python (that is, are made accessible to the programmer) for the exact reason python also has a for loop in addition to a while loop -- because they make things easier and more maintainable.
"In python, it would be proper to say that ALL functions are lambdas except that some/most of those lambdas are assigned to variables via syntactic sugar"
If a function is "assigned to a variable" then it's no longer a lambda, because it's bound to an identifier. But I suppose the lines are a tad blurred.
While I agree that the function "has a name", the function is still anonymous in the sense that pointers don't have types, only values have types. As a side effect of converting church's model to Turing's model, all "unnamed" lambda functions are given one-time symbols as are all anonymous numbers, arrays, classes, objects, etc.
x = lambda x: x+x #a lambda, but has a named pointer
x = 5
lambda is still in memory (until garbage collection), but it no longer has a pointer to it.
def x(y):
return y+y
x = 5 #the "named" function above is now without a pointer
A little off-topic (but answers your previous question and perhaps looks a little deeper into lambdas), but let's take a quick look at javascript. JS developers use lambda functions both because lexical closures don't pollute the global namespace, they offer modules and alternatives to prototypal objects, and because the language tends to make heavy use of callbacks (which demonstrate the bad side of lambda functions). JS also only makes subtle differences between lambdas and named functions.
//lambda expression
var x = function (){};
//named function
function x() {}
//named lambda expression
var x = function x() {};
//self-executing lambda (returns 5)
(function (x) {return x;}(5));
note that this would be a self-executing lambda in python
(lambda x: x)(5)
basic module pattern using self-executing lambda
my_module = (function (self) {
"use strict"
self = self || {};
//private
//pretty much inaccessible to the outside scope
var privateVar = 5;
//lambda expression again
var privateFunc = function () {};
//public
self.publicVar = 5;
self.publicFunc = function () {};
self.getPrivateVar = function () {
return privateVar;
};
return self;
}(my_module));
my_module.publicVar = 8;
my_module.publicFunc();
my_module.getPrivateVar();
This is where too many lambdas becomes a problem. Judicious use of naming (even though they're only used once) makes things more legible.
Why would zo1 want to read SICP? zo1 is clearly quite happy with Python's named functions, and has no immediate incentive to "read SICP and gain an appreciation for higher-order functions."
I either assume that the parent poster was serious about the question or rhetorical. Given the apparent lack of knowledge, I assume the question was serious. If the question was serious, does it not deserve a serious response?
Faced with squeezing an explanation into a limited post or recommending one of the best books on computer science (written by teachers with many years experience in teaching these self-same principles), should I not recommend the book that will allow the poster to expand his/her horizons?
The poster was sufficiently motivated to ask the question. I was willing to post what I believed at the time to be the best answer to the question. If the poster's desire to know is sufficient, then he/she will take the time to explore the suggestion. If not, then things will stay as they are. In either case, my only interaction is my choice to share a suggestion or remain silent.
Indeed, you are right. I was quite curious about them, and I wanted to know if there was something I was missing. Some reasoning that I knew not of, and that others did.
However, I'm about halfway through the chapter on "Higher order procedures" in the book you linked to, and have skimmed the rest of that chapter. And I've yet to find a valid pro for anonymous methods. Nothing I've found so far as to the benefit of using anonymous functions.
It does however, continuously reiterate the benefit of being able to pass "procedures as arguments" and to be able to return them from normal functions. So far, that's been the only "pro" and it's one that's solved by function pointers or having functions as objects, and not by anonymous methods. I'm still open for convincing, genuinely.
Here's another jewel from that chapter: "The resulting procedure is just as much a procedure as one that is created using define. The only difference is that it has not been associated with any name in the environment."
On a side note, your initial post was a tad condescending. The other post at least gave me the relevant chapter, instead of saying "here, read this for yourself". The least you could have done was give me a few short points why you think anonymous methods are superior in some way. Perhaps pointing me to a large blob of text was your way of deferring your own justifications for their use by not having to explain it to me (and as a consequence, yourself)?
My apologies. I was somewhat sleep deprived at the time and it sounded a lot better in my head.
SICP's point with higher-order functions is that code is data and that functions are no different than numbers or lists or whatever. In the function below, I've named everything.
def add4Mul2(x):
z = 4
y = 2
return (x + y) * z
lst = [1,2,3,4]
x = map(add4Mul2, lst)
the short version without any variable names.
x = map(lambda x: (x+2)*4, [1,2,3,4])
I don't name the function for the same reason that I don't name the 2, the 4 or the array -- I don't need to because I use them one time. If I were using (x+2)*4 all the time, then I'd give it a name, but since I only use it once, it's less of a problem to treat the function like any other constant data -- use the data without assigning it to a variable first.
What if LLVM based JITs with NaCL style security were available in all browsers except IE? (I'm just assuming Microsoft will drag their heels.) Then an LLVM IR based bytecode could become a standard bytecode for browsers.
I believe that in many ways, this would be ideal. Not only could any language be used (the implementers being responsible for interfacing with the browser API), but the LLVM pseudo-asm could be pre-optimized saving the browser a lot of work.
In the meantime, browsers would still be free to compete based on their JIT implementations.
With rust and by proxy, Mozilla/Samsung's experimental Servo layout engine targeting the LLVM, I can also see potential synergy in this situation.
Still trying to find a suitable "python to js compiler written in js that runs in the browser" project.PyPy.js is unfortunatly too big and all the other projects i found are either written in python or dont pass the browserify step.I need a transpiler that runs in the browser.
Python in the browser so far is a big disappointment. I still find it a lot easier to just split frontend and backend development. In the future, PyPy.js might become a viable alternative for big browser applications.
Am I the only one that considers the title of this very "bait"-y? I mean, if you go on and read the article, it clearly shows that native PyPy is leaps and bounds faster than the JS version. So in essence, it re-iterates what most of us already know... That the PyPy implementation is much faster than the standard CPython one. It's just now we get to "prove" it in JS as well, as if it's some "suck it, python users" flaunt.
On a side note: To the PyPy.js authors/devs, great job! Looking forward to playing around with this.
According to the author, it is deliberately bait-y:
OK OK, I couldn't resist that title but it probably goes a bit far. Let me try for
a little more nuance:
PyPy.js: Now faster than CPython, on a single carefully-tuned benchmark,
after JIT warmup.
Yes, it is baity and the article clearly admits to it. But it also clearly has PyPy and CPython in the title which is important. I don't read as "suck it python users" but as "great news, python users: if your code runs okay in CPython today, you could be running it in the browser with decent performance soon!". If anything it's a "suck it JS, we can now run real languages in the browser!"