The described issues that prompted the switch don't really have much to do with Python. The issues map back to running on whatever installation of Python the customer might already have.
Lua, in and of itself, didn't solve the problems. Shipping their own, known, Lua interpreter + extensions solved them. (which they could have done with Python)
The reason why people don't want you to ship your version of Python is that it takes a lot of space. Lua makes that hurdle much easier.
For example my local Python venv is about 130 MB. It symlinks in another 52 MB of standard Python libraries. And I know it won't run without some unknown amount of additional dependencies in C-libraries. If I wanted to send that to someone I'm starting at 200 MB. After compression that might be 40 MB or so. You're going to have to work to strip that down.
By comparison their download including Lua, code, and libraries is under 6 MB.
I have a minimal running Python application that was started recently and has done nothing. It takes close to 90 MB of RAM just to start.
Theirs is generally under 10 MB while it is running.
So at every step Python requires 5-10 times as much data. If you're trying to get other people to put you in their containers, this can matter a lot.
PyInstaller will generate a ~minimal installation. I've written an app which has a full python runtime, and images, and dozens of python files and modules in 6MB.
Or it can not matter at all. To the kind of enterprise that is still running RHEL 6, maybe shipping a 200mb install on a CD will show them that you're a serious business.
There are use cases where 200mb of disk space matters, but for the overwhelming majority of applications, a 5% (say) increase in developer productivity would be well worth adding 200mb to the install.
The point of the Distelli Agent is to deploy software. Think of it as the "bootstrapper". Serious customers would rather use that 200MB for their own software, not for the "bootstrapper" :). Here at Distelli we optimize for the customer, not for our own internal developer productivity.
Good soundbite, but if using 200MB of extra space would let you lower your prices while offering the same service (by increasing developer productivity) then I bet many customers would take that.
And did you try spending a person-week or so of dev time trying to minimize the size of the Python version? That would be a much smaller one-off cost than the costs of migrating everything to Lua, and I could easily imagine you'd get it down by one or two orders of magnitude. (You could do the same for Lua too I'm sure, but if we're talking about say 5mb vs 200k then that's "who cares?" size for most customers).
Nowhere in this article does it really state that they're less productive in Lua, though. In fact, the main engineer on it is well versed in it. Your 5% may actually be negative in this case.
From where do you get this impression that Python is the only high productivity language in the world? There are plenty of languages that can compete with Python in terms of productivity and a lot of them offer better abstractions for it. Lua is known for being one of the best escape hatches for scripting.
If you're more productive in Lua than in Python then I support using Lua 100%. It's switching languages to gain a measly 200MB of disk space that I'm objecting to. For 99% of projects 200MB is just an irrelevant consideration compared to how much difference language productivity makes.
For general cases, I agree that 200 MBs isn't enough to consider very important nowadays, but I still don't get how we're automatically talking about language productivity being a trade-off. Is the assumption that Python is a more productive language regardless of what it stands against?
btilly was talking about the disk space as if it were important.
My opinion is that Python is more productive than Lua, but if you disagree by all means use Lua. But if you're talking about saving 200mb of disk space as though it makes the difference, then I think you're using the wrong criteria for choosing your language.
This is exactly right. From "Challenges with Python " in TFA:
> I ended up introducing incompatibilities with older versions of Python when I tried using a finally block with an exception block—that syntax was not available until version 2.6
Lua changes major parts of the language in minor revisions. 5.1, 5.2, and 5.3 are not compatible, and in much larger ways than Python 2.6 and 2.7. Lua 5.1 is currently the most recent version supported by LuaJIT which he specifically mentions. Lua 5.3 introduces integers!
> For example, I needed to add support for detecting the Ethernet addresses of the host computer, which can be done with the netifaces native library (or shell out to ifconfig and parse the results, which would not be compatible with Windows).
You're going to have exactly the same problem with Lua, except that that library won't exist so you'll have to write it.
Lua's great, but this isn't a case of Lua being better than Python for your use case. It's a case of you redoing your deployment in such a way that you happen to have more control.
Lua is famous for breaking compatibility between versions. And Lua users are quite aware of the situation. But outsiders tend to overblow the situation, making more noise than the people actually having to deal with it.
The Lua creators are very careful and deliberate in how they break compatibility.
One such thing they do is they avoid subtle changes that can lead to incorrect/different results, and prefer hard things that lead to obvious compile time errors.
But most of the time, the changes between minor revisions aren't a big deal. On the Lua side, authors have been pretty good about figuring out how to make their scripts run across all versions because the changes haven't been that big. On the C side, the API changes can be a little more dramatic, but authors have been good about handling the differences.
Most are stuck with Lua 5.1 anyway as it's the version supported by LuaJIT2. But 5.1 is very good. And the Lua community is very loosely connected and quite different due Lua's embedded nature.
I made much Lua code, and one thing that makes a major difference, is that Lua devs frequently discuss what they are planning to break.
For example I wrote many games in Lua 5.1, that actually work in Lua 5 if I wanted to, and also work in Lua 5.2, despite 5.2 not existing when I wrote them.
I don't checked yet the changes on 5.3, but I suspect my games might work on 5.3 without changes too.
You pretty much always ship Lua embedded in your application though, as it is just a C library. It is the expected use case, rather than using a system copy.
This article does not say Lua is "better" than Python, it simply describes why we choose Lua. Bundling Python and rebuilding for each platform was certainly an option. However, luvi has the advantage that it has many platform specific builds prebuilt and extending luvi with more CMake files + C libraries is easy.
Plus me (the author of the Lua agent) was already familiar with Lua, so the easiest way to get from A to B was for me to use a known-good tool.
Exactly. It wasn't about Lua being better than Python, just that "in this challenge, the solution was Lua". Another challenge will have another solution.
I've only ever done game development in Lua and have never considered it for anything else honestly. Very cool use case here.
I don't think they could have created a statically-linked Python binary that contained the entire Python installation and core libraries, and the entire application in less than 5 MB. The entire app in a self-contained executable. No futzing with PYTHONPATH, LD_LIBRARY_PATH, virtualenv, installation, conflicting with the system-installed Python, etc. That's what Lua let them do.
But don't take it from me. Here is Guido van Rossum last year:
> The final question was about what he hates in Python. "Anything to do with package distribution", he answered immediately. There are problems with version skew and dependencies that just make for an "endless mess". He dreads it when a colleague comes to him with a "simple Python question". Half the time it is some kind of import path problem and there is no easy solution to offer.
In cca 2005 timeframe we used to ship py2exe-d windows desktop application (using wxwidgets framework), with exactly zero issues. The target was a wide range of windows releases, from 95(!) to XP.
I haven't seen any of problems mentioned in the blog post.
This is really meant for monkmartinez, but you're right.
I'd like to add, that blog can be dismissed outright. It's also out of date in regards to PyInstaller. It does support Python3 now. 2.7;3.3-3.5. I've used PyInstaller for years and prefer it over the other packaging solutions, it's really good.
I wouldn't be digging up old blogs as a source for dirt and relying on any information within without verifying it was or still is true.
Exactly my thought, that whole operation seems like it could have been done in Python. This has nothing to do with the Lua being a 'special' or 'faster' language.
Lua, in and of itself, didn't solve the problems. Shipping their own, known, Lua interpreter + extensions solved them. (which they could have done with Python)