If you are at this level of required complexity as in those examples, you should use a proper programming language, not shell. Half those snippets fail with spaces in the wrong place, newlines, control characters, etc.
I think all such shell "magic" should come with a huge disclaimer pointing the user at better alternatives such as perl or python. And all snippets should have the necessary caveats on horribly broken and dangerous stuff such as "if this variable contains anything other than letters, your 'eval' will summon demons" in eval "hello_$var=value" and stuff...
I enjoy using bash, and throw together little scripts now and then. It is convenient to be able to wrap up some bash commands and turn them into a script, when I realize I’ve been using them repeatedly.
But, every time I see examples of how to write sh scripts properly, it makes me wonder if this is just the wrong way to look at the world. Maybe it would be easier to extend Python down to make it better for command line use. Xonsh or something.
Ansible is like Python but for scripting (orchestration). They used a YAML format and with all the curly braces and quoting, made it just as bad as shell.
Easier, lightweight syntax for shell-like pipes, command execution and catching stdin/stdout/stderr.
Something like Perl's IPC::Run.
Also, more shell-relevant stuff in the default distribution, so that one doesn't need to care about any modules (which is the primary reason for using bash or even sh, those are installed practically everywhere along with at least coreutils and stuff). Edit: examples that a standard python doesn't really do would be quick and easy recursive directory traversal and doing stuff to files found there (like the unix 'find' tool), archive (de)compression, file attribute operations (not only simple permissions but also ACLs, xattrs, mknod, etc).
But the sister comment clarified it in another way, so this maybe irrelevant.
Sorry, I was sloppy. I meant using it as the system shell. So, processing arguments, I guess, would be less of a big deal.
Convenience features, like ls and other classic shell commands being run without parentheses would have to be handled… I’m not breaking any new ground here, actually this has gotten me to look into xonsh and it looks pretty decent.
Yes, but actually no: I was tempted to include it but didn't. The one big argument for bash and sh is ubiquity and compatibility. Perl also has those. Python is somewhat lacking in those. Ruby is very lacking on both.
Using a more advanced language just because you find shell syntax to be wacky is like using a car to get groceries because you find panniers or a backpack to be wacky. It's the use case that matters; if you're 60 meters from the store, just use your bike, or walk.
There are plenty of cases in which Perl or Python will make things much more complicated than 5 lines of spooky-looking shell script. Sometimes a little mystical sorcery is what the doctor ordered.
Shell is full of ridiculous footguns though. It's like saying if the store is only 60m away (across an Indiana Jones-tier trap gauntlet) then just walk there.
Remember that time bumblebee accidentally deleted everyone's /usr? Or that time steam deleted everyone's homedir? Both because of the easiest to avoid bash footguns - spaces and empty variables.
My hard and fast rule that I've never regretted is - if you use a bash if statement, consider python. If you're going to do a for loop, you must instead use python.
Typically as a side effect once the programmer is in python at my prompting, they find having such easy access to libraries suddenly lets them make the whole script more robust - proper argparsing is an immediate one that comes to mind.
Frequently the reticence I see from people, especially juniors, is that they're worried about people thinking "haha they have to pull an entire python into their image just to run this script" or "wow they're so newbie/young that they can't even write some shell". I reassure them: don't worry, there's a reason we used Perl even back then too.
Personally I've found Python to have significantly fewer foot-guns than bash.
The biggest reason why I don't use it all of the time is that calling / piping commands takes a lot more typing, so it's easier to use bash for very simple shell scripts. And while there are libraries that simplify shell scripting, that adds external dependencies to your scripts.
> Remember that time bumblebee accidentally deleted everyone's /usr? Or that time steam deleted everyone's homedir? Both because of the easiest to avoid bash footguns - spaces and empty variables.
I have written my large share of bash scripts at this point in my life. However, I recently started a new project. I opened up a file and started noodling out a sh script. I stopped exactly because of what you are saying. I then installed powershell. I have not decided if I am going to use powershell, python or ansible yet for this. But as it is gluing a bunch of commands together with some string manipulation and some very simple math calculations powershell seems better for the job in this case.
Bash is also good at these things but it feels like you are using weird archaic tools to get things done. They work very well but the syntax on some of the commands you end up having to use are entire oriley books by themselves. It has its own odd way of doing things. Bash is nice for when you know you can not really manipulate the environment. As it is fully complete and usually 'table stakes' for what is installed. It is just kind of odd the way it works. In this case I decided to start with something that is more akin to what I am used to writing.
Bash may or may not. It depends on how your script was called. If your script is sourced you need to remember to set and restore the flags.
In bash your data can accidentally become code. "rm $fn" usually deletes one file, but it might one day delete a few (spaces), or wildcard expansion makes it delete many. With Python, calling the function to delete one file will always delete one file. Your function will never run with a "continue on errors" mode.
oh come on, whatever is feeding files to the function I'll just trick into using some other data with different files. you don't need to "execute data" to have substitution bugs.
and it's easy to add status checks to your shell script just like you can for Python. exceptions are not the only way to stop on error. but it's sure a hell of a lot easier to have a non-working program in Python, whereas it's a lot easier for a shell script to keep working.
I think all such shell "magic" should come with a huge disclaimer pointing the user at better alternatives such as perl or python. And all snippets should have the necessary caveats on horribly broken and dangerous stuff such as "if this variable contains anything other than letters, your 'eval' will summon demons" in eval "hello_$var=value" and stuff...