function substitute(arg) {
if (arg instanceof ProcessOutput) {
return arg.stdout.replace(/\n$/, '')
}
return arg
}
// […]
let cmd = pieces[0], i = 0
for (; i < args.length; i++) cmd += substitute(args[i]) + pieces[i + 1]
for (++i; i < pieces.length; i++) cmd += pieces[i]
Sigh… yet another code injection vector. And to think the whole template literal syntax was specifically designed with this in mind. Have people not learned from 20 years of SQL injection?
This feels like a pretty dramatic response when this is already a problem with child_process.exec and most shell scripting languages in the first place.
I’m not sure what you’re expecting if you’re taking arbitrary IO output to build up a process with arguments separated by spaces. A lot of things had to go wrong before you get to this point.
If we’re being realistic here, this library’s likely intended use is for this is smallish scripts anyways… not large pieces of software that are creating commands on the fly to from arbitrary IO
Ordinary variable substitution in shells splits on spaces, which is still bad, but at least doesn’t immediately lead to arbitrary code execution. I’m expecting at the very least an equivalent of Python’s shlex.quote. This is supposed to be an improvement on the status quo, not a regression.
> If we’re being realistic here, this library’s likely intended use is for this is smallish scripts anyways… not large pieces of software that are creating commands on the fly to from arbitrary IO
If we're actually being realistic here, we know users will use this for whatever scenario, regardless of the author's intent.
You should use child_process.execFile or the execve equiv in your language. Shell has expansion issues but rigorous quoting helped by shellcheck make it safe.
And zx's `$` could make better use of tagged template literals.
Something like this, tho it isn't correct
function $(strings, ...args) {
const cmd = [];
for (const part of strings) {
cmd.push(...part.split(/\s+/));
if (args.length > 0) {
cmd.push(args.shift());
}
}
return child_process.execFile(cmd[0], cmd.slice(1));
}
But with child_process.exec you can at least pass values via env and have the shell script come from a file (which you can throw shellcheck at)
Also, spawning node from shell to spawn shell to spawn something like ls is madness. Node has fs.readdir already and there are util packs on NPM like fs-extra and friends.
Escaping in the $ interpolator is a planned feature. You definitely have a point, but the project was made public 4 days ago and deserves a little bit of slack don't you think?
Injection vulnerabilities are one of the most pervasive and well-known kinds of bugs, especially in shell scripting. This should have been considered on day 1 and implemented before anything else started depending on it being otherwise. Fixing bad designs after the fact is long and painful. Just look at PHP.
We are talking pre alpha stages of development here, nothing that depends on this can reasonably expect for the API to not break several times over in the near future. This software is still figuratively on it's day 1 of development. Weather you or someone else would have implemented this feature faster matters very little to me.
"are you bringing in user input from a web form into your shell scripts?"
This is shell scripting. We have literally decades of experience with these. We know for a positive fact that simple concatenation is dangerous. The contents of files, the names of files on the file system, and all the other things that shell scripts normally encounter are perfectly sufficient to wreck your day if you are too casual about them. Should you reply with something incredulous about this claim, be prepared for dozens of people to jump in with their war stories about how a single space in a filename in an unexpected place cost their business millions of dollars because of their shell script screwup. (Obviously, they are not that consequential on average, but the outliers are pretty rough in this case.)
Shell scripting is frankly dangerous enough just with what is on your system already; actually hooking up user input to it is borderline insane. Shell scripting would have to level up quite a bit to only be something to be concerned about when "user input" was being fed into it.
Learning to do half-decent shell scripting in most shells consists about half of learning the correct way to do things like pass through arguments from a parent to a child script, because for backwards compatibility reasons, all the easy ways are the wrong ways from the 1970s. It's nice when a more modern take on shell scripting is nicer than that.
I will also say when I'm evaluating libraries for things like shell scripting, I look for things like this, and it definitely doesn't score any points when I see stuff like this.
Just using `set -euo pipefail` will prevent many stupid things, but then again, conventional wisdom these days just seems to be to not use a shell if you can help it.
That's another example of what I mean, learning the magic invocations that amount to "Oh, please shell, act halfway like this is the 21st century, please?" It's like how I still have "#!/usr/bin/bash NL use strict; NL use warnings;" still burned into my fingers for Perl. (AIUI that's obsolete now but I never got to upgrade to the versions where that became obsolete, and now I'm just out of it.)
> We have literally decades of experience with these. We know for a positive fact that simple concatenation is dangerous.
Yes and no. I agree with the overall premise that the footguns are well documented, but at the same time, projects like this show that there are still large segments of developers who will gleefully shoot themselves in the foot because they never took the time to learn shell, or they just never had the opportunity to earn the battle scars.
The developers of a not-insignificant portion of IoT firmware absolutely are bringing user input in from web forms and chucking it into shell scripts. And unfortunately, it's that same class of developers that are disproportionately likely to pick up zx and run with it.
The point is, at this stage in the evolution of internet security, we pretty much know where the bugs come from. Injection attacks are still a huge practical problem. It would be nice if new scripting languages reduced that attack surface rather than increasing it.
In English, my daughter's given name has an optional apostrophe (Ma'ayan). I've seen systems that escape last-name apostrophes but not in the first name.
By not using eval (and sh -c, and everything equivalent) unless it’s absolutely unavoidable and always quoting variables. The $ construct acts basically like (the shell’s) eval. Proper escaping is an absolute must here.
I'm pretty sure that "eval is a bad idea" was mentioned in the "thought terminating cliches" topic on Ask HN a few days ago :)
In bash, `eval` is a footgun like any other. You can use it, but you just need to be aware of where your toes are when it shoots.
It's usually a "bad idea" in the sense that if you think you need to use it, you probably don't, and 90% of the time, there is an easier way to accomplish what you want to do. The next 5% of the time, using `eval` might be easier but will also create maintenance debt with its overgeneralization. And the final 5% of the time might be actual legit use cases for `eval`.
I just grepped my codebase for `eval` and I almost never use it. One example of the "overgeneralized" 5% might be when I realized I could use `eval` to set "variable variables" (i.e. the name of the variable is itself a variable, taken from a function argument). It was cool, but I ended up deleting it in favor of a more concrete solution.
Personally, if I'm hesitating to use `eval`, it's usually not for any security reasons. In general, my bash scripts only exist in dev machines and CI runners, and I don't copy them into the application containers that are exposed to a live runtime environment with untrusted users. So for CI/dev scripts, I can safely assume the code will only run in CI/dev, and therefore I can trust arbitrary user input (which I can of course still validate).
Thanks for the detailed response! I’ve been trying to level up my bash lately, and I definitely have seen a lot of ‘poor’ examples where they bailed out & use eval plus something else, for something native bash can in fact handle just fine, if you dig deep enough!
People always focus on the footguns that exist in Bash the language but ignore how much systems programming is abstracted away from you in the shell environment.
In Bash you can enter a Linux namespace with a single nsenter invocation. If you want to do the same in Python you have use ctypes and call libc.clone manually.
How is that a remotely real comparison? The ansible mv function deals with preserving SELinux context, which mv doesn't do, and it automatically deals with a whole lot of common error conditions, whereas mv just fails. If you just want to replicate mv, Python has shutil.move, one line of code. Ansible is trying to do a lot more.
- it's portable accross OSes (and keep flags if the OS supports it, deal with encoding, etc)
- it ensures selinux context is saved if there is such a thing
- it has proper and rich error communication with the calling code
- it's includes documentation and comments
- it outputs json par parsing and storage
No to say "mv" is not awesome, because it is. There is much more boiler plate in python, and is why I'll often do subprocess.check_call(['mv', 'src', 'dst']) if my script is linux only.
i think its not fair comparison.
mv command implementation in 'C' might have more lines of code. Maybe we should complain that there are no OOTB library functions in python to move the file.
I don't really care how many lines of code there are in an implementation. I care how many lines of code I actually have to write.
Python has shutil.move and os.rename but the Ansible example is to illustrate that there's a lot of code that needs to surround those calls to make them useful and they're not 1-1.
It also means that accidentally adding a space somewhere (“$HOME/go” -> “$HOME /go”) can have catastrophic effect. I wouldn’t dare write a single “rm” if I’m not 100% sure the argument is being quoted.
I wrote essentially the same thing (with proper interpolation, using zsh’s builtin quoting system) for [Python](https://github.com/NightMachinary/brish). I have been using it extensively for months, and I’m very happy with it.
Honestly the more appealing aspect than the "await $`cmd`" syntax is that it imports a bunch of handy libs by default.
For some reason there's just something that rubs me the wrong way about having a bunch of requires at the top of a bash script (or cluttering my package.json with handy tools I use once in a script). But you're gonna want things like (promisified) fs, os, chalk, readline, node-fetch, etc quite a lot of the time.
Definitely wish they'd included shelljs though; almost strange not to.
I hope they add (or already have) syntax highlighting as .mjs without the file extension (just the shebang) for GitHub, VSCode, etc.
Neat idea, but what's the point of async-await when you just put await before 100% of calls like in the first example? Now you've got more to type for no gain.
I don't get this popularity of async-await, especially in JS where I find its combination of syntax and absence of pre-run checks overly confusing and error-prone.
All IO in JS is async, you either have a Promise-based API (which can be sugared with await) or callbacks. The only exceptions are Node APIs which are explicitly synchronous in their names and CommonJS require. All of which are either discouraged or being generationally phased out (CJS->ESM).
That it extends to the $ function (anything preceding a backtick in JS is a “tagged literal” but also a function call) is just API consistency because it can’t know whether you’re calling exit versus something which actually performs IO. So it always returns a Promise.
> All IO in JS is async, you either have a Promise-based API (which can be sugared with await) or callbacks.
No it's not. There are all kind of *Sync functions even in nodejs and in browsers (there was sync variant of XHR), and JS engine generally doesn't care if you block in your functions or not. JS engines don't even have an event loop, that's just a construct within the app that uses the engine, like nodejs or whatever.
I'm really unclear on what repository you're commenting on. The first example shows an example of executing commands in parallel, and does 3 jobs with 1 await.
when I write scripts in js or python over bash, its to leverage their access to data structures, error handling, and libraries. Often I use TypeScript, because types are good and concurrent i/o is a first class feature.
This means that promise.all probably looks something more like
Id certainly encapsulte my calls, parsing, and validation differently in real code, but you get the gist.
In shell I can wrangle jq, date, xargs and get the same result; only this is waay easier to write, gives me validation and usable error messages, and can be altered much more easily than shell.
I write complex bash scripts somewhat often. They can be quicker to write, they are great for one-offs. But if I'm coming back to something non-trivial or if its getting deployed into production I want a language like js or python (or nearly any other language. C++, OCaml, Java all help more than they hinder).
If process in the middle exists with non zero exit code, your whole script won't exit with non zero exit code. You have to collect exit codes and check them.
The whole thing will exit if any of those fail because of the double ampersands. Semicolons are the ones that ignore exit codes.
Things like this are why I don't love bash. Whether a script will fail if a step fails is too important to be hidden behind a single character change in a place most people ignore.
I agree that for complex scripts bash isn't ideal. But my criticism was about unnecessarily verbose syntax. Bash is synchronous by default, this project is the exact opposite.
I totally get your point! As someone who is not coming from JS, I agree with you!
But you must understand that I/O in JS is async, it's how the language is build! People who are advanced in async programming find this very comfortable.
It depends on how sequential your script is. It's not very uncommon to start a background task, do something in the meantime and continue when the bg task has finished. These types of async control flows are very easy to model in JS.
And as far as I can see you don't have to await every single statement, because you can do multiple statements await $`echo 1; mkdir test; exit 0`
It's been my experience that doing it that way can fiddle with the `this` value of those B and C functions, although I do in general agree there's usually no need for the outer wrapping arrow function if the interior one doesn't care about `this`
I believe your .catch can similarly be `.catch(console.log)` for the same reason
i was being a bit facetious. "Real-life" code is usually more complex and nesting is needed when you have conditional logic (call doC or doD depending on the return value of the previous call etc..)
The complaint really is that if everything is sequential it is a fault in the language to make you explicitly say it every time, not that await is somehow worse than promises.
When I need to run something async in Bash I can just add a `&`. Done. The majority of my scripts are purely sequential, though, as the language lends itself well to sequential data processing/piping.
I'm not seeing a benefit by wrapping almost every single command in 'await $`...`'. I get why you'd want to wrap Bash in a different language, especially when handling numbers. But I'd rather use something like Python than this verbose trickery.
I agree. My scripts are also rigidly sequential and verbose which is easier for me and other devs after me to have a simple mental model of what is going on at a glance. Which I think saves time in the long run because any dev can easily understand and make intelligent additions to make it better. For complex scripts, Python is the goto and isn't terribly difficult to grasp either.
I didn't say it was, but in the example given basically everything was, hence the complaint the parent made was that in JS we are often writing a lot of await, or promises or what have you because we have a lot of things that need to be done sequentially in a particular part of the program, or, in a small script, where everything needs to be sequential.
It's not really a JS issue here, just the ZS project decided to make `$` function async (for some good reason I'd think) but they could have gone the other way and make it synchronous by default.
I don't believe this is a language specific choice, they could have done the same with C#
It seems to me async/await is a desperate attempt to try to make JavaScript less painful to deal with because of its asynchronous execution nature.
I am not a JS expert and have not had many opportunities to work with it extensively, but I confess I have always struggled with this when dealing with JS; I have often wondered if the (alleged?) performance gains from executing JavaScript like this outweighs [what I have always perceived as] the significant extra verbosity and complexity required to manage simply executing things in order.
I agree with this syntax, but it’s too late now (for JavaScript).
To clarify, JavaScript could have made await the default for async functions, so that if you called an asynchronous function without putting any keyword in front of it, execution would block until the operation completed. In that design, there would be some keyword you would use when you did not want to wait.
What they did: you have to opt-out of async execution by adding the await keyword.
What they could have done: you opt-in to async execution by adding a keyword.
They just chose the wrong default (IMO). It’s not a big deal — it’s easy enough to type “await” here and there. It’s bad, though, that you don’t really see the basic flow of control from examining the code making function calls. You also need to consult the function definitions to see which ones are async.
Another option would be to have no default, and instead require that async functions be called with an explicit indication of whether they are to execute sync or async. That too heavy-handed, IMO. Fine for a statically checked language but not a dynamic one.
I don't understand your example, if would only work if your doA,B,C functions were synchronous, which completely changes the paradigm and use case. It's simpler yes, but very different. (simple example: if that function were to be run when you click on a button, now you have a frozen UI.)
Yep. I am just wondering if the paradigm of JS being asynchronous is just not worth the added effort in code complexity and all the workarounds that have had to be added in over the years to try to make it manageable.
I personally find it a chore to work in JS compared to other languages that work the opposite way - everything is synchronous and things become a chore when you want to deal with async stuff. Again I have limited JS experience and have never enjoyed working with it (I just am not interested in front end stuff) so I'm sure it's stuff people get used to.
Could you clarify what the paradigm of JS being asynchronous means? Usually, async calls are actually “async” in that you’re doing a network request or something actually asynchronous. The reason they have to be async is that you cannot just block all execution or else other things can’t happen while the network request is happening. Which means your entire application grinds to a halt. Other languages solve this in a similar way IMO, with async/await and actual threads. JS doesn’t have threads, so you can’t fork a process to handle network stuff apart from UI. I personally think that async/await is a lot more straightforward to use than forming a process.
Additionally, async/await allows you to wait when you need to. You don’t have to await a promise — you can just call it, and then it will work in the background. You’ll just not be able to respond when it finishes work. (Because you aren’t “wait”ing for it.)
What are some other examples with easier async in other languages? I’m just confused what you’re trying to get at. Everything in JS is synchronous. You only have async when you have async. Which is a lot, because a lot of stuff in the web is async.
Yep, sorry & thanks for your polite post in response to my basically unhinged ramblings (late night after a tough week, always a bad time to try to do anything).
The poster jmull above basically wrote[1] what I was trying to say much more succinctly; this other post[2] is also more clear.
Really I was just being snarky about the idea of shell scripting where you have to type 'await' in front of every command you want to run in a three-line file, like in the examples presented for this tool.
I wasn't attempting to defend JavaScript design choices. I'm just stating that working with blocking vs non-blocking code is now MUCH easier (and less error prone) since we have async/await.
Callback-hell and Promise-hell were real issues that plagued any project of significant size.
Oh yeh sorry I didn't mean to give the impression I thought you were wrong. I 100% agree with you.
I remember the first time I experienced "callback hell" (2016, for me, but I'm sure it was a huge problem for others before then) when I was doing some JavaScript stuff implementing Keybase's library for GPG support - I learned they'd built a whole separate JavaScript thing called IcedCoffeeScript[1] specifically to add await/defer support to get rid of those huge callback pyramids.
If you want to have the same code structure as this, but without await, you’re gonna implement your own queuing system (every call adds an operation to a queue). Many old tools operated this way, before ES6.
The problem with that it doesn’t meld together with most third party libs, since they will have no knowledge of your queue and will just execute immediately or out of order.
IMO it was always a fundamental mistake to force the programmer to deal with the event loop by default. Run async in the background, but present as sync to the developer unless otherwise requested, would have been a much saner design. Unless you're developing a framework or library, odds are good your ratio of leveraging async versus contorting yourself to make it (from your script's perspective) go away will be 1:10 at best.
JS keeps coming up with new ways to make this less painful, but it's ridiculous every time because it's a fundamental problem with how Node presents itself. A comical sea of "await"s in front of damn near every call is the modern version of this, and is utterly typical in real JS codebases, but before it's been callback hell, or screwing around with promises all over your codebase when 90+% of the time you just wanted things to execute in order (from your perspective), and so on.
I think it's also made much worse by how JS doesn't care about whole classes of bugs. Forgot an `await` somewhere or called an async function assuming it's sync? Now you've got a bug and in some cases no idea where to look for it. TypeScript is also blind to it (although now that I think of it a linter might flag it?).
A really good point, and all the more reason to make developer-visible async behavior something the developer has to to ask for, even if the call is in fact async under the hood and might let, say, code handling another request run while it's waiting on I/O.
I think a pattern where there are one or two great places at the lowest level of a Node program for program flow to act async, and then a bunch of business logic where it rarely is (probably running "under" the part where async makes sense, if you take my meaning) is far more common than those where async-friendly flow is what you want for over 50% of calls. "Call this chunk of code async, but run everything in it exactly in order" is super-common, and the interface to achieve that is exactly backwards in Node.
I can't even count the number of times I've seen a unit test giving a false positive (or, perhaps more accurately, not even run) because the developer forgot to properly use async/await or use the done callback.
Linter flags when you put an await in front of a-non async function, but, alas, the opposite is not true (call of async without await). This has always been source of bugs in my code.
This embodies so much of what is wrong with some JavaScript programmers' (that I've known personally) mindset that I find it hard to distinguish from satire...
But you don't get to have that! You get the "insane" syntax oh bash, encapsulated in a ridiculously verbose and frankly quite unintuitive JavaScript contraption that seems to provide little, if any, benefit.
Also, Shell scripting is mostly about correctly and safely interacting with other CLI-based tools - which this new thing wrapping a shell (and badly at that, see other comments in this thread) won't help you get right, either.
There are some projects which I've migrated from bash to js, in the middle of the migration, we had a mix of both. This project would have been really helpful to make that migration quicker. Of course, it wasn't ideal, but there's always a journey before the destination.
i feel like you could avoid some of the kind of annoying syntax like array iteration and conditional syntax. would it be better to know that stuff 100%, and write it in pure bash? yeah. is it one of those todos that people like me never seem to get around to, and this is kind of a stop gap? maybe. it doesnt seem entirely useless
I pretty much got rid of my bash scripts and just replaced them using shelljs. Makes it a lot easier and quicker to maintain, it means any developer can jump in fix it and add to it.
I think use case of shelljs and this zx is more for developers who need to perform file system and subprocess operations but rather stay away from bash or other footgun-ridden shells and instead write things in safe code with access to richer data types and more libraries.
bashojs seem to start in the other end, add js statement-execution to shells and and yet another pipeline-language in the mix. More like AWK with js-syntax.
Looks good as a "jq" on steroids but i wouldn't compare it to shelljs.
I was hoping for an article on how to use Node.js for normal scripting, since it's already pretty close to what it's shown in this library. I've written two libraries to help with scripting in Node.js:
import { read, walk } from 'files';
// Find all of the readmes
const readmes = await walk('demo')
.filter(/\/readme\.md$/) // Works as expected!
.map(read);
console.log(readmes);
// ['# files', '# sub-dir', ...]
import cmd from 'atocha';
// Any basic command will work
console.log(await cmd('ls'));
// Using a better Promise interface, see the lib `swear`
console.log(await cmd('ls').split('\n'));
// Can pipe commands as normal
console.log(await cmd('sort record.txt | uniq'));
Both of them are wrapped with Swear, a "promise extension" (totally compatible with native promises!) so that's why the first example works. You can build operations on top of the return of the promise, so that these two are equivalent:
// Without `swear`:
const list = await walk('demo');
const readmes = list.filter(item => /\/readme\.md$/.test(file));
const content = await Promise.all(readmes.map(read);
// With `swear`:
constn content = await walk('demo').filter(/\/readme\.md$/).map(read);
`raw` would allow us to pass the input as is without escaping it using JS rules, but otherwise the code would remain the same, we still have to "stitch" both arrays together with the loops.
It's not the "if" in particular, it's more of a signal that I am moving from a simple set of commands to something more complex with logic that will need objects, arrays and possibly a package manager.
Well, the whole reason to choose bash in the first place is that it just works on every single machine I have access to.. Ruby and js aren't installed everywhere. As for python, I have to be willing deal with versioning hell and the whole virtualenv vs venv vs conda mess.
Maybe C could work, but I have to make sure it's statically linked.
Nashorn (Oracle's, now discontinued ECMAScript implementation for the JVM and Rhino's successor) can be used for Shell scripting, too, when started with the `-scripting` parameter. Here is a little overview[1]. I think, GraalVM's new ECMAScript implementation is compatible with both NodeJS and Nashorn, including the scripting features.
GraalVM's JS engine[1] is an implementation of ECMAScript 2021 and as such, it's a drop-in replacements for Node.js and npm.
It also supports some Nashorn features for interop with Java, but for many things to work, you need to enable it explicitly, especially the compatibility mode flag (`-Dpolyglot.js.nashorn-compat=true`).
I had very strong anti-javascript opinions for the longest of times, and still feel that writing any non-weekend-project in JS is a mistake.
That being said, I have grown very fond of writing "plumbing"-scripts in nodeJS.
The syntax is so much more sane than bash, the documentation of nodeJS is actually pretty good in my opinion, and once nodeJS is installed and available, its super easy to mash something together to get working. For small scripts, the uglier parts of JS do not really matter, and while nodeJS does have some APIs that feel a bit odd to use at times, it's at least well-documented with examples/guides all over the web.
Some folks use python for that exact purpose, but I always felt that getting something going takes me more effort than nodeJS (the fact that I'm just not fond of Python as a language might also have something to do with this).
All in all... for folks still using bash, I really recommend to give JS a try.
I prefer Python for this because more OSes ship with it, it has a fairly rich standard library, and SREs are more comfortable with it, so you're more likely to find others using it as glue.
Agree, but it would be nice to have access to the entire composer community without setting up a composer environment, so not to toot my own horn, but I wrote proof of concept script to download PHP composer dependencies by parsing comments from the PHP script itself.
> I prefer Python for this because more OSes ship with it, it has a fairly rich standard library,
I mean nodejs =/= javascript, the problem is nodejs and some of the people who designed it who basically made it so that it would rely heavily on NPM, a commercial solution, thanks to the virtually non existing standard library. Now Microsoft virtually owns Nodejs since it owns NPM. Nodejs creator himself regretted this decision, publicly.
For what I called "plumbing"-scripts above, I rarely ever need any dependencies via npm. What nodeJS delivers out of the box suffices completely.
Sure, doing things like http directly with node is a bit cumbersome and having a library that does some abstraction comes in handy if you're working on something that goes way beyond what you'd ever consider doing in bash. But especially for that very purpose of things where I used nodeJS instead of using bash, nodeJS always had everything I needed out-of-the-box, almost never required any dependencies.
That being said, if you re-use code, plan to share scripts etc. with other people and all these things - using something like typescript (and possibly some testing-libraries) comes in very handy for maintainability, and there you can't get around npm.
But for one-off-"plubming-scripts", nodeJS had all I ever needed.
TS is probably my favorite language out there, so yes, I definitely did :)
But just for "one-off"-scripts, JS usually does the job pretty well. For bigger scripts or scripts where I want to reuse/share parts, typescript is an option - but it comes with a few hurdles: Most of the scripts I'm talking about work with zero npm-dependencies, while typescript adds at least one.
Furthermore, I either have to take care of compiling it, or use ts-node - one more dependency; and in both cases, I can't avoid the usual es6 vs. commonJS shenanigans.
It's a bit of overhead to get going; always worth it if I spend more than a few hours on a script, but I can do without if it's just a few lines that I will hardly ever see again.
I think Deno may be a good choice in this use case: single binary that has a good standard-lib makes it seem like a good candidate for simple shell and pipeline scripting.
> Why is Javascript a « perfect choose » for shell scripts?
It isn't, that project looks like perl re-invented but worse, since it mixes Javascript via nodejs and shell script syntax.
Shell scripts already support loops, functions, conditions, variables, ... it makes more sense to write a nodejs script that works as a regular shell process and include it in a shell script than the other way around...
JS is the most used language for web development both client and server side.
It's easy to find fullstack repos with only JS code and a bunch of shell scripts to launch the tooling like webpack bundler and cypress e2e tester.
By wrapping the tooling also in JS it's possible to take advantage of shared configuration written in JS and write reusable functions to share across scripts.
Currently I have all my tooling scripts in bash (not yet in JS) and it's becoming hard to manage. Sharing functions and constant values across folders in shell bash script is very cumbersome.
Javascript is just the new utility knife that all devs have in their pocket now mostly because you don't need to master it to build something with it, but also because it's acceptable to write improper JS code.
PHP used to be the utility knife for a while for the same reasons. It was easy to learn, and run it. And it was ok to write bad PHP code. I remember most of the exploits and remote shells were written with PHP because it was the easiest language for hackers to learn.
Python is a much better utility knife in my opinion, because it's used in more domains than just web development, and comes bundled with most operating systems.
Would be good to have a function which outputs all stdout from a sub command in real time, rather than after it's finished.
I generally use "spawn" from child_process instead of "exec" (which this tool uses), which can pipe output to the terminal as it happens. Great for creating build scripts.
I should, but not sure if learning bash specifically is best for productivity. Especially if I don't use it daily, I will forget syntax etc, will still have issues understanding what is going on quickly, more likely to introduce bugs when trying to improve something etc.
The shell is a long-term investment, and decades of programmers are happy to quietly testify to its continuing dividends.
POSIX shells (and OS) are the common thread underlying all *nix-based software.
Bash prioritizes terseness and backwards compatibility; which is occasionally contradictory with UX, but are valid prioritizations nonetheless.
Also, learning something new gives you a more nuanced appreciation of what you already knew. It’s a mind-expanding experience, even if it’s occasionally frustrating.
TLDR. I’d advocate for it. It’s OK if things break or fall apart on the road, this is the price of knowledge acquisition.
Makes sense, but I'm still afraid if I don't use it daily I will tend to forget it. I have learned everything I know by doing it in practice, so personally unless I would take some huger type of side project to learn up bash I don't see how I could do it. And in the end if I'm working with other frontend devs, I'd also have to convince them to learn bash as opposed to just using JavaScript/TypeScript for scripting.
Do you have any suggestions on how one should go about learning it?
If I do some course right now, then in a year's time if I use it maybe on few occasions in a year, I won't remember it, and it will stop being practical again.
What is it good for? Why would you want to execute shell commands from JavaScript. If you already know the command, then why don't you just use the shell?
Or why don't you just use JavaScript libraries/frameworks/whatevers if you can't make a shell script work but you know JavaScript?
What's the point? Most people choose to suffer, even when you offer them something that requires learning the thing just once. No matter how nice the thing is.
Why would you want a shell script to be async? All this means is that you're going to type await 47 times. This isn't a network server, I'm not sure why this is the right tool for this job.
Edit: that parallel rsync example answers my question nicely. I should have read more carefully.
Curious. When will JS tooling ecosystem support executing typescript by stripping the type information? Are there any plans to add generic soft type annotations to the spec? It’d be great to write these in TS without changing the runtime tooling.
It’s a great tool. It wouldn’t work in this case would it? Also, IIRC it checks types, doesn’t just strip (sometimes I prefer to just run the thing and do my type checking at the IDE/build level). Also still leaves the multiple tools problem.
Well `tsc` is the program that "strips" types after checking them. To be clear what you're asking for is a JavaScript runtime that supports parsing types but ignoring any errors it finds? Is there any computer programing language that has this behavior?
You can use the --transpile-only option to skip type checking. I'm going start using that myself for scripts I know are already good! Just tried it and makes a good difference to startup time.
Better than bash but nothing beats Python for scripting IMO. I’m a mediocre programmer and search results are rich with Python examples for anything under the sun.
> Bash is great, but when it comes to writing scripts, people usually choose a more convenient programming language. JavaScript is a perfect choice
Awaiting citations on usually, convenient, and perfect.
> await $`cat package.json | grep name`
I mean, in the first line they're well into "I don't understand shells" territory with a superfluous cat pipe (you can just grep name package.json). This is seriously published under the google banner?
This isn't a browser-based script, so no real fear of conflict. Plus, it's basically past time to retire jQuery anyways, outside of maintaining legacy apps.
You can load JQuery into a different global variable if you want. There are cases where you might want to parse a DOM in the context of shell scripts, such as scrapping some web page. Although JQuery wouldn't be the tool I'd personally reach for in this case.
But then it happened. Don't hate. I was upvoter no. 2 and I just knew this thing was gonna be on PAGE1. I know it's an "non substantial comment", but still! :P ;) xx
Ah, who am I kidding.
https://github.com/google/zx/blob/5ba6b775c4c589ecf81a41dfc9...
Sigh… yet another code injection vector. And to think the whole template literal syntax was specifically designed with this in mind. Have people not learned from 20 years of SQL injection?