Simple scripting will solve some very inefficient patterns, such as retrieving a long list of values over the network, performing some operation on them in the client, and storing them back to Redis.
To completely close the gap, I'd want two more things:
1. Store multi-line scripts as Redis objects. Trying to cram a loop and several expressions into one line will not be easy to understand or maintain. I'd rather deal with the cluster consistency issues in the application than be limited to one-liners.
2. Ability to execute a script in a separate thread. If the script doesn't require isolation, execution shouldn't necessarily block other operations. Some scripts might take 250ms to run, which is too long to block the main thread.
Where does it say scripts need to be single lines? The Redis protocol prepends all arguments with a description of their byte-length, so the scripts should be able to have as many newlines as you like.
Definitely agree about being able to use separate threads, though.
Indeed the script can be multiple lines without problems.
About threads, I think we'll stay single threaded for scripting as well since otherwise we have troubles, that is, Lua scripts will not be atomic from the point of view of the caller. There is to take care and not write commands that do complex stuff, or to be aware that this commands do complex stuff :) In Redis there is the tendency of doing a simple raw idea and try to make the user aware that it is easy to shot yourself on your foots instead of making it more complex to avoid problems, and it is probably a good idea to follow this path for scripting as well. After all there is always time to make it more complex.
If you stay single threaded, I would strongly recommend having a yield command, which would just exit the script, do other things, and then come back and execute the script again with state intact.
That could be implemented fairly easily using coroutines. Redis could start the script in a coroutine, then if it yields, return whatever values it yielded back to the user and schedule the coroutine to be finished later. When Redis has nothing to do, it could go back and restart the coroutine, then discard it once it finally returns. (Though of course the user would have no way of getting whatever values it yielded after the command returned - it would have to communicate by storing a key somewhere, or possibly with publish/subscribe.)
On second thought, doing this automatically might be a waste of time for most one-shot commands. Perhaps a separate COEVAL command would work better for this.
That sounds right to me. Only a few commands would need this facility. But when you need it, you need it. A long-running script should NOT lock up Redis indefinitely.
Atomic is a good default, but it's not hard to imagine a case where a non-atomic script would be preferable to sending multiple scripts to avoid overly long execution time.
But as you point out, that can always be added later.
To completely close the gap, I'd want two more things:
1. Store multi-line scripts as Redis objects. Trying to cram a loop and several expressions into one line will not be easy to understand or maintain. I'd rather deal with the cluster consistency issues in the application than be limited to one-liners.
2. Ability to execute a script in a separate thread. If the script doesn't require isolation, execution shouldn't necessarily block other operations. Some scripts might take 250ms to run, which is too long to block the main thread.