Linux

There's always nushell. It's fairly new, not quite to 1.0 yet (0.96.1 at time of writing), but the constant breaking changes seemed to have stopped. It hits all your points and it's quite fun to use when writing scripts. Bonus that it's also pretty much tailor-made to manipulate data.

Elixir checks most of those boxes. If you want a good functional scriptibg language, Elixir soynds like the go to. Some lisp language like guile should also be sufficient, and probably have a lighter footprint.

This requirement stands out though:

has a simple/straightforward setup (ideally, it should be a single executable that I can just copy to a remote system, use to run a script and then delete)

Thats basically what ansible does. If you plan on doing this to multiple machines you should just use ansible. Also how do you plan on ensuring the scripting interpreter is installed on the machines?

Tried bash, Make, and awk/sed. All hit brick walls. Finally landed on pyinvoke. Two dependencies to install on any new machine. Never had problems. Also, easy to debug and modify as projects evolve.

Quickly came to write "AWK!!!!!!!!!" but yeah... you don't want its superiority... 😜

You should probably check out Guile.

posix sh + awk for manipulating data?

Perl would be my candidate for more advanced text handling than what sh can do.

Never used Lua but I think it's fun.

If nothing else works, just learn C/Rust. There's plenty of that on Linux systems, I think you'll be able to manage. Yes, it doesn't meet a lot of your requirements.

Why not give (Common)LISP a try?

https://nim-lang.org/

It is possible to wrap something like python into a single file, which is extracted (using standard shell tools) into a tmpdir at runtime.

You might also consider languages that can compile to static binaries - something like nim (python like syntax), although you could also make use of nimscript. Imagine nimscript as your own extensible interpreter.

Similarly, golang has some extensible scripting languages like https://github.com/traefik/yaegi - go has the advantage of easy cross compiling if you need to support different machine architectures.

Bro seriously just slap pyenv + pyenv-virtualenv on your systems and you’re good to go. They’re absolutely trivial to install. Iirc the latter is not a thing in windows, but if you’re stuck on windows for some reason and doing any serious scripting, you should be using WSL anyways.

Could use a hipster shell like fish, nushell or elvish. I know the latter two have the functional support you're looking for.

Why does it need to be a scripting (by this I assume interpreted) language? For your requirements - particularly lightweight distribution - a precompiled binary seems more appropriate. Maybe look into Go, which is a pretty simple language that can be easily compiled to native binaries.

You could use Ansible for automation just keep in mind it needs python.

@gomp Small footprint? Why not try forth. https://forth-standard.org/

Not sure how big node footprint is but would fit the bill. Would only recommend if you wanna go into web dev career in the future tho 🙃