A Surprise with How '#!' Handles Its Program Argument in Practice

Seems like it only works in zsh, not bash or fish

  [tombert@puter:~/testscript]$ ./myscript.sh
    bash: ./myscript.sh: bash: bad interpreter: No such file or directory

  execve(pth, argv, newenvp);
  // [...]
  if ((eno = errno) == ENOEXEC || eno == ENOENT) {
              // [...]
              if (ct >= 2 && execvebuf[0] == '#' && execvebuf[1] == '!') {
                                // [...]
                                (pprog = pathprog(ptr2, NULL))) {

`#!/usr/bin/env bash` is the most portable form for executing it from $PATH

Anything other than ”#!/usr/bin/env bash” is doomed to fail at some time.

You can use `#!/usr/bin/env bash` on NixOS

  - "#!" line.
  - /bin/sh as the location of a POSIX compliant shell, or any shell.
  - /usr/bin/env as the location of an env program.
  - The -S option to env.

> apparently I can just have done `#!bash`

I think you're mixing two concepts: relative paths (which are allowed after #! but not very useful at all) and file lookup through $PATH (which is not done by the kernel, maybe it's some shell trickery).

Or, like, if you aren't reading and caring about what the interpreter is--as that's the only time this can burn you: it isn't doing a PATH lookup, so you can't walk into this one on accident--then it could literally be something like /bin/rm on some key file. This entire article is based on an assumption that this is somehow so obviously bad that there doesn't even need to be an explanation or defense of any kind of that idea.

There could be. Say you have '#!sh', someone could have put a broken 'sh' somewhere, that could be the first one the '#!' sees.

Worked for me, but the way you described it has issues:

1. You chmod my_script twice.

2. Did you chmod u+x /bin/my_lang too? Since you put it in /bin, are you sure the owner isn't root?, in which case your user wouldn't have execute permission. Try +x instead of u+x.

3. Do you have python in that path? Try `/usr/bin/env python` instead.

4. In case you expected otherwise, my_script wouldn't be passed through stdin. It's just provided as an argument to my_lang.

Wait... your interpreter reads from stdin. Shouldn't it read its first arg, instead?

tbh it is lame for any program reading a text file to not support BOM. It's just one if.

UTF-8 doesn't have a BOM. UTF-16 does. UTF-32 does. "UTF-8 with BOM" is not a standards-compliant text format, it's a proprietary binary format that happens to have a bunch of embedded UTF-8. Just because you can run `strings` on a file & get a bunch of text out doesn't mean it's a text file!

Remember how slow early machines were. An unnecessary additional 'exec' would have seemed very wasteful, especially as most of the initial uses would have been shell scripts, so you would be execing the shell twice.

In a call `execve("/my/script", ...)` of course the kernel has to figure out how to run it, there is no shell involved.

As for scripts vs elf executables, there's not much of a difference between the shebang line and PT_INTERP, just that parsing shebangs lines is simpler.

It's a vulnerability via pathing, not a worry that the shebang script could be malicious.

Someone may have dropped a malicious executable somewhere in the user's path that the shebang calls. The someone shouldn't be able to do that, but "shouldn't" isn't enough for security.

Or maybe the relatively pathed executable has unexpected interactions with the shebanged script, compared to what the script author expected.

Etc.

I'm on FF 145 and don't see this message.

See also the perlrun documentation[0].

This example works on many platforms that have a shell compatible with Bourne shell:

    #!/usr/bin/perl
    eval 'exec /usr/bin/perl -wS $0 ${1+"$@"}'
     if 0; # ^ Run only under a shell

The system ignores the first line and feeds the program to /bin/sh, which proceeds to try to execute the Perl program as a shell script. The shell executes the second line as a normal shell command, and thus starts up the Perl interpreter. On some systems $0 doesn't always contain the full pathname, so the "-S" tells Perl to search for the program if necessary. After Perl locates the program, it parses the lines and ignores them because the check 'if 0' is never true. If the program will be interpreted by csh, you will need to replace ${1+"$@"} with $*, even though that doesn't understand embedded spaces (and such) in the argument list. To start up sh rather than csh, some systems may have to replace the #! line with a line containing just a colon, which will be politely ignored by Perl. Other systems can't control that, and need a totally devious construct that will work under any of csh, sh, or Perl, such as the following:

    eval '(exit $?0)' && eval 'exec perl -wS $0 ${1+"$@"}'
    & eval 'exec /usr/bin/perl -wS $0 $argv:q'
        if 0; # ^ Run only under a shell

[0]: https://perldoc.perl.org/perlrun#-S

Never thought of it this way; isn’t it always safe to assume env is in PATH?

Maybe `#! env <shell>` could be considered a DSL for hashbangs. My reasoning is that `/usr/bin/env` is the thing that seems to be hard-coded to a system path, in most cases.

I've encountered systems that only have bash in /bin/bash, or in /usr/bin/bash, and it's a hell of a pain to have to fix every script when using different distros (I think it must've been an old Fedora and Ubuntu?).

Nowadays, most distros are moving towards having /bin be a symlink to /usr/bin, so it's mattering less and less, but I see no reason not to just do /usr/bin/env which is supposed to be on the same place on every distro.

NixOS only has /usr/bin/env and /bin/sh.

I use `#!/usr/bin/env bash` because the /bin/bash on macOS is usually stupidly ancient, and I install a recent version of bash to ~/.local/bin/bash

Is it bad? Well it's less secure. But if you're worried about `/usr/bin/env` calling a malicious program then you need to call out the path for every executable in the script and there's a hell of a lot more other things to worry about too.

It's the same for `#!/usr/bin/env python3` in python scripts. Python3 itself might be ancient at system install, but you might need to be using a venv. So /usr/bin/env python3 works correctly while /usr/bin/python3 works incorrectly.

So is it bad? No.

If you're okay with running the script only from the parent directory of the venv (I guess you have things set up so that's the project root), fine.

You never have to "worry about whether the environment was activated", unless your code depends on those environment variables (in which case your shebang trick won't help you). Just specify the path to the venv's Python executable.

You aren't really intended to put your own scripts in the venv's bin/ directly, although of course you can. An installer will create them for you, from the entry points defined in pyproject.toml. (This is one of the few useful things that an "editable install" can accomplish; I'm generally fairly negative on that model, however.)

If you have something installed in a venv and want to run it regardless of CWD, you can symlink the wrapper script from somewhere on your PATH. (That's what Pipx does.)

    #!/bin/bash
    cd $(dirname $0)
    venv/bin/python3 myscript.py

  project_root = os.path.dirname(os.path.abspath(__file__))
  sys.path.insert(0, project_root)

This is terrible. I hope you're not adding a whole venv to version control.

What if the user doesn't have a venv created? What if they created it in a different directory? What if they created a second venv and want to use that instead? What if the user uses `.venv` instead of `venv`?

`#!/usr/bin/env python3` solves most of that.