Add ability to source files from the command line before entering interactive shell

[gerph]

charles@work ~> fish --version ; echo $version
fish, version 2.6.0
2.6.0
charles@work ~> uname -a ; echo $TERM
Linux work 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
screen-256color

Under bash it is possible to introduce additional variables and environment when entering a shell directly, before the shell is interactive. That is, a script executes something along the lines of (from one of the scripts I use):

bash -i --rcfile <(echo "source ~/.bashrc ; source '${_PYTHONLIB_ENVDIR}/bin/activate'")

(or if you wanted to do it with the login shells, through the profile command line options, and then emulating the bash startup sequence in the new profile file)

Why might you do this ? I use it to set up the environment for working, from a script, which ensures that everything is setup and then drops to the shell in the new environment, with things set up like the user had before.

Anyhow. This is all fine and dandy for bash. For fish however, there's no easy way (as far as I can see) to introduce additional environment when starting the shell interactively. In the case of the virtualenv example, above, this has to happen after the user's conf.d has executed, in order that the prompt setup can be modified to reflect the new environment.

The solution I came up with was... let's say it's ugly and leave it at that (this is a fragment of a bash script that sets things up to honour the user's shell):

        if [[ "$SHELL" =~ /fish ]] ; then
            # Fish does not appear to be able to be given a startup file,
            # as far as can be seen.
            # BUT we can insert ourselves into the initialisation by
            # adding files to the conf.d directory.
            __magic_value="$RANDOM.$$"
            mkdir -p "${XDG_CONFIG_HOME:-${HOME}/.config}/fish/conf.d"
            cat <<EOM > "${XDG_CONFIG_HOME:-${HOME}/.config}/fish/conf.d/pythonlib.$__magic_value.fish"
set -l me "${XDG_CONFIG_HOME:-${HOME}/.config}/fish/conf.d/pythonlib.$__magic_value.fish"
if [ "\$__PYTHONLIB_MAGIC" = "$__magic_value" ]
    source "${_PYTHONLIB_ENVDIR}/bin/activate.fish"
    # Remove ourselves now that we've been used.
    rm "\$me"
else
    set -l hours_ago (math \\( (date +%s) - (stat --format '%Z' "\$me") \\) / \\( 60 \\* 60 \\))
    if test \$hours_ago -ge 2
        # This script hasn't been used for over 2 hours - something has
        # gone wrong, and it's not going to be invoked. It should really
        # have been run within moments.
        rm "\$me"
    end
end
EOM
            echo "Press ctrl-d, or use 'exit' to end shell."
            __PYTHONLIB_MAGIC=${__magic_value} fish
        elif [[ "$SHELL" =~ /bash ]] ; then
            # Start the bash shell, with the environment activated by
            # making the rcfile run the user's file, then the activation.
            echo "Press ctrl-d, or use 'exit' to end shell."
            bash -i --rcfile <(echo "source ~/.bashrc ; source '${_PYTHONLIB_ENVDIR}/bin/activate'") 

(anyone wishing to cry should can join me now)

As you can see, the bash solution is a little simpler.

So, what would be awesome, to avoid this 'ugliness', would be if the fish command line offered the ability to source some files after the user's configuration has started. I can obviously work around this by changing my own configuration files to source a file, if a specific variable has been set, but it would be nice if this could be achieved without user-specific intervention in their own shell - the whole point of this is that executing a single command will ensure that variables, paths, and tools are setup and installed from a single command, and that the user's preference for shell is honoured (I don't think it's all that nice to force people to work with only bash in the custom environment; after all, the niceness of the alternative is why we use fish 😄)

[zanchey]

I think the simplest, and possibly best, way of going about this is to add an option to the fish binary to enter interactive mode after parsing the command-line files or arguments; I'm still unsure why bash and fish don't do this with the combination of -c and -i.

[krader1961]

I added the "RFC" label because it is unclear what, if anything, we want changed with respect to how fish works.

Would you agree, @gerph, that the bash example you provided is not friendly? Are you aware that zsh does not provide an equivalent feature? At least AFAICT. I'm quite confident we do not want to emulate the bash --rcfile behavior. At least not literally.

I also use Anaconda. But I wrote my own con and coff fish functions. I can do con ip3, for example, and start a fish subshell and it uses that conda activated environment which I can then manipulate with my con and coff functions without affecting the parent shell. I don't need something like the --rcfile flag to get the subshell (possibly created indirectly via another program like vim or emacs) to honor my previous selection. Do you have another example for how this would be useful to you? Or can you clarify why you're doing this in bash?

I like @zanchey's idea of allowing -c a_command in conjunction with -i to mean execute command then enter interactive shell mode. This shouldn't cause problems but there might be uses of sh -i -c a_command to force interactive context while executing a_command. That is fundamentally broken but we can't change it in a minor release. We could, however, change it in a major release and we've recently decided to make a fish 3.0 release. So the question is whether this change, assuming it solves a real problem, should be included.

[faho]

I like @zanchey's idea of allowing -c a_command in conjunction with -i to mean execute command then enter interactive shell mode. This shouldn't cause problems but there might be uses of sh -i -c a_command to force interactive context while executing a_command.

One thing I sometimes recommend when benchmarking a fish config's startup time is to run fish --profile /tmp/fish.prof -ic 'fish_prompt; fish_right_prompt'. This would then have to include "exit" as well.

Alternatively, we could add a new option ("-C" or so) that doesn't exit after executing the command - this would essentially be an opt-in to the new behavior.

[zanchey]

Yes, that makes good sense to me.

[krader1961]

Yeah, something like a -C flag would work as a means of implicitly sourcing additional scripts after the normal rc script but before issuing the first prompt or whatever command was specified by -c. But before we implement it I'd like clarification from the O.P. on why they need this. Other examples would help make a compelling argument for doing the work to implement this.

Also, retargeting to fish-future since the tentative solution could be introduced in a minor release.

[gerph]

Thanks for the consideration 😄 - yes, I like the idea of '-C' option as well.

To answer the questions asked...

Would you agree, @gerph, that the bash example you provided is not friendly?

Oh absolutely... It's ugly, but it's at least a way to achieve my goal. Which, is about on par with most of bash really 😆

Are you aware that zsh does not provide an equivalent feature?

Bother. No I wasn't. I had been hoping to cross that bridge when I came to it, and only support bash and fish right now. But I did want to handle zsh too.

Do you have another example for how this would be useful to you? Or can you clarify why you're doing this in bash?

I guess I've not explained my intention very well. This is part of a development setup and CI system. The goal is to have a command that a developer (or automated system) runs in order to set up the environment for the work they are doing. They run it, the environment is set up, and they're dropped into a clean shell where they can do the work. On leaving the shell, they return to the previous environment.

So in the case of python, the tool sets up a virtualenv, installs the necessary requirements. In the perl case, a local cpan installation is created and the path updated to use it. Similarly, an environment for gems, shell scripts, and others are set up as necessary. The languages can be combined, and the resulting environment is set up for those languages. Once an environment has been constructed, the shell is started to let the user work. Ideally this should support many of the shells that people might use, but initially it's fish and bash that I'm starting on, as the shell I use, and the most common shell.

Why write it in bash? 'Lowest Common Denominator' means that bash is most likely to be available on a given system, and is neutral. I have an aversion for using a language to bootstrap its own environment, because the mental (and technical) hoops that are involved in doing that make it more difficult, and using an unrelated language then places an extra burden on the user to ensure that language is present and to have the right feature set - which was something that the script itself was trying to ensure for the target language, and kinda defeats the purpose of having a tool that sets everything up for you to work with the project.

The example commands given for anaconda are nice, but the purpose of the tool is to be able to be run by anyone and it will leave them with an environment set up, in a shell that they like/know, without the user having any special awareness of what's going on or installing things to make that happen.

[...] we could add a new option ("-C" or so) that doesn't exit after executing the command - this would essentially be an opt-in to the new behavior.

I like that; I was surprised that the -c option, together with -i, in bash didn't do what I needed, and after solving that problem with the ugliness you saw, I was disheartened that it also wasn't the case in fish either. Maybe I'm attacking the problem in an awkward way, but I feel this all ought to be possible.

But before we implement it I'd like clarification from the O.P. on why they need this. Other examples would help make a compelling argument for doing the work to implement this.

I hope I've managed to give a reasonable explanation of what I'm trying to achieve. I don't have any particular examples of other uses of such a switch, I'm afraid.

[krader1961]

I hope I've managed to give a reasonable explanation of what I'm trying to achieve.

You have. Thanks for taking the time to provide additional details. It appears you're creating a framework that will run on the developer workstation to provide a nested interactive shell. In which they can then run commands relevant to to completing tasks relevant to that framework while otherwise using the developers home directory. Yes? Because if that isn't the case I don't see why you couldn't just provide a custom config.fish (which contains the appropriate setup commands) and related files in a temporary directory and use the env HOME=xxx fish trick to use that custom environment.

In any case implementing this should be relatively straightforward, even borderline trivial, and can't break existing behavior. So I don't see any reason not to do it. Would you be interested in creating a pull-request, @gerph? If you do I promise quick turnaround on reviewing and merging it. Otherwise it might be many months before this gets implemented since we're about to embark on creating fish 3.0.

[gerph]

You have. Thanks for taking the time to provide additional details. It appears you're creating a framework that will run on the developer workstation to provide a nested interactive shell. In which they can then run commands relevant to to completing tasks relevant to that framework while otherwise using the developers home directory. Yes?

That's correct, yep :-)

Because if that isn't the case I don't see why you couldn't just provide a custom config.fish (which contains the appropriate setup commands) and related files in a temporary directory and use the env HOME=xxx fish trick to use that custom environment.

Indeed; the HOME solution had occurred to me, but since I want a shell and the HOME might not work (plus, of course, fish uses XDG_CONFIG_DIR), because other tools might rely on things related to the home directory, etc. But yes, that would have been a solution were it to be a simpler goal.

I'll have a look at the source when I get home this evening and see whether a change would be simple enough to add, for me. I'm hoping it does end up trivial :-D

[krader1961]

I'll have a look at the source when I get home this evening...

Great. We're always looking for users willing to roll up their sleeves and modify the core code. We're happy to answer questions about fish C++ source layout, APIs, coding style, unit tests, etc. Don't worry about things like running make lint and make style mentioned in the contributing doc unless you find yourself contributing on a regular basis.

[gerph]

Thanks, the implementation was actually quite simple, as the code was pretty well structured. Hopefully it will be acceptable anyhow. The testing on the other hand took about 5 times as long. Hopefully they are generic enough to be used for other parts of the invocation testing.

[krader1961]

There is now a new fish -C flag that addresses this issue and is likely to prove useful in other use cases thanks to the work by @gerph.

[gerph]

Thank you - nice to have constructive and encouraging comments from an open source project. And to be able to contribute a minor feature without having the fight tooth-and-nail to justify it :)