123

I'm writing a shell script that should be somewhat secure i.e. does not pass secure data through parameters of commands and preferably does not use temporary files. How can I pass a variable to the stdin of a command? Or, if it's not possible, how to correctly use temporary files for such task?

Jonathan Leffler
  • 666,971
  • 126
  • 813
  • 1,185

9 Answers9

227

Passing a value to stdin in bash is as simple as:

your-command <<< "$your_variable"

Always make sure you put quotes around variable expressions!

Be cautious, that this will probably work only in bash and will not work in sh.

Slava Fomin II
  • 21,036
  • 19
  • 98
  • 176
Martin
  • 33,250
  • 14
  • 68
  • 78
  • 43
    Herestrings `<< – J. M. Becker Aug 07 '12 at 13:38
  • While this may be the case, sensitive information is likely to be more easily leaked if using the `echo` method due to the creation of a pipe. The herestring command will be processed entirely by `bash`, although in this situation (as as noted) you had better know that it will work on your bash, otherwise any error message produced as a result of not supporting herestrings would also leak said information. – Steven Lu Aug 13 '13 at 17:13
  • @StevenLu Wait, `echo` is a built-in. No pipe there, right? It's Unix, but it can't be *that much Unix*. – Camilo Martin Jun 22 '14 at 04:35
  • @CamiloMartin I don't know if bash will take e.g. `printf "abc" | cmd` and treat it the same as `cmd <<< "abc"`... – Steven Lu Jun 22 '14 at 10:47
  • 4
    @StevenLu `printf '%s\n' "$var"` produces the same results as `echo "$var"` but won't break if, e.g., `var=-en`, and doesn't even require bash. – Camilo Martin Jun 23 '14 at 01:28
  • @CamiloMartin Yeah I caught myself and edited my comment – Steven Lu Jun 23 '14 at 06:22
  • I tried this with the characters " and ! in the string. It failed. I tried again using """$your_variable""" instead and it worked. – Robert Jacobs Feb 27 '15 at 14:56
  • I think this is the only method shown on this question where the environ doesn't get passed though the process table at any point – ThorSummoner Aug 02 '18 at 20:31
  • 3
    Note also that a newline is appended to the string for here strings. – pdr Oct 14 '19 at 12:41
  • One disadvantage I found to this syntax is that piping output from the first command into further commands isn't as intuitive as what you'd get with `echo "$your_variable" | command1 | command2`. What would the syntax be with here strings in this case? – chunk_split Nov 02 '19 at 02:10
  • @chunk_split you could put the here string at the beginning of the command if you prefer: `<<< "$your_variable" command1 | command2` – Martin Nov 16 '19 at 13:00
84

Simple, but error-prone: using echo

Something as simple as this will do the trick:

echo "$blah" | my_cmd

Do note that this may not work correctly if $blah contains -n, -e, -E etc; or if it contains backslashes (bash's copy of echo preserves literal backslashes in absence of -e by default, but will treat them as escape sequences and replace them with corresponding characters even without -e if optional XSI extensions are enabled).

More sophisticated approach: using printf

printf '%s\n' "$blah" | my_cmd

This does not have the disadvantages listed above: all possible C strings (strings not containing NULs) are printed unchanged.

Charles Duffy
  • 235,655
  • 34
  • 305
  • 356
Oliver Charlesworth
  • 252,669
  • 29
  • 530
  • 650
  • 7
    Beware spaces in your variable: `echo "$blah"` is better. – Jonathan Leffler Jan 23 '11 at 19:27
  • 2
    The `(cat < – Evgeni Sergeev Dec 10 '13 at 07:02
  • 16
    Let's see how you handle `blah=-n`, `blah=-e`... use `printf` instead. http://unix.stackexchange.com/questions/65803/why-is-printf-better-than-echo – Camilo Martin Jun 22 '14 at 04:33
  • 2
    this seems like it would be fragile and error-prone, no? – ThorSummoner Nov 14 '17 at 22:02
  • 27
    6 years on, if I could delete this answer I would (but I can't because it's been accepted...) – Oliver Charlesworth Nov 15 '17 at 10:03
  • 1
    @OliverCharlesworth, why not edit it to use `printf '%s\n' "$blah"`? With that one change (avoiding the many pitfalls in `echo`'s specification, which Stephane's excellent answer goes into in detail at [Why is `printf` better than `echo`?](https://unix.stackexchange.com/questions/65803/why-is-printf-better-than-echo) on [unix.se], or which the APPLICATION USAGE section of the [`echo` specification](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/echo.html) touches on more briefly) it's a perfectly reasonable answer. – Charles Duffy Feb 21 '20 at 16:07
  • @CharlesDuffy there we go, you have it. – Per Lundberg May 21 '21 at 18:24
19

Note that the 'echo "$var" | command operations mean that standard input is limited to the line(s) echoed. If you also want the terminal to be connected, then you'll need to be fancier:

{ echo "$var"; cat - ; } | command

( echo "$var"; cat -   ) | command

This means that the first line(s) will be the contents of $var but the rest will come from cat reading its standard input. If the command does not do anything too fancy (try to turn on command line editing, or run like vim does) then it will be fine. Otherwise, you need to get really fancy - I think expect or one of its derivatives is likely to be appropriate.

The command line notations are practically identical - but the second semi-colon is necessary with the braces whereas it is not with parentheses.

Jonathan Leffler
  • 666,971
  • 126
  • 813
  • 1,185
  • `( echo "$LIST"; cat - ) | sed 1q` this works for me but I need to press ctrl d when I run this script? – Gert Cuykens Oct 21 '12 at 23:44
  • Yes; the `cat -` continues to read from the keyboard until EOF or interrupt, so you need to tell it EOF by typing control-D. – Jonathan Leffler Oct 21 '12 at 23:48
  • is there a way around EOF? sed needs cat – Gert Cuykens Oct 21 '12 at 23:55
  • You don't have to use `cat` at all if you don't want terminal input, as in the first line. Or you can use `cat` to list a file. Or ... If you want the command to read terminal input, you have to tell it when it has reached the end of the input. Or you could use `( echo "$var"; sed /quit/q - ) | command`; this continues until you type a line containing 'quit'. You can be endlessly inventive with how you handle it. Beware the old urban legend of a program that stopped working when the users began working with Ecuador. They'd type in the name of the capital, Quito, and the program exited. – Jonathan Leffler Oct 22 '12 at 00:17
  • OK if you say so. But why not just `echo "$LIST" | sed 1q | ...`? It all depends on what you're about. The `< – Jonathan Leffler Oct 22 '12 at 00:33
  • `echo "$LIST" | head -n1` works but `echo "$LIST" | sed 1q` did not? – Gert Cuykens Oct 22 '12 at 01:38
  • Don't know; they should be equivalent. – Jonathan Leffler Oct 22 '12 at 02:11
18
(cat <<END
$passwd
END
) | command

The cat is not really needed, but it helps to structure the code better and allows you to use more commands in parentheses as input to your command.

Michael
  • 5,910
  • 4
  • 52
  • 74
PoltoS
  • 1,133
  • 9
  • 32
  • But this method allows you to pass multiple lines to your command and also allows spaces in the `$passwd` – PoltoS Jan 25 '11 at 23:52
  • 4
    This is the best answer so far that does not leak variable contents to pipe or process snooping. – user2688272 Mar 31 '17 at 17:06
16

I liked Martin's answer, but it has some problems depending on what is in the variable. This

your-command <<< """$your_variable"""

is better if you variable contains " or !

Robert Jacobs
  • 2,932
  • 1
  • 15
  • 29
  • 4
    But why? Also, I can't reproduce any problems: `foo1=-; foo2='"'; foo3=\!; cat<< – phk May 27 '16 at 08:22
  • Try something real. Like your command is ssh somehost. and your variable is a shell script. – Robert Jacobs Jan 30 '17 at 13:48
  • This seems to append a newline – masterxilo Feb 11 '21 at 16:36
  • `"""foo"""` is treated **exactly** the same way as `"foo"` by the bash parser. `""` is just an empty quote pair -- starting and ending a quoted string without any content within it; so you have `"$your_variable"` concatenated with an empty quoted string at the front and end. – Charles Duffy May 21 '21 at 18:33
  • Granted, `!` (even when double-quoted) causes a lot of problems in interactive shells with history expansion turned on in general, but that's a good reason to turn history expansion _off_, so interactive shells parse the code the same way ones running scripts do. – Charles Duffy May 21 '21 at 18:35
  • (why does `!` behave in a way that doesn't follow the rule I described above? Because history expansion happens _before regular parsing even starts_; it's a very messy feature, and everyone's better off if it's just disabled in the first place). – Charles Duffy May 21 '21 at 18:49
15

This robust and portable way has already appeared in comments. It should be a standalone answer.

printf '%s' "$var" | my_cmd

or

printf '%s\n' "$var" | my_cmd

Notes:

  • It's better than echo, reasons are here: Why is printf better than echo?
  • printf "$var" is wrong. The first argument is format where various sequences like %s or \n are interpreted. To pass the variable right, it must not be interpreted as format.

  • Usually variables don't contain trailing newlines. The former command (with %s) passes the variable as it is. However tools that work with text may ignore or complain about an incomplete line (see Why should text files end with a newline?). So you may want the latter command (with %s\n) which appends a newline character to the content of the variable. Non-obvious facts:

    • Here string in Bash (<<<"$var" my_cmd) does append a newline.
    • Any method that appends a newline results in non-empty stdin of my_cmd, even if the variable is empty or undefined.
Kamil Maciorowski
  • 260
  • 2
  • 11
8

As per Martin's answer, there is a bash feature called Here Strings (which itself is a variant of the more widely supported Here Documents feature).

http://www.gnu.org/software/bash/manual/bashref.html#Here-Strings

3.6.7 Here Strings

A variant of here documents, the format is:

<<< word

The word is expanded and supplied to the command on its standard input.

Note that Here Strings would appear to be bash-only, so, for improved portability, you'd probably be better off with the original Here Documents feature, as per PoltoS's answer:

( cat <<EOF
$variable
EOF
) | cmd

Or, a simpler variant of the above:

(cmd <<EOF
$variable
EOF
)

You can omit ( and ), unless you want to have this redirected further into other commands.

cnst
  • 21,785
  • 2
  • 73
  • 108
4

Try this:

echo "$variable" | command
unbeli
  • 26,619
  • 5
  • 51
  • 55
-2

Just do:

printf "$my_var" | my_cmd

If the var doesn't contain spaces then the quotes may be omitted.
If using bash then you may also do:

echo -n "$my_var" | my_cmd

Avoid using echo without -n because it will pipe the vraiable with an added linebreak at the end.

Clox
  • 1,922
  • 5
  • 28
  • 43
  • 2
    doesn't work if $my_var is `%s`, printf won't print anything. `my_var="%s"; printf "$my_var"; ` - maybe try `printf "%s" "$my_var" | my_cmd` ? – hanshenrik Sep 21 '18 at 14:38