0

I have lots of data that needs to be processed, and have access to 3 separate remote servers. The logic is to split up the data crunching among the 3 different servers instead of running on a single one. Note, that all three remote servers are able to point to a single directory, which is where I have the master scripts to process all of the data. The problem I am have is carrying over my arguments when I call different bash scripts.

For example, I have the master script which looks something like:

processing stuff
more stuff
# call the first script
$scriptdir/step1.csh $date $time $name

Within step1.csh, if I have something very simple where I am able to connect to one of the remote servers and output the hostname to a text file, such as:

#!/bin/bash
ssh name@hostname bash -c '
echo `hostname` > host.txt

I get the desired outcome, where 'host.txt' will be the hostname of the desired connected hostname. However, If step1.csh looks like:

#!/bin/bash
mydate=$1
mytime=$2
myname=$3

ssh name@hostname bash '
echo `hostname` > host.txt
echo ${mydate} > host.txt

I get the error saying that 'mydate: undefined variable'

Furthermore, If I do something along the lines of:

#!/bin/bash
mydate=$1
mytime=$2
myname=$3
ssh name@hostname "python /path/to/somewhere/to/run/${mydate}/and/${mytime}

It still runs on the local, and not remote server. What am I missing here?

Miss_Orchid
  • 69
  • 6
  • if you are trying to distribute work across several servers, you really need to be using some kind of compute cluster with scheduler setup, like SLURM, SGE, etc. – user5359531 Jun 07 '18 at 19:58
  • potential answers [here](https://stackoverflow.com/questions/305035/how-to-use-ssh-to-run-a-shell-script-on-a-remote-machine?rq=1) and [here](https://stackoverflow.com/questions/4412238/what-is-the-cleanest-way-to-ssh-and-run-multiple-commands-in-bash?rq=1) as well – user5359531 Jun 07 '18 at 20:16

1 Answers1

0

So the first part:

#!/bin/bash
mydate=$1
mytime=$2
myname=$3

ssh name@hostname bash '
echo `hostname` > host.txt
echo ${mydate} > host.txt

The solution is:

#!/bin/bash
mydate=$1
mytime=$2
myname=$3

ssh -T name@hostname << EOF
echo `hostname` > host.txt
echo ${mydate} > host.txt
EOF

However, I am still having issues as in where I try to run a python script on the remote server; it is always ran on the local server.

Miss_Orchid
  • 69
  • 6