Run Remote Commands over SSH

In this post, we will discuss about the details on communication between two nodes in a network via SSH and executing/running remote commands over SSH on a remote machine.

These two nodes in the cluster can be treated as server/client machines for easy understanding. To allow secure communications between Server and client machines, on the server side, we will need a public key and an authorization file, and on the client side, we will need a private key and an identification file. The public key on the server and private key on the client must be a matching pair of keys as generated by the $ ssh-keygen command.

In simpler words, 

If we need to connect to a remote machine m2 from m1 we need to generate ssh-keygen on m1 and copy ~/.ssh/ file from m1 into ~/.ssh/authorized_keys in m2 machine. We can add any no of keys into this file for providing connection to m2 machine via ssh from many machines, but there should not be any empty lines between keys in authorized_keys file.

Now if we connect to m2 machine from m1 via $ ssh username@m2 it will directly connect without asking for password, if we have generated passwordless key in m1.

This is useful when there are N number of machines connected in a network and need to communicate with each other via ssh or scp without prompting for password. One real time example for this scenario is Hadoop Cluster.

The above setup will allow us to remotely login to another machine and and submit command on the remote machine. It is easy to submit single commands over SSH as shown below.

But it is a bit tricky to submit multiple commands over SSH. Below are the ways to submit multiple commands on a remote machine via SSH.

  • If commands are less and there are no control flow statements, (if, loops, etc…) then we can use them in single quotes, each command separated by semicolon as shown below.

  • If we need to use SSH in shell scripting and need to many commands on remote machine, including control flow statements like if, loop statements, then we can use it as follows.

But this method fails if we use to refer any local variables defined above the SSH login as shown below.

Here if we need to use local variables we can follow either of the below two options.

  • Write all the commands in a shell script file and the .sh file can be feed to SSH as shown below.

  • Embed all the commands in between tags like EOF, ENDSSH, ENDFTP, etc… as shown below. We can nest commands with this syntax.

  • For Interactive shell on remote machine from current machine’s command line we can fire below command.

The -t flag tells ssh that we’ll be interacting with remote shell. Without the -t flag top option will return results of commands and after that ssh will log out of the remote machine immediately. With the -t flag, ssh keeps us logged in until we exit the interactive command.

Sample Use Case to Run Remote Commands over SSH:

Below is the shell script for the sample use case where there are three machines m1, m2, m3. We need to copy files from m2 to m3 but this commands need to be submitted from machine m1. Copy below code into file.

About Siva

Senior Hadoop developer with 4 years of experience in designing and architecture solutions for the Big Data domain and has been involved with several complex engagements. Technical strengths include Hadoop, YARN, Mapreduce, Hive, Sqoop, Flume, Pig, HBase, Phoenix, Oozie, Falcon, Kafka, Storm, Spark, MySQL and Java.

Leave a comment

Your email address will not be published. Required fields are marked *

Review Comments
default image

I have attended Siva’s Spark and Scala training. He is good in presentation skills and explaining technical concepts easily to everyone in the group. He is having excellent real time experience and provided enough use cases to understand each concepts. Duration of the course and time management is awesome. Happy that I found a right person on time to learn Spark. Thanks Siva!!!

Dharmeswaran ETL / Hadoop Developer Spark Nov 2016 September 21, 2017