Expect
Extension to the Tcl scripting language
- 10 min read
- 3 chapters
Related on Lably
Through Microsoft Windows, Application software and Telnet
Basics
Expect is used to automate control of interactive applications such as Telnet, FTP, passwd, fsck, rlogin, tip, SSH, and others. Expect uses pseudo terminals (Unix) or emulates a console (Windows), starts the target program, and then communicates with it, just as a human would, via the terminal or console interface. Tk, another Tcl extension, can be used to provide a GUI.
Sign in to keep reading
2 more chapters are free with a Lably account.
Usage
Expect serves as a "glue" to link existing utilities together. The general idea is to figure out how to make Expect use the system's existing tools rather than figure out how to solve a problem inside of Expect.
A key usage of Expect involves commercial software products. Many of these products provide some type of command-line interface, but these usually lack the power needed to write scripts. They were built to service the users administering the product, but the company often does not spend the resources to fully implement a robust scripting language. An Expect script can spawn a shell, look up environmental variables, perform some Unix commands to retrieve more information, and then enter into the product's command-line interface armed with the necessary information to achieve the user's goal. After retrieving information by interacting with the product via its command-line interface, the script can make intelligent decisions about what action to take, if any.
Every time an Expect operation is completed, the results are stored in a local variable called $expect_out. This allows the script to harvest information to feedback to the user, and it also allows conditional behavior of what to send next based on the circumstances.
A common use of Expect is to set up a testing suite for programs, utilities or embedded systems. DejaGnu is a testing suite written using Expect for use in testing. It has been used for testing GCC and remote targets such as embedded development.
Expect script can be automated using a tool called 'autoexpect'. This tool observes your actions and generates an Expect script using heuristics. Though generated code may be large and somewhat cryptic, one can always tweak the generated script to get the exact code.
<span id="mwQQ"></span><span class="c" id="mwQg"># Assume $remote_server, $my_user_id, $my_password, and </span> <span class="c" id="mwQw"># $my_command were read earlier in the script.</span> <span class="c" id="mwRA"># Open a Telnet session to a remote server, and wait </span> <span class="c" id="mwRQ"># for a username prompt.</span> <span class="nv" id="mwRg">spawn</span><span class="w" id="mwRw"> </span>telnet<span class="w" id="mwSA"> </span><span class="nv" id="mwSQ">$remote_server</span> <span class="nv" id="mwSg">expect</span><span class="w" id="mwSw"> </span><span class="s2" id="mwTA">"username:"</span> <span class="c" id="mwTQ"># Send the username, and then wait for a password prompt.</span> <span class="nv" id="mwTg">send</span><span class="w" id="mwTw"> </span><span class="s2" id="mwUA">"$my_user_id\r"</span> <span class="nv" id="mwUQ">expect</span><span class="w" id="mwUg"> </span><span class="s2" id="mwUw">"password:"</span> <span class="c" id="mwVA"># Send the password, and then wait for a shell prompt.</span> <span class="nv" id="mwVQ">send</span><span class="w" id="mwVg"> </span><span class="s2" id="mwVw">"$my_password\r"</span> <span class="nv" id="mwWA">expect</span><span class="w" id="mwWQ"> </span><span class="s2" id="mwWg">"%"</span> <span class="c" id="mwWw"># Send the prebuilt command, and then wait </span> <span class="c" id="mwXA"># for another shell prompt.</span> <span class="nv" id="mwXQ">send</span><span class="w" id="mwXg"> </span><span class="s2" id="mwXw">"$my_command\r"</span> <span class="nv" id="mwYA">expect</span><span class="w" id="mwYQ"> </span><span class="s2" id="mwYg">"%"</span> <span class="c" id="mwYw"># Capture the results of the command into a variable. This </span> <span class="c" id="mwZA"># can be displayed, or written to disk.</span> <span class="k" id="mwZQ">set</span><span class="w" id="mwZg"> </span>results<span class="w" id="mwZw"> </span><span class="nv" id="mwaA">$expect_out</span><span class="k" id="mwaQ">(</span><span class="nv" id="mwag">buffer</span><span class="k" id="mwaw">)</span> <span class="c" id="mwbA"># Exit the Telnet session, and wait for a special </span> <span class="c" id="mwbQ"># end-of-file character.</span> <span class="nv" id="mwbg">send</span><span class="w" id="mwbw"> </span><span class="s2" id="mwcA">"exit\r"</span> <span class="nv" id="mwcQ">expect</span><span class="w" id="mwcg"> </span>eof
Another example is a script that automates FTP:
<span id="mwdg"></span><span class="c" id="mwdw"># Set timeout parameter to a proper value.</span> <span class="c" id="mweA"># For example, the file size is indeed big and the network </span> <span class="c" id="mweQ"># speed is really one problem, you'd better set this </span> <span class="c" id="mweg"># parameter a value.</span> <span class="k" id="mwew">set</span><span class="w" id="mwfA"> </span>timeout<span class="w" id="mwfQ"> </span><span class="o" id="mwfg">-</span><span class="mi" id="mwfw">1</span> <span class="c" id="mwgA"># Open an FTP session to a remote server, and </span> <span class="c" id="mwgQ"># wait for a username prompt.</span> <span class="nv" id="mwgg">spawn</span><span class="w" id="mwgw"> </span>ftp<span class="w" id="mwhA"> </span><span class="nv" id="mwhQ">$remote_server</span> <span class="nv" id="mwhg">expect</span><span class="w" id="mwhw"> </span><span class="s2" id="mwiA">"username:"</span> <span class="c" id="mwiQ"># Send the username, and then wait for a password prompt.</span> <span class="nv" id="mwig">send</span><span class="w" id="mwiw"> </span><span class="s2" id="mwjA">"$my_user_id\r"</span> <span class="nv" id="mwjQ">expect</span><span class="w" id="mwjg"> </span><span class="s2" id="mwjw">"password:"</span> <span class="c" id="mwkA"># Send the password, and then wait for an 'ftp' prompt.</span> <span class="nv" id="mwkQ">send</span><span class="w" id="mwkg"> </span><span class="s2" id="mwkw">"$my_password\r"</span> <span class="nv" id="mwlA">expect</span><span class="w" id="mwlQ"> </span><span class="s2" id="mwlg">"ftp>"</span> <span class="c" id="mwlw"># Switch to binary mode, and then wait for an 'ftp' prompt.</span> <span class="nv" id="mwmA">send</span><span class="w" id="mwmQ"> </span><span class="s2" id="mwmg">"bin\r"</span> <span class="nv" id="mwmw">expect</span><span class="w" id="mwnA"> </span><span class="s2" id="mwnQ">"ftp>"</span> <span class="c" id="mwng"># Turn off prompting.</span> <span class="nv" id="mwnw">send</span><span class="w" id="mwoA"> </span><span class="s2" id="mwoQ">"prompt\r"</span> <span class="nv" id="mwog">expect</span><span class="w" id="mwow"> </span><span class="s2" id="mwpA">"ftp>"</span> <span class="c" id="mwpQ"># Get all the files</span> <span class="nv" id="mwpg">send</span><span class="w" id="mwpw"> </span><span class="s2" id="mwqA">"mget *\r"</span> <span class="nv" id="mwqQ">expect</span><span class="w" id="mwqg"> </span><span class="s2" id="mwqw">"ftp>"</span> <span class="c" id="mwrA"># Exit the FTP session, and wait for a special </span> <span class="c" id="mwrQ"># end-of-file character.</span> <span class="nv" id="mwrg">send</span><span class="w" id="mwrw"> </span><span class="s2" id="mwsA">"bye\r"</span> <span class="nv" id="mwsQ">expect</span><span class="w" id="mwsg"> </span>eof
Below is an example that automates SFTP (with a password):
<span id="mwtg"></span><span class="c" id="mwtw">#!/usr/bin/env expect -f</span> <span class="c" id="mwuA"># Procedure to attempt connecting; result 0 if OK, 1 otherwise</span> <span class="k" id="mwuQ">proc</span><span class="w" id="mwug"> </span>connect<span class="w" id="mwuw"> </span><span class="k" id="mwvA">{</span><span class="nv" id="mwvQ">passw</span><span class="k" id="mwvg">}</span><span class="w" id="mwvw"> </span><span class="k" id="mwwA">{</span> <span class="w" id="mwwQ"> </span><span class="nv" id="mwwg">expect</span><span class="w" id="mwww"> </span><span class="k" id="mwxA">{</span> <span class="w" id="mwxQ"> </span><span class="s2" id="mwxg">"Password:"</span><span class="w" id="mwxw"> </span><span class="k" id="mwyA">{</span> <span class="w" id="mwyQ"> </span><span class="nv" id="mwyg">send</span><span class="w" id="mwyw"> </span><span class="s2" id="mwzA">"$passw\r"</span> <span class="w" id="mwzQ"> </span><span class="nv" id="mwzg">expect</span><span class="w" id="mwzw"> </span><span class="k" id="mw0A">{</span> <span class="w" id="mw0Q"> </span><span class="s2" id="mw0g">"sftp*"</span><span class="w" id="mw0w"> </span><span class="k" id="mw1A">{</span> <span class="w" id="mw1Q"> </span><span class="k" id="mw1g">return</span><span class="w" id="mw1w"> </span><span class="mi" id="mw2A">0</span> <span class="w" id="mw2Q"> </span><span class="k" id="mw2g">}</span> <span class="w" id="mw2w"> </span><span class="k" id="mw3A">}</span> <span class="w" id="mw3Q"> </span><span class="k" id="mw3g">}</span> <span class="w" id="mw3w"> </span><span class="k" id="mw4A">}</span> <span class="w" id="mw4Q"> </span><span class="c" id="mw4g"># Timed out</span> <span class="w" id="mw4w"> </span><span class="k" id="mw5A">return</span><span class="w" id="mw5Q"> </span><span class="mi" id="mw5g">1</span> <span class="k" id="mw5w">}</span> <span class="c" id="mw6A"># Read the input parameters</span> <span class="k" id="mw6Q">set</span><span class="w" id="mw6g"> </span>user<span class="w" id="mw6w"> </span><span class="k" id="mw7A">[</span><span class="nb" id="mw7Q">lindex</span><span class="w" id="mw7g"> </span><span class="nv" id="mw7w">$argv</span><span class="w" id="mw8A"> </span><span class="mi" id="mw8Q">0</span><span class="k" id="mw8g">]</span> <span class="k" id="mw8w">set</span><span class="w" id="mw9A"> </span>passw<span class="w" id="mw9Q"> </span><span class="k" id="mw9g">[</span><span class="nb" id="mw9w">lindex</span><span class="w" id="mw-A"> </span><span class="nv" id="mw-Q">$argv</span><span class="w" id="mw-g"> </span><span class="mi" id="mw-w">1</span><span class="k" id="mw_A">]</span> <span class="k" id="mw_Q">set</span><span class="w" id="mw_g"> </span>host<span class="w" id="mw_w"> </span><span class="k" id="mwAQA">[</span><span class="nb" id="mwAQE">lindex</span><span class="w" id="mwAQI"> </span><span class="nv" id="mwAQM">$argv</span><span class="w" id="mwAQQ"> </span><span class="mi" id="mwAQU">2</span><span class="k" id="mwAQY">]</span> <span class="k" id="mwAQc">set</span><span class="w" id="mwAQg"> </span>location<span class="w" id="mwAQk"> </span><span class="k" id="mwAQo">[</span><span class="nb" id="mwAQs">lindex</span><span class="w" id="mwAQw"> </span><span class="nv" id="mwAQ0">$argv</span><span class="w" id="mwAQ4"> </span><span class="mi" id="mwAQ8">3</span><span class="k" id="mwARA">]</span> <span class="k" id="mwARE">set</span><span class="w" id="mwARI"> </span>file1<span class="w" id="mwARM"> </span><span class="k" id="mwARQ">[</span><span class="nb" id="mwARU">lindex</span><span class="w" id="mwARY"> </span><span class="nv" id="mwARc">$argv</span><span class="w" id="mwARg"> </span><span class="mi" id="mwARk">4</span><span class="k" id="mwARo">]</span> <span class="k" id="mwARs">set</span><span class="w" id="mwARw"> </span>file2<span class="w" id="mwAR0"> </span><span class="k" id="mwAR4">[</span><span class="nb" id="mwAR8">lindex</span><span class="w" id="mwASA"> </span><span class="nv" id="mwASE">$argv</span><span class="w" id="mwASI"> </span><span class="mi" id="mwASM">5</span><span class="k" id="mwASQ">]</span> <span class="c" id="mwASU">#puts "Argument data:\n";</span> <span class="c" id="mwASY">#puts "user: $user";</span> <span class="c" id="mwASc">#puts "passw: $passw";</span> <span class="c" id="mwASg">#puts "host: $host";</span> <span class="c" id="mwASk">#puts "location: $location";</span> <span class="c" id="mwASo">#puts "file1: $file1";</span> <span class="c" id="mwASs">#puts "file2: $file2";</span> <span class="c" id="mwASw"># Check if all were provided</span> <span class="k" id="mwAS0">if</span><span class="w" id="mwAS4"> </span><span class="k" id="mwAS8">{</span><span class="w" id="mwATA"> </span><span class="nv" id="mwATE">$user</span><span class="w" id="mwATI"> </span><span class="o" id="mwATM">==</span><span class="w" id="mwATQ"> </span><span class="s2" id="mwATU">""</span><span class="w" id="mwATY"> </span><span class="o" id="mwATc">||</span><span class="w" id="mwATg"> </span><span class="nv" id="mwATk">$passw</span><span class="w" id="mwATo"> </span><span class="o" id="mwATs">==</span><span class="w" id="mwATw"> </span><span class="s2" id="mwAT0">""</span><span class="w" id="mwAT4"> </span><span class="o" id="mwAT8">||</span><span class="w" id="mwAUA"> </span><span class="nv" id="mwAUE">$host</span><span class="w" id="mwAUI"> </span><span class="o" id="mwAUM">==</span><span class="w" id="mwAUQ"> </span><span class="s2" id="mwAUU">""</span><span class="w" id="mwAUY"> </span><span class="o" id="mwAUc">||</span><span class="w" id="mwAUg"> </span><span class="nv" id="mwAUk">$location</span><span class="w" id="mwAUo"> </span><span class="o" id="mwAUs">==</span><span class="w" id="mwAUw"> </span><span class="s2" id="mwAU0">""</span><span class="w" id="mwAU4"> </span><span class="o" id="mwAU8">||</span><span class="w" id="mwAVA"> </span><span class="nv" id="mwAVE">$file1</span><span class="w" id="mwAVI"> </span><span class="o" id="mwAVM">==</span><span class="w" id="mwAVQ"> </span><span class="s2" id="mwAVU">""</span><span class="w" id="mwAVY"> </span><span class="o" id="mwAVc">||</span><span class="w" id="mwAVg"> </span><span class="nv" id="mwAVk">$file2</span><span class="w" id="mwAVo"> </span><span class="o" id="mwAVs">==</span><span class="w" id="mwAVw"> </span><span class="s2" id="mwAV0">""</span><span class="w" id="mwAV4"> </span><span class="k" id="mwAV8">}</span><span class="w" id="mwAWA"> </span><span class="k" id="mwAWE">{</span> <span class="w" id="mwAWI"> </span><span class="nb" id="mwAWM">puts</span><span class="w" id="mwAWQ"> </span><span class="s2" id="mwAWU">"Usage: <user> <passw> <host> <location> <file1 to send> <file2 to send>\n"</span> <span class="w" id="mwAWY"> </span><span class="nb" id="mwAWc">exit</span><span class="w" id="mwAWg"> </span><span class="mi" id="mwAWk">1</span> <span class="k" id="mwAWo">}</span> <span class="c" id="mwAWs"># Sftp to specified host and send the files</span> <span class="nv" id="mwAWw">spawn</span><span class="w" id="mwAW0"> </span>sftp<span class="w" id="mwAW4"> </span><span class="nv" id="mwAW8">$user</span>@<span class="nv" id="mwAXA">$host</span> <span class="k" id="mwAXE">set</span><span class="w" id="mwAXI"> </span>rez<span class="w" id="mwAXM"> </span><span class="k" id="mwAXQ">[</span><span class="nv" id="mwAXU">connect</span><span class="w" id="mwAXY"> </span><span class="nv" id="mwAXc">$passw</span><span class="k" id="mwAXg">]</span> <span class="k" id="mwAXk">if</span><span class="w" id="mwAXo"> </span><span class="k" id="mwAXs">{</span><span class="w" id="mwAXw"> </span><span class="nv" id="mwAX0">$rez</span><span class="w" id="mwAX4"> </span><span class="o" id="mwAX8">==</span><span class="w" id="mwAYA"> </span><span class="nv" id="mwAYE">0</span><span class="w" id="mwAYI"> </span><span class="k" id="mwAYM">}</span><span class="w" id="mwAYQ"> </span><span class="k" id="mwAYU">{</span> <span class="w" id="mwAYY"> </span><span class="nv" id="mwAYc">send</span><span class="w" id="mwAYg"> </span><span class="s2" id="mwAYk">"cd $location\r"</span> <span class="w" id="mwAYo"> </span><span class="k" id="mwAYs">set</span><span class="w" id="mwAYw"> </span>timeout<span class="w" id="mwAY0"> </span><span class="o" id="mwAY4">-</span><span class="mi" id="mwAY8">1</span> <span class="w" id="mwAZA"> </span><span class="nv" id="mwAZE">send</span><span class="w" id="mwAZI"> </span><span class="s2" id="mwAZM">"put $file2\r"</span> <span class="w" id="mwAZQ"> </span><span class="nv" id="mwAZU">send</span><span class="w" id="mwAZY"> </span><span class="s2" id="mwAZc">"put $file1\r"</span> <span class="w" id="mwAZg"> </span><span class="nv" id="mwAZk">send</span><span class="w" id="mwAZo"> </span><span class="s2" id="mwAZs">"ls -l\r"</span> <span class="w" id="mwAZw"> </span><span class="nv" id="mwAZ0">send</span><span class="w" id="mwAZ4"> </span><span class="s2" id="mwAZ8">"quit\r"</span> <span class="w" id="mwAaA"> </span><span class="nv" id="mwAaE">expect</span><span class="w" id="mwAaI"> </span>eof <span class="w" id="mwAaM"> </span><span class="nb" id="mwAaQ">exit</span><span class="w" id="mwAaU"> </span><span class="mi" id="mwAaY">0</span> <span class="k" id="mwAac">}</span> <span class="nb" id="mwAag">puts</span><span class="w" id="mwAak"> </span><span class="s2" id="mwAao">"\nError connecting to server: $host, user: $user and password: $passw!\n"</span> <span class="nb" id="mwAas">exit</span><span class="w" id="mwAaw"> </span><span class="mi" id="mwAa0">1</span>Using passwords as command-line arguments, like in this example, is a huge security hole, as any other user on the machine can read this password by running "ps". You can, however, add code that will prompt you for your password rather than giving your password as an argument. This should be more secure. See the example below.
<span id="mwAbI"></span><span class="nv" id="mwAbM">stty</span><span class="w" id="mwAbQ"> </span><span class="o" id="mwAbU">-</span>echo <span class="nv" id="mwAbY">send_user</span><span class="w" id="mwAbc"> </span><span class="o" id="mwAbg">--</span><span class="w" id="mwAbk"> </span><span class="s2" id="mwAbo">"Enter Password: "</span> <span class="nv" id="mwAbs">expect_user</span><span class="w" id="mwAbw"> </span><span class="o" id="mwAb0">-</span>re<span class="w" id="mwAb4"> </span><span class="s2" id="mwAb8">"(.*)\n"</span> <span class="nv" id="mwAcA">send_user</span><span class="w" id="mwAcE"> </span><span class="s2" id="mwAcI">"\n"</span> <span class="nv" id="mwAcM">stty</span><span class="w" id="mwAcQ"> </span>echo <span class="k" id="mwAcU">set</span><span class="w" id="mwAcY"> </span>PASS<span class="w" id="mwAcc"> </span><span class="nv" id="mwAcg">$expect_out</span><span class="k" id="mwAck">(</span><span class="nv" id="mwAco">1</span>,string<span class="k" id="mwAcs">)</span>
Another example of automated SSH login to a user machine:
<span id="mwAc8"></span><span class="c" id="mwAdA"># Timeout is a predefined variable in Expect which by </span> <span class="c" id="mwAdE"># default is set to 10 seconds.</span> <span class="c" id="mwAdI"># spawn_id is another predefined variable in Expect.</span> <span class="c" id="mwAdM"># It is a good practice to close spawn_id handle </span> <span class="c" id="mwAdQ"># created by spawn command.</span> <span class="k" id="mwAdU">set</span><span class="w" id="mwAdY"> </span>timeout<span class="w" id="mwAdc"> </span><span class="mi" id="mwAdg">60</span> <span class="nv" id="mwAdk">spawn</span><span class="w" id="mwAdo"> </span>ssh<span class="w" id="mwAds"> </span><span class="nv" id="mwAdw">$user</span>@machine <span class="k" id="mwAd0">while</span><span class="w" id="mwAd4"> </span><span class="k" id="mwAd8">{</span><span class="nv" id="mwAeA">1</span><span class="k" id="mwAeE">}</span><span class="w" id="mwAeI"> </span><span class="k" id="mwAeM">{</span> <span class="w" id="mwAeQ"> </span><span class="nv" id="mwAeU">expect</span><span class="w" id="mwAeY"> </span><span class="k" id="mwAec">{</span> <span class="w" id="mwAeg"> </span><span class="nb" id="mwAek">eof</span><span class="w" id="mwAeo"> </span><span class="k" id="mwAes">{break}</span> <span class="w" id="mwAew"> </span><span class="s2" id="mwAe0">"The authenticity of host"</span><span class="w" id="mwAe4"> </span><span class="k" id="mwAe8">{</span><span class="nv" id="mwAfA">send</span><span class="w" id="mwAfE"> </span><span class="s2" id="mwAfI">"yes\r"</span><span class="k" id="mwAfM">}</span> <span class="w" id="mwAfQ"> </span><span class="s2" id="mwAfU">"password:"</span><span class="w" id="mwAfY"> </span><span class="k" id="mwAfc">{</span><span class="nv" id="mwAfg">send</span><span class="w" id="mwAfk"> </span><span class="s2" id="mwAfo">"$password\r"</span><span class="k" id="mwAfs">}</span> <span class="w" id="mwAfw"> </span><span class="s2" id="mwAf0">"*\]"</span><span class="w" id="mwAf4"> </span><span class="k" id="mwAf8">{</span><span class="nv" id="mwAgA">send</span><span class="w" id="mwAgE"> </span><span class="s2" id="mwAgI">"exit\r"</span><span class="k" id="mwAgM">}</span> <span class="w" id="mwAgQ"> </span><span class="k" id="mwAgU">}</span> <span class="k" id="mwAgY">}</span> <span class="nv" id="mwAgc">wait</span> <span class="nb" id="mwAgg">close</span><span class="w" id="mwAgk"> </span><span class="nv" id="mwAgo">$spawn_id</span>Alternatives
Various projects implement Expect-like functionality in other languages, such as C#, Java, Scala, Groovy, Perl, Python, Ruby, Shell and Go. These are generally not exact clones of the original Expect, but the concepts tend to be very similar.
C#
- Expect.NET — Expect functionality for C# (.NET)
- DotNetExpect — An Expect-inspired console automation library for .NET
Erlang
- lux - test automation framework with Expect style execution commands.
Go
- GoExpect - Expect-like package for the Go language
- go-expect - an Expect-like Go language library to automate control of terminal or console based programs.
Groovy
- expect4groovy - a Groovy DSL implementation of Expect tool.
Java
- ExpectIt — a pure Java 1.6+ implementation of the Expect tool. It is designed to be simple, easy to use and extensible.
- expect4j — an attempt at a Java clone of the original Expect
- ExpectJ — a Java implementation of the Unix expect utility
- Expect-for-Java — pure Java implementation of the Expect tool
- expect4java - a Java implementation of the Expect tool, but supports nested closures. There is also wrapper for Groovy language DSL.
Perl
- Expect.pm — Perl module (newest version at metacpan.org)
Python
- Pexpect — Python module for controlling interactive programs in a pseudo-terminal
- winpexpect — port of pexpect to the Windows platform
- paramiko-expect — A Python expect-like extension for the Paramiko SSH library which also supports tailing logs.
Ruby
- RExpect — a drop in replacement for the expect.rb module in the standard library.
- Expect4r — Interact with Cisco IOS, IOS-XR, and Juniper JUNOS CLI
Rust
- rexpect - pexpect-like package for the Rust language.
Scala
- scala-expect — a Scala implementation of a very small subset of the Expect tool.
Shell
- Empty — expect-like utility to run interactive commands in the Unix shell-scripts
- sexpect — Expect for shells. It's implemented in the client/server model which also supports attach/detach (like GNU screen).