Web Tools

web_tools.bsh

Tools to help interface with the internet

download_to_stdout_wget

Download a file from the internet and output stream to stdout using wget

Return value:

0 - Successful download
Non-zero - Failed download
100 - wget not found

Arguments:

$1 - URL to download

Inputs:

WGET - Variable to specify wget executable, defaults to wget WGET_ARGS - Array of optional args to add to wget call. -qO - is always added

download_to_stdout_curl

Download a file from the internet and output stream to stdout using curl

Return value:

0 - Successful download
Non-zero - Failed download
100 - curl not found

Arguments:

$1 - URL to download

Inputs:

CURL - Variable to specify curl executable, defaults to curl CURL_ARGS - Array of optional args to add to wget call, -fsSL is always added

download_to_stdout_python

Download a file from the internet and output stream to stdout using python

Return value:

0 - Successful download
Non-zero - Failed download
100 - python not found

Arguments:

$1 - URL to download

Inputs:

PYTHON - Variable to specify python executable, defaults to python

Compatible with python2 and python3, and attempts to use requests, then urllib2 and urllib

download_to_file_python

Download a file from the internet and save it to a file using python

Return value:

0 - Successful download
Non-zero - Failed download
100 - python not found

Arguments:

$1 - URL to download
$2 - Filename to save to

Inputs:

PYTHON - Variable to specify python executable, defaults to python

Compatible with python2 and python3, and attempts to use requests, then urllib2 and urllib

download_to_stdout_ruby

Download a file from the internet and output stream to stdout using ruby, using open-uri.

Return value:

0 - Successful download
Non-zero - Failed download
100 - ruby not found

Arguments:

$1 - URL to download

Inputs:

RUBY - Variable to specify ruby executable, defaults to ruby

download_to_stdout_perl

Download a file from the internet and output stream to stdout using perl, using LWP::Simple.

Return value:

0 - Successful download
Non-zero - Failed download
100 - perl not found

Arguments:

$1 - URL to download

Inputs:

PERL - Variable to specify perl executable, defaults to perl

download_to_stdout

Download a file from the internet and output stream to stdout

Arguments:

$1 - URL to download

Output:

stdout - Binary stream of the url

Return Value:

0 - URL downloaded successfully
Non-zero - Failed download
100 - No download method found

Tries to download a file via various methods, in order:

wget using wget -q
curl using curl -fsSL
python using the requests, urllib2 or urllib library
ruby using the open-uri library
perl using the LWP::Simple library

download_to_file

Download a file from the internet and save it to a file

Arguments:

$1 - URL to download
$2 - Filename to save to

Return Value:

0 - URL downloaded successfully
Non-zero - Failed download
100 - No download method found

Tries to download a file via various methods, in order:

wget using wget -q
curl using curl -fsSL
python 2 and 3 using the requests, urllib2 or urllib library
ruby using the open-uri library
perl using the LWP::Simple library

This version is more persistent than download_to_stdout because it saves to a file. If a download fails when partially completed, the next method will attempt again from the beginning. The streaming version cannot start over (and the filename used should be an actual filename, not a file object without seek support, like a pipe).

is_port_listening

Checks to see if the port is currently listening.

Arguments:

$1 - Port to check
VSI_LIST_PROTO - The protocol to check, default is -t (TCP).

Return Value:

0 - Port is listening
1 - Port is not listening

Uses:

ss or netstat

find_available_port

Find the first available port for listening

Arguments:

$1 - Minimal port number to check
[$2] - Max port number to check

Output:

stdout - Port number found

Return Value:

0 - Available port found in the specified range
1 - No port found in the specified range