VMware ESXi and vSphere Cluster Management
Fetch: Downloading Files from the Command Line
Learn how to use the BSD fetch command to download HTTP, HTTPS, and FTP resources, resume transfers, configure proxies, verify checksums, and automate safe downloads.
What fetch does
fetch is a command-line client for downloading a remote resource to a local file. A remote resource is usually identified by a URL, or Uniform Resource Locator. Depending on the operating system implementation, fetch can retrieve resources over HTTP, HTTPS, and FTP.
A download copies data from a server to your computer. It is different from uploading, which sends a local file to a server, and from synchronization, which compares and updates directory trees. fetch is primarily a retrieval tool; it is not a general-purpose directory synchronization program.
Basic command structure
fetch [options] URLA URL has several possible parts:
- Scheme: The protocol, such as
http,https, orftp. - Host name: The server name, such as
example.org. - Path: The location of the resource on the server, such as
/files/package.tar.gz. - Query string: Optional data after
?, often used to select or customize a server response. - Local output name: The name under which the downloaded data is stored on your computer. This is normally selected by
fetchfrom the URL unless you specify one.
A URI is the broader term for a resource identifier. A URL is a URI that identifies a resource by its network location and access scheme.
Download one file
fetch https://example.org/files/package.tar.gzThis asks fetch to retrieve the file from the HTTPS server. The file is normally written in the shell's current working directory, which is the directory reported by pwd.
pwd
ls -l
fetch https://example.org/files/package.tar.gz
ls -l package.tar.gzIf the URL contains shell metacharacters such as ?, &, spaces, or parentheses, quote it:
fetch 'https://example.org/download?id=42&format=tar.gz'Choosing where to save files
When no output option is supplied, fetch commonly derives the local filename from the final component of the URL. Redirects, query strings, server behavior, and implementation-specific rules can affect that choice, so do not rely on it in automation.
Use an explicit output name when the name matters:
fetch -o package.tar.gz https://example.org/releases/latest.tar.gzThe output argument can usually include a path:
fetch -o /tmp/package.tar.gz https://example.org/releases/latest.tar.gzCreate or enter the intended directory before downloading when you want predictable relative paths:
mkdir -p downloads
cd downloads
fetch -o package.tar.gz https://example.org/files/package.tar.gzFor scripts, prefer an absolute or deliberately constructed destination and quote it:
destination="$HOME/downloads/package.tar.gz"
fetch -o "$destination" 'https://example.org/files/package.tar.gz'Multiple downloads and overwrites
Download each URL with an explicit, distinct destination when retrieving multiple files:
fetch -o package-amd64.tar.gz https://example.org/releases/package-amd64.tar.gz
fetch -o package-arm64.tar.gz https://example.org/releases/package-arm64.tar.gzBefore using an output name, check how the local implementation handles an existing file. Some versions may refuse, replace, or create a related name depending on options. In scripts, avoid accidental replacement by checking first:
if [ -e package.tar.gz ]; then
printf '%s\n' 'Refusing to replace existing package.tar.gz' >&2
exit 1
fi
fetch -o package.tar.gz https://example.org/files/package.tar.gzSome implementations support reading a sequence of URLs from a file:
fetch -f urls.txtVerify this option and the required file format with man fetch. Implementations differ in how they choose output names for a list of URLs.
Protocols and authentication
HTTP and HTTPS
HTTP is the standard protocol used by many web servers. HTTPS is HTTP protected by TLS. HTTPS provides encrypted transport and allows the client to validate that it is connected to the intended server, assuming certificate verification succeeds.
fetch https://example.org/files/package.tar.gzPrefer HTTPS when it is available, especially for software archives, credentials, and files whose contents must not be observed or modified in transit.
FTP
FTP is an older file-transfer protocol. FTP support depends on the installed fetch implementation and may be unavailable or restricted. When supported, an FTP URL has the form:
fetch ftp://example.org/pub/package.tar.gzFTP does not provide the same built-in transport protection as HTTPS. Use an authenticated and trusted service, and verify the downloaded file independently.
Authentication
A server may require authentication before allowing a download. The supported method depends on the server and local implementation. Consult man fetch for username options, password prompts, and URL authentication syntax.
Do not put passwords directly in a URL, shell command, shared script, process-visible command line, or shell history. A URL such as https://user:password@example.org/file can expose the password to history files, logs, process listings, or other users. Prefer an interactive prompt, a protected credential mechanism, or the service's documented noninteractive authentication method.
Redirects, responses, and exit status
An HTTP redirect is a server response telling the client to request another URL. Release links and shortened download URLs often redirect to a versioned or geographically selected location. A redirect is not automatically an error, but it changes where the content comes from. Read the command output and check the local manual for redirect-related options and limits.
HTTP status codes describe the server's response:
- 2xx: The request generally succeeded.
- 3xx: The resource may be at another URL, or additional action may be required.
- 401 Unauthorized: Authentication is missing or invalid.
- 403 Forbidden: The server understood the request but will not allow it.
- 404 Not Found: The path or resource does not exist at that location.
- 5xx: The server encountered an error or is temporarily unable to serve the request.
The shell receives an exit status from fetch. A zero status conventionally means success; a nonzero status means the command reported a failure. Check it immediately if you need to diagnose a command:
fetch -o package.tar.gz https://example.org/files/package.tar.gz
status=$?
printf 'fetch exit status: %s\n' "$status"
[ "$status" -eq 0 ]A zero exit status means the transfer completed according to the client. It does not prove that the data is the intended file or that the file is trustworthy. Inspect the file and verify its checksum when possible.
Resuming and retrying downloads
A large transfer can be interrupted, leaving a partial download. Resuming asks the server for the missing byte range rather than downloading the entire file again. This requires both a client mode that supports resuming and a server that supports HTTP range requests or the equivalent protocol feature.
fetch -r -o large-image.iso https://example.org/images/large-image.isoThe exact resume behavior is implementation-specific. Read man fetch before relying on -r. A server may ignore range requests, and a redirect or changing resource can make a partial file unsuitable.
Restart from the beginning when:
- The server does not support range requests.
- The remote file has changed since the partial download began.
- The partial file may be corrupt or came from an untrusted source.
- The client reports a range mismatch or produces a file with an unexpected size.
Retrying is useful for temporary DNS, connection, or server failures, but repeated retries do not fix an incorrect URL, authorization failure, certificate error, or missing file. Check the local manual for built-in retry options. If the implementation has no suitable retry mode, a shell loop can retry a command, but it should limit attempts and preserve useful error output.
Proxy and network configuration
A proxy is an intermediary that makes network requests for clients. Organizations may require an HTTP or HTTPS proxy for Internet access, traffic inspection, access control, or auditing. A proxy can also be required for FTP, depending on the network and local implementation.
Many command-line clients recognize environment variables. A common configuration for the current shell is:
export HTTPS_PROXY='http://proxy.example.org:8080'
export HTTP_PROXY='http://proxy.example.org:8080'To bypass the proxy for local or internal hosts, use NO_PROXY when supported:
export NO_PROXY='localhost,127.0.0.1,.example.internal'Variable names, case sensitivity, URL formats, and whether a particular fetch implementation reads these variables can vary. Some systems use fetch-specific configuration files or settings instead. Check man fetch and your network administrator's instructions.
Do not put proxy passwords in shared environment files or shell history. Environment variables can be visible to programs launched from the shell and may be exposed through diagnostics.
Conceptual network diagnosis
- DNS: Converts a host name into an address. A DNS failure prevents the client from finding the server.
- Firewall: May block outbound connections, proxy connections, or particular ports.
- Captive portal: A public network may redirect unauthenticated requests to a login page.
- TLS interception: A managed proxy may decrypt and re-encrypt HTTPS traffic. The system must trust the organization's certificate authority for validation to succeed.
TLS and certificate validation
TLS protects HTTPS connections. An SSL certificate, more precisely a TLS certificate, binds a server identity to a public key. A certificate authority, or CA, is an organization whose trusted certificates help clients validate that identity.
Certificate verification checks the certificate chain, expiration period, and hostname. A failure can indicate an incorrect system clock, a missing or outdated trusted CA store, a hostname mismatch, an expired certificate, or an intercepting proxy that has not been configured correctly.
Do not disable certificate validation merely to make a download work. Bypassing validation allows an attacker or misconfigured intermediary to impersonate the server or alter the download. Correct the system date and time, install the approved CA configuration, verify the exact hostname, or configure the approved proxy instead.
Noninteractive and scripted downloads
In automation, make the destination explicit, stop when fetch fails, check that the expected file exists, and avoid replacing a known-good file until the new file has been verified.
#!/bin/sh
set -eu
url='https://example.org/releases/latest.tar.gz'
tmp=$(mktemp "${TMPDIR:-/tmp}/package.XXXXXX")
destination="$HOME/downloads/package.tar.gz"
expected='REPLACE_WITH_A_TRUSTED_SHA256_DIGEST'
cleanup() {
rm -f "$tmp"
}
trap cleanup EXIT
mkdir -p "${destination%/*}"
fetch -o "$tmp" "$url"
test -s "$tmp"
actual=$(sha256 -q "$tmp")
if [ "$actual" != "$expected" ]; then
printf '%s\n' 'Checksum mismatch; refusing to install the file.' >&2
exit 1
fi
if [ -e "$destination" ]; then
printf 'Refusing to replace existing file: %s\n' "$destination" >&2
exit 1
fi
mv "$tmp" "$destination"
printf 'Downloaded and verified: %s\n' "$destination"Replace the placeholder digest with a SHA-256 value obtained from a trusted publisher source. The sha256 command and its quiet-output option can vary by operating system; use the checksum utility available on your system.
This pattern is useful because the temporary file is not treated as final until the download succeeds and the digest matches. It also makes the operation more idempotent: rerunning it does not silently replace an existing destination.
Logging and shell quoting
Capture command output in the automation system's logs, but avoid logging credentials, authorization headers, or proxy secrets. Quote URLs and filenames:
fetch -o "$output" "$url"Without quoting, spaces split one filename into multiple arguments, and characters such as &, ?, $, and parentheses can be interpreted by the shell rather than passed to fetch.
Inspecting and verifying downloads
After downloading, inspect the file before opening, extracting, or executing it:
ls -lh package.tar.gz
file package.tar.gz
sha256 package.tar.gzCompare the resulting SHA-256 digest with a value published by the software distributor through a trusted channel. A checksum detects accidental corruption and, when obtained from a trustworthy source, helps detect alteration.
A successful network transfer does not prove that the content is safe. A server can successfully deliver the wrong file, an HTML login page, malware, or a tampered archive. Never immediately execute a downloaded script or binary without checking its source, file type, checksum, signature when provided, and intended contents.
Common tasks and option categories
Troubleshooting download and network failures
fetch compared with curl and wget
These tools overlap, but they are not interchangeable in every script. If a command depends on a particular option, document the required operating system and verify the local manual page.
Practical workflow: an interrupted release download
- Change to a directory with enough free space.
- Choose a clear output name with
-o. - Start the download from the publisher's HTTPS URL.
- If the connection stops, use resume mode only if the local implementation and server support it.
- Check the resulting size and file type.
- Calculate and compare the published checksum.
- Only then extract or use the archive.
mkdir -p downloads
cd downloads
fetch -r -o large-image.iso https://example.org/images/large-image.iso
ls -lh large-image.iso
file large-image.iso
sha256 large-image.isoProxy environment variables
Exam-relevant notes
fetch [options] URLdownloads a remote resource; it does not synchronize directories.- The current working directory affects the location of relative output names.
- An explicit output option makes scripts more predictable.
- A redirect is a response directing the client to another URL, not necessarily a failed download.
- HTTP status codes and the shell exit status answer different questions: the server status describes the HTTP response, while the exit status reports the command's result to the shell.
- Resume mode depends on both client support and server range support.
- Certificate validation should be fixed through clock, CA, hostname, or proxy configuration rather than disabled.
- A successful transfer is not proof of authenticity; verify the file and its SHA-256 checksum when a trusted digest is available.
- Use
man fetchbecause implementations and flags differ across operating systems.
For a related command-line download topic, see Fetch: Downloading Files from the Command Line.