Linux online course

Join Two Files by a Common Field with the Linux join Command

Learn how to combine sorted Linux text files by a shared field using join, including custom fields, delimiters, output layouts, and unmatched records.

The Linux join command combines records from two text files when a selected field contains the same value in both files. This shared value is the join field, also called the key.

A record is one logical input line. A field is one portion of that line, separated from other fields by whitespace or a delimiter. For example, the record 101 Ada contains two fields: 101 and Ada.

join is different from cat, which simply puts files one after another, and paste, which pairs line 1 with line 1, line 2 with line 2, and so on. join pairs records by key, even when matching records are not at the same line number.

How join combines records

By default, join compares field 1 in the first file with field 1 in the second file. For each matching key, its normal output contains the key followed by the remaining fields from the first matching line and then the remaining fields from the second matching line.

Input structure and field numbering

By default, fields are separated by blanks or other whitespace. Field numbering begins at 1, not 0. A stable identifier such as an employee number, account ID, or product code is a good join key.

Basic join syntax

The basic form is:

join file1 file2

Suppose first_names.txt contains:

101 Ada
102 Linus
103 Grace

Suppose last_names.txt contains:

101 Lovelace
102 Torvalds
103 Hopper

Because the common numeric identifier is field 1 in both files, run:

join first_names.txt last_names.txt

The result is:

101 Ada Lovelace
102 Linus Torvalds
103 Grace Hopper

The key appears once in the output. The first file contributes Ada, Linus, or Grace, and the second file contributes the surname.

Sort both inputs before joining

join expects each input to be sorted by its own join field. The first file must be ordered by the field selected from file 1, and the second must be ordered by the field selected from file 2. Unsorted input can cause missing matches, surprising output, or an ordering diagnostic.

For whitespace-separated files joined on field 1, create sorted copies with:

sort -k1,1 first_names.txt > first_names.sorted
sort -k1,1 last_names.txt > last_names.sorted
join first_names.sorted last_names.sorted

Use the same field and delimiter rules for sort that you will use for join. If identifiers are numeric and lexical order is inappropriate, consider numeric sorting, for example sort -k1,1n. The ordering used by both tools should be consistent.

You can avoid permanent intermediate files with process substitution in Bash:

join <(sort -k1,1 first_names.txt) <(sort -k1,1 last_names.txt)

Process substitution is a Bash feature. For portability, or when you need to inspect the sorted data, use temporary sorted files instead. Quote filenames containing spaces:

join "first names.sorted" "last names.sorted"

Selecting different join fields

Use -1 FIELD to choose the join field in the first input and -2 FIELD to choose it in the second input. The two files do not need to store the same logical key in the same field position.

Consider these slash-delimited files:

people.txt:
Ada/Lovelace/101
Grace/Hopper/103
Linus/Torvalds/102

departments.txt:
Engineering/101
Systems/102
Research/103

In people.txt, the ID is field 3. In departments.txt, the ID is field 2. Sort each file by its selected key:

sort -t '/' -k3,3 people.txt > people.sorted
sort -t '/' -k2,2 departments.txt > departments.sorted

Then join field 3 from the first file to field 2 from the second:

join -t '/' -1 3 -2 2 people.sorted departments.sorted

The result is:

101/Ada/Lovelace/Engineering
102/Linus/Torvalds/Systems
103/Grace/Hopper/Research

Using a custom delimiter

The delimiter is the character or whitespace that separates fields. Without options, join treats blank-separated text as fields. For comma-, colon-, tab-, pipe-, or slash-separated data, use -t CHAR.

For slash-separated records, quote the delimiter:

join -t '/' -1 3 -2 2 people.sorted departments.sorted

The delimiter setting applies to both input files in that invocation and also controls the separators in normal output. For a comma-separated pair, the form would be:

join -t ',' -1 2 -2 1 first.csv second.csv

This simple text processing does not provide full CSV parsing. Quoted CSV fields containing commas, escaped quotes, or embedded newlines require a CSV-aware tool such as Python, csvkit, or an appropriate awk workflow.

Understanding and controlling output

Normally, output consists of the join key, the non-key fields from file 1, and the non-key fields from file 2. Use -o LIST when a report needs a precise column order.

Output field references use this form:

• 0 means the join field.
• 1.1 means field 1 from the first input.
• 2.1 means field 1 from the second input.

For the slash-delimited example, this command explicitly requests the ID, first file's name fields, and second file's department:

join -t '/' -1 3 -2 2 -o '0,1.1,1.2,2.1' people.sorted departments.sorted > employee_departments.txt

The shell redirection operator > writes the result to a new file, replacing that file if it already exists. Use >> only when appending is intentional.

Join options used in this lesson

Handling unmatched records

Standard join produces only records whose keys occur in both inputs. This is an inner-style join. Records present in only one file are omitted unless you request them.

Use -a 1 to retain unpaired records from file 1, or -a 2 to retain unpaired records from file 2. Use both options to retain unmatched records from both inputs:

join -t '/' -1 3 -2 2 -a 1 -a 2 people.sorted departments.sorted

When missing fields need a visible value, add -e:

join -t '/' -1 3 -2 2 -a 1 -e 'N/A' people.sorted departments.sorted

Use -v 1 to output only records whose key exists in file 1 but not file 2. Use -v 2 for the reverse:

join -v 1 file1 file2
join -v 2 file1 file2

Practical field-mapping example

The key mapping for the slash-separated example is:

people.txt        Ada / Lovelace / 101
fields             1      2        3  <- key

departments.txt   Engineering / 101
fields             1           2  <- key

Because the key positions differ, the complete command needs both field-selection options and the delimiter option:

join -t '/' -1 3 -2 2 people.sorted departments.sorted

Troubleshooting join commands

No output appears even though identifiers look identical

The files may not be sorted by the selected fields, or the command may be comparing the wrong fields. Sort each input by its actual key and verify the -1 and -2 values.

Values match incorrectly or an ordering error appears

Check that the sort delimiter and key positions match the join command. For numeric identifiers, use a consistent numeric sort strategy when needed, such as sort -k1,1n.

A slash- or comma-separated line is treated as one field

join defaults to blank-separated fields. Add the correct delimiter, such as join -t '/' ... or join -t ',' ....

Expected records disappear

Ordinary join returns matches only. Add -a 1, -a 2, or both to retain unpaired records. Use -v when you want only the unmatched records.

A header row appears in the data or affects sorting

join does not provide a universal header-aware mode. Handle headers separately, join only the data rows, and add an appropriate output header afterward.

Fields containing delimiters produce incorrect columns

join is intended for simple delimited text and does not implement full CSV quote parsing. Use a CSV-aware tool or parse the data with a language and library that understands the format.

Safe command-line practices

• Quote delimiter characters when shell interpretation could be confusing: -t '/|' is not a valid single-character delimiter, but -t '|' should be quoted for clarity.
• Quote filenames containing spaces, for example join "employee data.sorted" departments.sorted.
• Use process substitution only in shells that support it, such as Bash; otherwise create temporary sorted files.
• Keep the sorting key, delimiter, and join-field numbers synchronized between preparation and the final command.

Summary

• join merges records by a common key rather than by line number.
• Both inputs must be sorted on their selected join fields.
• The default key is field 1 in each file; field numbering starts at 1.
• Use -1 and -2 when the key occupies different field positions.
• Use -t for non-whitespace delimiters and -o for a custom output layout.
• Use -a to retain unpaired records, -v to show only them, and -e for missing-value placeholders.

For related shell fundamentals, see Bourne Again Shell Bash, showing the full path of shell commands, and the Linux command-line topics.