VMware ESXi and vSphere Cluster Management

VMware VMFS: Virtual Machine File System and Shared Datastores in ESXi

Learn how VMware VMFS datastores organize virtual machine files, enable shared ESXi storage, support vMotion and HA, and require careful capacity and path management.

VMFS, or Virtual Machine File System, is VMware's clustered datastore file system for ESXi. It is designed to organize virtual-machine files on block storage and allow multiple ESXi hosts to access the same datastore in a coordinated way.

VMFS is not the file system inside a virtual machine. ESXi uses VMFS to store files such as VMDKs and virtual machine configuration files. The guest operating system sees a virtual disk and formats that disk with its own file system, such as NTFS, ext4, or XFS.

This distinction is central to understanding VMware storage. A shared VMFS datastore can help vSphere move or restart virtual machines across hosts, but VMFS alone does not provide every cluster feature. Networking, host compatibility, storage connectivity, licensing, and cluster configuration are also important.

VMFS and shared datastore reference

What VMFS Does

In a physical server, an operating system usually formats a disk with a file system and stores files directly on it. In an ESXi environment, storage is virtualized into layers. ESXi discovers block-storage devices, creates or mounts datastores, and presents virtual disks to virtual machines.

VMFS provides the file-system layer used by a VMFS datastore. It tracks files, directories, free space, virtual machine metadata, and access coordination on a block-storage volume. The underlying storage may be disks in a storage array, a Fibre Channel SAN, an iSCSI SAN, or storage attached directly to a host.

• Physical storage: Disks, flash devices, or an array provide the actual capacity.
• LUN: A logical block-storage unit is presented to ESXi.
• VMFS volume: The LUN or device is formatted with VMFS.
• Datastore: ESXi exposes the VMFS volume as a named storage container.
• VM directory: A directory holds the files belonging to a virtual machine.
• Virtual machine files: Configuration, disk, log, snapshot, swap, and metadata files are stored in the directory.

VMFS, VMDK, and the Guest File System

A VMDK is a virtual disk file or virtual disk representation used by a virtual machine. A VMDK stored in VMFS is connected to the VM's virtual SCSI controller. The guest sees the resulting device as a virtual SCSI disk, not as a VMFS file.

The guest operating system then partitions, formats, and mounts that virtual disk using its native file system. For example, a Windows guest may create an NTFS volume, while a Linux guest may use ext4 or XFS. The guest administrator normally cannot see or manage the VMFS structures that contain the VMDK.

Typical Files in a VM Directory

• VMX: Virtual machine configuration file.
• VMDK: Virtual disk descriptor and data files, depending on the disk format.
• LOG: Runtime and diagnostic log files.
• VMEM or swap-related files: Files associated with virtual machine memory state or host swap behavior.
• Snapshot files: Descriptor and delta files created by snapshots.
• NVRAM: Nonvolatile firmware state for virtual machines using supported virtual firmware.
• Other metadata: Lock, inventory, suspend, and management-related files may also be present.

Do not delete files merely because their names are unfamiliar. Use the vSphere or ESXi management workflow and verify ownership before removing anything.

VMFS Datastore Architecture

A datastore is a logical ESXi storage container. It may be backed by one device or by multiple extents, depending on the supported design and release. The datastore name is an ESXi management label; it does not replace the underlying device identity, LUN mapping, or VMFS metadata.

When a VM is created on a datastore, ESXi creates a directory for that VM and places its configuration and virtual disk files there. A virtual disk is therefore represented through several layers:

1. The storage system presents a LUN.
2. ESXi discovers the LUN through a storage adapter and one or more paths.
3. The LUN contains a VMFS volume.
4. ESXi mounts the VMFS volume as a datastore.
5. The datastore contains the VM directory and VMDK files.
6. The VM's virtual SCSI controller presents the VMDK to the guest.
7. The guest formats and mounts the virtual disk with its own file system.

Shared VMFS and Concurrent Host Access

A shared datastore is visible to multiple ESXi hosts at the same time. Each host must have appropriate access to the same underlying LUN and must identify the VMFS volume consistently.

VMFS coordinates metadata operations so that multiple hosts can work with the same datastore without independently corrupting its directory and allocation structures. Conceptually, this includes distributed metadata coordination and file locking. Locks help prevent unsafe simultaneous operations on files, such as two hosts both attempting to own or modify the same VM's execution state.

Shared access is essential in a cluster because a destination host must be able to open a VM's files when the VM moves or is restarted there. However, shared access does not normally mean that two hosts run the same VM simultaneously. Under normal operation, one ESXi host owns and runs a VM. Other hosts can access the datastore and can become the VM's destination after a controlled migration or a failure event.

Storage Types Used with VMFS

VMFS is used with SCSI-based block storage. The block device can be provided through local disks, Fibre Channel, or iSCSI.

A local VMFS datastore can be useful, but it is not equivalent to a shared VMFS datastore. In a two-host cluster, both hosts must see the same datastore and the same intended LUN. Storage-side masking, zoning, host groups, LUN IDs, and ESXi discovery should be consistent. A datastore visible on only one host cannot provide shared-storage migration or failover for that host's VMs.

VMFS and vSphere Availability and Mobility

Do not confuse HA restart with vMotion. HA responds to a host failure by powering on or restarting affected VMs on a surviving host. vMotion moves a running VM in a planned operation and is intended to avoid a service interruption. Both can depend on shared storage, but they solve different problems.

VMFS is therefore an important storage dependency, not a complete implementation of HA, DRS, vMotion, or FT. Always validate the requirements for the deployed vSphere version and the chosen feature.

Two-Host Example

Suppose ESXi-01 and ESXi-02 are connected to the same iSCSI LUN. The LUN is formatted as a VMFS datastore named cluster-ds01, and both hosts see it with healthy paths. A VM stored there can be migrated from ESXi-01 to ESXi-02 with vMotion when all other checks pass. If ESXi-01 fails, vSphere HA can restart the VM on ESXi-02 if the datastore and required networks remain available.

Capacity Management and Provisioning

Datastore capacity is different from the configured size of virtual disks. Three values must be considered:

• Provisioned capacity: The logical capacity configured for VMDKs. This may exceed current physical consumption for thin disks.
• Consumed datastore space: Physical space currently occupied by VMDK data, snapshots, swap, logs, templates, ISOs, and other files.
• Available physical capacity: Space that remains on the VMFS datastore and its backing storage.

Thin provisioning allows a VMDK to advertise a large capacity while consuming less datastore space initially. For example, three thin VMDKs configured at 2 TB each may initially consume only a fraction of their combined 6 TB logical capacity. As guest data is written, the VMDKs grow and consume datastore space.

This is a form of overcommitment: more logical disk capacity is provisioned than the datastore can immediately provide physically. Overcommitment can improve utilization, but it creates an operational risk. Guest growth, snapshot delta files, and simultaneous workload changes can exhaust the datastore.

Datastore Expansion

When the storage administrator increases the size of a backing LUN, ESXi can often discover the larger device after a storage rescan. The administrator can then expand the existing VMFS datastore rather than formatting a new file system. With supported storage, release, and change procedures, capacity can be added while resident VMs continue operating.

Expansion is not the same as reformatting. Reformatting creates a new file system and can destroy existing data. Confirm the device identity, existing datastore, backups, and storage-side change before selecting any format operation.

Use alerts and headroom policies for free space, snapshot growth, thin-disk growth, and storage-array capacity. Do not wait for a datastore to reach 0 percent free space. A full datastore can prevent new writes, snapshot consolidation, VM power operations, logging, or other management tasks.

Raw Device Mapping

Raw Device Mapping (RDM) provides a mapping from a virtual machine to a raw LUN. The VM uses a small mapping file stored in a datastore, while the bulk of the data resides on the underlying raw LUN.

At a high level, RDM has two compatibility modes:

• Virtual compatibility mode: Presents a more virtualized view and generally permits more VM-like storage behavior.
• Physical compatibility mode: Passes more SCSI characteristics through to the guest and may be considered when an application requires lower-level SCSI behavior.

RDM may be considered for workloads with specific raw-device, SCSI-identity, or array-integration requirements. VMDKs are the normal choice for many workloads because they are easier to provision, move, snapshot where supported, protect with common backup workflows, and manage as virtual disks.

Before choosing RDM, check current platform support, feature limitations, backup compatibility, portability, replication behavior, and the operational procedures for mapping and recovering the raw LUN. An RDM mapping file is not a backup of the raw device's data.

VMFS Versions, Limits, and Compatibility

VMFS version affects supported features, interoperability, upgrade paths, and limits. Identify the VMFS version of each datastore and confirm that all hosts intended to access it support that version.

VMFS-5 is an earlier VMFS generation commonly encountered in existing environments. One characteristic of VMFS-5 is its 1 MB block-size design. That historical detail should not be used as a universal statement about current datastore limits or behavior.

Maximum datastore size, file size, extent count, virtual-disk limits, snapshot behavior, and feature support vary by ESXi and vSphere release. Older VMFS-5 limits should not be presented as current limits for every deployment. Consult the VMware documentation and compatibility information for the exact ESXi and vSphere version in use.

For upgrades or migrations between VMFS generations, verify whether the target release supports the desired operation, whether hosts must be upgraded first, and whether migration to a new datastore is safer. Test the procedure and maintain backups before changing production storage.

Creating a VMFS Datastore

Prerequisites

• Storage connectivity is configured on the ESXi host.
• The intended LUN is discovered and has the correct presentation and masking.
• Fibre Channel zoning or iSCSI target and initiator configuration is correct.
• Multipathing is configured and paths are healthy where redundancy is required.
• You have permissions to create and manage datastores.
• The device has been verified as unused or as the intended device for a documented operation.

Client Workflow

1. Open the target host or cluster storage view in the ESXi Host Client or vSphere Client.
2. Choose to create a new datastore.
3. Select VMFS as the datastore type.
4. Select the eligible storage device or LUN.
5. Assign a clear, environment-appropriate datastore name.
6. Select the available VMFS version and capacity options supported by the target hosts.
7. Review the warning about formatting and confirm the device identity.
8. Complete creation, then verify that the datastore is mounted and visible to every intended cluster host.

After a new LUN is presented, or after a storage administrator expands a device, rescan the relevant storage adapters. Confirm device identity, path state, VMFS version, capacity, and free space before placing production VMs on the datastore.

esxcli storage filesystem list
esxcli storage core device list
esxcli storage core path list
esxcli storage core adapter rescan --all
esxcli storage vmfs extent list

esxcli storage filesystem list shows mounted file systems and datastore capacity information. esxcli storage core device list helps inspect detected devices. esxcli storage core path list helps review path visibility and state. The rescan command requests storage rediscovery. esxcli storage vmfs extent list associates VMFS datastores with backing device extents.

Mounting, Unmounting, and Deleting

Mounting makes an existing, recognized VMFS volume available to an ESXi host. Before mounting, verify that the device is the intended datastore and that it is not an unsafe duplicate or an independently used production volume.

Before unmounting a datastore, confirm that no powered-on VM, template, ISO, swap file, snapshot, system service, or management operation depends on it. Unmounting is not a substitute for removing storage mappings safely.

Deleting a datastore or formatting its device is potentially destructive. Confirm backups, VM placement, replication state, storage-side mappings, and change approval. Never format a LUN merely because it is not immediately visible on one host.

Operational Design Practices

• Present the intended LUN to every cluster host that must access it.
• Use consistent LUN identities and presentation across hosts.
• Design redundant storage paths and monitor path health.
• Validate Fibre Channel zoning, host groups, array masking, iSCSI sessions, and network dependencies.
• Use meaningful datastore names that identify purpose, location, or service tier without hiding the underlying LUN identity.
• Separate workloads when performance, protection, or failure-domain requirements differ.
• Maintain capacity headroom for normal growth, snapshots, swap files, maintenance, and recovery operations.
• Use change control and tested backups before expansion, resignaturing, unmounting, deletion, or storage snapshot operations.

Troubleshooting VMFS

Duplicate Signatures and Resignaturing

A storage snapshot or clone can contain a copy of the original VMFS metadata and signature. If ESXi detects both the original and duplicate, it may treat the copy cautiously to avoid confusing two volumes as the same datastore.

Resignaturing assigns a new datastore signature to the duplicate volume. The correct procedure depends on whether the goal is disaster recovery, testing, replication, or temporary inspection. First identify the production source and the intended copy. Use supported procedures and prevent simultaneous unsafe access to duplicate and production volumes.

Capacity and Path Failures

When a datastore becomes inaccessible, determine whether the issue affects one host or all hosts. A one-host issue suggests local adapter, driver, zoning, session, or path configuration problems. A cluster-wide issue may indicate an array, fabric, iSCSI network, LUN mapping, or storage-system problem.

For capacity alarms, inspect snapshots and thin-disk growth before assuming the datastore needs reformatting. For inaccessible files, check storage paths and VM ownership before investigating locks. File-lock investigation should remain within supported diagnostic boundaries; do not delete lock files or modify VMFS metadata as a first response.

Exam-Relevant Notes

• VMFS is the ESXi datastore file system; NTFS, ext4, and XFS are examples of guest file systems.
• A VMDK is stored by ESXi but presented to the guest through a virtual SCSI controller.
• Multiple hosts can access a shared VMFS datastore through coordinated metadata and locking.
• Shared visibility is required for many storage-based cluster operations, but VMFS alone does not configure vMotion, HA, DRS, or FT.
• HA restarts a VM after host failure; vMotion live-migrates a running VM during a planned operation.
• Thin provisioning can create overcommitment and out-of-space risk.
• RDM maps a VM to a raw LUN through a mapping file; VMDKs are the normal choice for many workloads.
• VMFS-5 used a 1 MB block-size design, but its historical limits are not universal current limits.
• Always verify limits and compatibility for the exact deployed vSphere release.
• Never format, resignature, unmount, or delete a storage device without confirming identity, dependencies, backups, and the intended change.