VMware ESXi and vSphere Cluster Management
Install vCenter Server on Windows Using the Custom Install Method
Learn how to install the Windows-based vCenter Server component with Custom Install after deploying Single Sign-On and Inventory Service.
vCenter Server is the centralized management platform for a VMware vSphere environment. It provides a single place to manage ESXi hosts, virtual machines, inventory objects, permissions, authentication, and related vSphere services.
An ESXi host is the hypervisor that runs virtual machines. vCenter Server manages one or more ESXi hosts as a coordinated environment. The vSphere management client is the user interface used to connect to vCenter Server; it is not the management service itself.
Installation Approaches
Custom Install deploys vSphere components separately and in an administrator-selected order. In this workflow, the Windows vCenter Server service is installed after its required supporting services.
- Custom Install: Installs components individually, giving administrators control over deployment order, service locations, databases, and configuration choices.
- Simple Install: Installs required vSphere components together through a more consolidated workflow.
- vCenter Server Appliance: An appliance-based alternative that runs vCenter Server in a packaged virtual appliance rather than as a Windows installation.
The Custom Install method is useful when the deployment requires separate component placement or when the supporting services have already been configured independently.
Prerequisites and Deployment Order
Install and verify the following components before starting the vCenter Server installation:
- A supported Windows Server system dedicated or allocated for the vCenter Server service.
- vCenter Single Sign-On (SSO). SSO provides authentication and the Lookup Service used during registration.
- vCenter Inventory Service. This service supports inventory-related operations required by this workflow.
Ensure that the Windows server can resolve and reach the SSO server, Inventory Service, selected database server, and every ESXi host that will be managed. Verify DNS in both directions where required by the environment, and confirm that host and network firewalls permit the selected service ports.
Information to Collect
| Requirement | Why It Is Needed | Information to Confirm |
|---|---|---|
| Supported Windows Server | Hosts the Windows vCenter Server service | Operating system support, hostname, FQDN, resources, and installation volume |
| SSO installed | Provides authentication and service registration | SSO host, administrator account, password, and Lookup Service endpoint |
| Inventory Service installed | Supports inventory operations | FQDN, port, service status, and reachability |
| SSO administrator credentials | Registers vCenter Server with SSO | Usually administrator@vsphere.local, unless the SSO domain was customized |
| Lookup Service URL | Allows vCenter Server to locate vSphere services | https://<SSO-host-FQDN-or-IP>:7444/lookupservice/sdk |
| Database plan | Stores vCenter Server data | Bundled SQL Server Express or an external supported database |
| DNS and network reachability | Allows component communication | FQDN resolution, firewall rules, ports, and routing |
| License decision | Determines whether the installation starts licensed or in evaluation mode | License key or evaluation approval |
Launch the Custom Install Workflow
- Mount or insert the vCenter Server installation media.
- Start the launcher by running
autorun.exe. - In the Custom Install section, choose vCenter Server. Do not select SSO, Inventory Service, or another vSphere component when installing this component.
- Select the installation language.
- Continue through the welcome screen and read the license agreement.
- Accept the agreement to proceed.
Configure the Installer Decisions
| Installer Screen | Available Choice | Recommended Use Case | Impact |
|---|---|---|---|
| License key | Enter a key or leave the field blank | Use the production key when available; use evaluation mode for testing or initial setup | A blank key enables a time-limited evaluation and requires licensing before it expires |
| Database option | New SQL Server 2008 Express instance or external database | Express for a small lab; external database for larger environments | Determines capacity, administration, resilience, and growth options |
| Service account | Local System or a dedicated Windows account | Local System for a simple deployment; dedicated account when organizational controls require it | Controls the Windows identity used by the vCenter Server service |
| Linked Mode | First vCenter Server instance or join an existing configuration | Choose first instance for a new linked deployment | Determines whether this server establishes or joins the linked configuration |
| Port settings | Default ports or approved custom ports | Retain defaults unless a conflict or security policy requires a change | Ports must be permitted through local and network firewalls |
| JVM memory size | A profile matching inventory size | Select the profile appropriate for expected hosts, virtual machines, and growth | Insufficient JVM memory can reduce performance in larger inventories |
| Installation folder | Default or selected destination | Use a planned volume with sufficient free space | Determines where program files and related components are placed |
Licensing and Evaluation Mode
Enter a vCenter Server license key when one is available. If the key is left blank, the installer creates a time-limited evaluation installation. Evaluation mode is useful for a lab or for validating the deployment, but it is not a permanent licensing strategy.
Record the evaluation expiration date and apply an appropriate vCenter Server license before the evaluation period ends. A license for ESXi hosts does not automatically license vCenter Server.
Choose a Database
The database stores vCenter Server configuration and inventory information. The installer provides two broad choices.
Bundled Microsoft SQL Server Express
The installer can create a new bundled Microsoft SQL Server 2008 Express instance. This option is appropriate for a small lab or limited environment with a modest number of managed objects.
Express reduces initial database administration, but it has capacity and operational limitations. Consider expected growth, backup requirements, recovery objectives, and the number of hosts and virtual machines before selecting it.
External Database
An external database is separately managed and may be hosted on a database server maintained by a database administration team. Larger environments should use an external, supported database platform, such as Oracle in the applicable installer workflow.
External databases provide more control over capacity planning, backup, high availability, maintenance, and operational resilience. Confirm that the database platform and version are supported by the intended vCenter Server release.
| Database Option | Suitable Environment | Advantages | Limitations |
|---|---|---|---|
| New SQL Server Express instance | Small lab or limited deployment | Simple setup and minimal separate database administration | Limited scale and fewer enterprise operational options |
| External supported database | Growing or enterprise deployment | Greater control over scale, backup, resilience, and administration | Requires database planning, credentials, permissions, connectivity, and ongoing maintenance |
Select the vCenter Server Service Account
The installer asks which Windows identity should run the vCenter Server service. The default option is the built-in Local System account.
Local System is convenient for a simple deployment. An organization may instead select a dedicated service account to support separation of duties, password policies, auditing, or centralized service-account management. Before choosing a dedicated account, verify that it has the required local rights, access to dependencies, and a lifecycle process for password changes and account disablement.
Do not choose a service account only because it exists. Confirm that the account can run the service and that its permissions match the organization's security policy.
Configure Linked Mode
Linked Mode allows multiple vCenter Server instances to share selected information, making it possible to manage related vCenter environments through a coordinated configuration.
- Choose the option identifying this server as the first vCenter Server instance when no compatible linked vCenter Server exists yet.
- Choose the option to join an existing linked configuration only when another compatible vCenter Server is already deployed and its topology, credentials, and connectivity are confirmed.
For a first deployment, selecting the join-existing option is incorrect because there is no existing linked configuration to join.
Review Ports and Select JVM Memory
Review the vCenter Server port configuration. Retain the defaults unless a port conflict, security policy, or documented network design requires a change. Any changed port must be documented and allowed through the Windows firewall and applicable network firewalls.
JVM memory is the memory allocated to the Java Virtual Machine used by relevant vCenter Server processes. Select the size that matches the anticipated inventory and environment growth. Under-sizing the JVM can cause poor service performance as the number of hosts, virtual machines, and inventory objects increases.
Register vCenter Server with Single Sign-On
- Enter the SSO administrator account and password.
- Enter the SSO Lookup Service URL using the required HTTPS pattern:
https://<SSO-host-FQDN-or-IP>:7444/lookupservice/sdkThe commonly used internal SSO domain is vsphere.local. Unless the SSO configuration was customized, the administrator identity is:
administrator@vsphere.localThe Lookup Service endpoint identifies where vCenter Server registers and discovers vSphere services. The default HTTPS port in this installer workflow is 7444.
When the installer displays the SSO certificate fingerprint, compare it with the expected fingerprint obtained from a trusted source. Accept the certificate only after confirming that the host identity and fingerprint are correct.
Connect to Inventory Service
After SSO information is supplied, the installer normally populates the Inventory Service endpoint. Confirm that the value identifies the deployed Inventory Service instance, including its fully qualified domain name and port.
Do not assume that a populated value is correct. Verify the FQDN, port, DNS resolution, service status, and network reachability. The Inventory Service must have been installed and available before this vCenter Server component is configured.
| Service | Endpoint Format | Credential or Trust Requirement | Validation Step |
|---|---|---|---|
| SSO Lookup Service | https://<SSO-host-FQDN-or-IP>:7444/lookupservice/sdk | SSO administrator credentials and verified certificate fingerprint | Resolve the host and confirm HTTPS reachability to port 7444 |
| Inventory Service | HTTPS endpoint using the Inventory Service FQDN and configured port | Correct service registration and endpoint information | Confirm the service is installed, running, reachable, and associated with the intended deployment |
| vCenter Server | FQDN and configured vCenter Server ports | Valid vCenter authentication and trusted service registration | Connect with the management client and verify inventory access |
Select the Installation Path and Complete Setup
- Choose the destination installation folder on a planned volume.
- Review the license, database, service account, Linked Mode, port, JVM memory, SSO, Inventory Service, and path settings.
- Start the installation.
- Wait for the installer to finish configuring the service and its dependencies.
- Open Windows service management and verify that the vCenter Server service is installed and starts successfully.
Document the final configuration, including the Windows host FQDN, SSO endpoint, Inventory Service FQDN and port, database selection, service account, vCenter ports, JVM profile, installation directory, and license or evaluation status.
Post-Installation Validation
- Connect to vCenter Server with the appropriate vSphere management client.
- Authenticate using the expected SSO or configured administrative identity.
- Confirm that SSO registration succeeds and that the vCenter Server service can communicate with the Lookup Service.
- Confirm Inventory Service connectivity.
- Confirm database connectivity and verify that the vCenter inventory loads.
- Add the intended ESXi hosts to the vCenter inventory.
- Verify that hosts appear as connected and that basic inventory operations work.
- After host registration, proceed with virtual machine inventory and management tasks.
Practical Deployment Examples
Small Lab
Install SSO and Inventory Service first, then use Custom Install to deploy vCenter Server on Windows. Choose the bundled SQL Server Express option, retain standard ports, and use evaluation mode if a production key is not available. After validation, add a small number of ESXi hosts.
Growing Enterprise Environment
Use an externally managed supported database, select JVM sizing for the planned inventory, and make an explicit service-account decision. Before installation, confirm DNS, firewall access, certificate identity, service endpoints, database permissions, backup planning, and resource capacity.
First Member of a Linked Deployment
When this is the first vCenter Server in a linked deployment, select the first-instance option. Do not select join-existing mode unless another compatible vCenter Server is already available and the linked topology has been planned.
Troubleshooting
Lookup Service Connection Fails
Likely causes include an incorrect SSO hostname, IP address, URL path, or port; DNS failure; a blocked firewall connection; an unavailable SSO service; or incorrect credentials.
- Verify the URL format and the
7444port. - Confirm DNS resolution from the Windows vCenter Server host.
- Check local and network firewall rules.
- Verify SSO service status and administrator credentials.
Unexpected Certificate Fingerprint
The installer may be reaching a different host, the SSO certificate may have changed, or DNS may point to the wrong endpoint. Compare the displayed fingerprint with the expected certificate fingerprint and verify the SSO host identity before accepting it.
Inventory Service Endpoint Is Missing or Invalid
Confirm that Inventory Service was installed first and is running. Then verify its FQDN, port, DNS resolution, and network connectivity. Ensure that the endpoint belongs to the intended Inventory Service instance.
Database Setup or Connection Fails
Reconfirm whether the deployment should use SQL Server Express or an external database. For an external database, validate server availability, instance details, credentials, permissions, network access, and support for the intended vCenter Server version.
vCenter Server Does Not Start
Check Windows service status and relevant installation or service logs. Review the service account, database connectivity, SSO and Inventory Service availability, configured ports, available resources, and selected JVM memory profile.
Linked Mode Join Fails
Confirm that the intended topology is correct. Use the first-instance option for an initial deployment. When joining an existing configuration, verify that a compatible vCenter Server exists and that connectivity and authentication between the instances work.
Exam-Relevant Notes
- Custom Install deploys vSphere components separately; Simple Install deploys required components together.
- SSO and Inventory Service must be installed and available before this Windows vCenter Server component.
administrator@vsphere.localis the usual default SSO administrator identity when the default SSO domain is used.- The Lookup Service URL uses HTTPS and, in this workflow, commonly uses port
7444. - Verify the SSO certificate fingerprint before trusting the connection.
- SQL Server Express is intended for smaller environments; larger deployments should use an external supported database.
- Select the first-instance Linked Mode option for the first vCenter Server in a new linked configuration.
- Changed ports must be permitted through host and network firewalls.
For a repeatable deployment, record every selected value before beginning and validate SSO, Inventory Service, database connectivity, service startup, client access, and ESXi host registration after installation.