vSAN Proactive Support with VMware Skyline

We are excited to announce the introduction of vSAN proactive support in VMware Skyline. Customers utilizing vSAN will receive proactive findings that identify potential issues for vSAN. In order to enable vSAN proactive support in Skyline, customers must upgrade to Skyline Collector 1.4. The Release Notes for Skyline Collector 1.4 can be found here.

To upgrade to Skyline Collector 1.4, either enable Auto-Upgrade within Skyline Collector 1.3, or login to the Skyline Collector VAMI and check for and install updates.

VMware Skyline documentation is now available on VMware Docs. Visit VMware Docs for instructions to enable Auto-Upgrade or manually upgrade your Skyline Collector.

vSAN Proactive Findings

The following are two of the vSAN proactive findings being delivering in Skyline.

Proactive Finding: Internal vSAN Memory (LSOM) or SSD Device Congestion

For this finding, Skyline utilizes the VC_EVENT endpoint to check for the vSAN LSOM congestion state exceeded message. If the congestion state exceeded message is observed by Skyline, this proactive finding will appear within your next Operational Summary Report (OSR).

If a virtual machine, or virtual machines, perform a high number of write operations, write buffers could fill up on flash cache devices. These buffers must be de-staged to magnetic disks in hybrid configurations. De-staging can only be performed at a rate at which the magnetic disks in a hybrid configuration can handle.

vSAN Proactive Finding: vSAN memory or SSD congestion reached threshold limit

VMware Knowledge Base article 2071384 provides you with information to remediate this issue.

Proactive Finding: H730 Mini-Controller running Firmware may cause vSAN Host Instability

This finding determines if the H730 RAID Controller is being utilized for vSAN. If the H730 RAID Controller is being utilized, Skyline then checks the version of firmware. If the firmware version is, this proactive finding is added to the customer’s Operational Summary Report (OSR).

VMware Knowledge Base (KB) article 52783 is provided within the proactive finding. This KB article provides additional symptoms related to this potential issue, such as specific entries in log files. This issue is resolved in firmware version Additionally, the link to the PowerEdge RAID Controller H730 support page is provided within the KB article.

If your vSAN environment matches the characteristics detailed in either of these proactive findings, you can expect to see these vSAN proactive findings in your OSR. These are just two examples of the vSAN proactive findings now available within Skyline. There are more vSAN proactive findings available today, and we will continue to add new vSAN proactive findings in the future.

vSAN proactive support with Skyline does not replace vSAN Support Insight. We are continuing to evaluate how to utilize the information available within both Skyline and Support Insight to deliver a single, holistic support experience for our customers.

Additional vSphere Proactive Findings

While we’re on the topic of new proactive findings added to Skyline, I wanted to highlight two vSphere proactive findings added a short-time ago.

Proactive Finding: Network becomes Unavailable with ntg3 Driver on ESXi 6.5

This finding determines if the native ntg3 driver is being used on a passthrough Broadcom Gigabit Ethernet Adapter in your vSphere environment. If it is, it is possible that the network on the ESXi 6.5 host becomes unavailable. To resolve this potential issue, the recommendation is to run the ntg3 driver in legacy mode. As with all of the proactive findings delivered within the Operational Summary Report (OSR), a Knowledge Base (KB) article is provided for your reference. For this particular finding, KB article 2150889 is provided.

Proactive Finding: ESXi 6.5 host fails with PSOD when IPv6 disabled

This finding is interesting because there is no resolution at this time. If you disable IPv6 within ESXi 6.5, the ESXi host could fail with a purple screen of death (PSOD). To workaround this issue, enable IPv6. Additional information, including log excerpts, are provided within KB 2150794.

You’re probably thinking, why would Skyline alert me of a potential issue that does not have a resolution. First, one goal of Skyline is to notify customers of potential issues. In most instances, a resolution is provided for the potential issue. For this finding, a workaround is provided. However, once a resolution is available, customers will be notified that the KB article has been updated, and that a resolution is now available for this potential issue. By better understanding how our customer’s are utilizing our products and services, we can better provide relevant, personalized information and recommendations.

From VMware Docs, how to enable or disable IPv6 on an ESXi host.


We continue to expand the features and functionality available within Skyline. vSAN is the latest product to be added, joining VMware vSphere and VMware NSX. We will be announcing additional features and functionality for Skyline in less than two weeks at VMworld US. If you’re attending VMworld US and want to learn more about proactive support with Skyline, keep an eye out for our upcoming blog post that will provide details for where to find Skyline content at VMworld.

Additionally, follow @VMwareCSE (VMware Customer Support Experience) on Twitter for additional information related to proactive support with Skyline, along with Communities, Intelligent Search, My VMware and other VMware customer support programs.

The post vSAN Proactive Support with VMware Skyline appeared first on Support Insider.

Posted in Knowledge Base, skyline, vsan | Comments

What happens if all hosts in a vSphere HA cluster are isolated?

Advertise here with BSA

I received this question through twitter today from Markus who was going through the vSphere 6.7 Clustering Deep Dive. And it is fairly straightforward: what happens when all hosts are isolated in a cluster, will the isolation response be triggered?

I wrote about this a long long time ago, but it doesn’t hurt to re-iterate this. Before triggering the isolation response HA will actually verify the state of the rest of the cluster. Does anyone own the datastore on which the VMs that are impacted by this isolation run? If the answer is no, the ownership of a datastore is dropped during the election, then HA will not trigger the isolation response. I will try to update the book when I have time to include that, hopefully, that means a new version of the ebook will be pushed out to all owners automatically.

The post What happens if all hosts in a vSphere HA cluster are isolated? appeared first on Yellow Bricks.

Posted in 6.7, BC-DR, ha, Server, VMware, vsphere ha | Comments

New KB articles published for week ending 11th August 2018

VMware NSX for vSphere

Publishing Distributed Firewall rules fails with error “Unmatched rules found in the configuration”
Date Published: 2018-08-09

Environments running vRNI 3.8 with NSX may experience NSX Controller memory issues and interruption to networking
Date Published: 2018-08-09

VMware ESXi

SNMPD crashes frequently after upgrading to ESXi 6.7
Date Published: 2018-08-08

VMware Horizon

Instant Clone Pool creation might fail
Date Published: 2018-08-07

VMware vCenter Server

Custom permissions are not working as expected after upgrading/migrating from vCenter Server 6.0 to 6.5
Date Published: 2018-08-11

VMware vCloud Availability for vCloud Director

“Internal server error” message when running the vcav org-vdc enable-replication command on the vCloud Availability Installer
Date Published: 2018-08-06

Understanding the replication disk space for an Outgoing Replication to vCloud Availability for vCloud Director
Date Published: 2018-08-06

How to determine the current storage used by replication instances via the vCloud Availability for vCloud Director API
Date Published: 2018-08-08

“Unable to delete principal with username com.vmware.vr-…” error when deploying vCloud Availability for vCloud Director
Date Published: 2018-08-09

VMware vRealize Log Insight

Exporting Hosts Return Incomplete Results vRealize Log Insight 4.x
Date Published: 2018-08-08

Update or Delete on Shared Extracted fields take a long time in vRealize Log Insight 3.x and up
Date Published: 2018-08-09

VMware vRealize Operations for Horizon

The Total Logon Time metric displays “No Data” in vRealize Operations for Horizon 6.x
Date Published: 2018-08-09

The post New KB articles published for week ending 11th August 2018 appeared first on Support Insider.

Posted in KB Digest, Knowledge Base | Comments

Top vSAN product articles for July 2018

  1.  Component metadata health check fails with invalid state error
  2. “Host cannot communicate with all other nodes in vSAN enabled cluster” error
  3. vCenter Server 6.0 Update 2 displays on non-vSAN enabled ESXi hosts displays the message: Retrieve a ticket to register the vSAN VASA Provider
  4. Status of TLSv1.1/1.2 Enablement and TLSv1.0 Disablement across VMware products
  5. The ramdisk ‘vsantraces’ is full
  6. Cannot view or add vSAN Storage Providers in the vSphere Web Client
  7. Certification of Dell PERC H730 and FD332-PERC Controllers with vSAN 6.x
  8. “An error occurred while consolidating disks: msg.disklib.NOSPACE” in vSAN
  9. ESXi host in a vSAN cluster reports a warning alarm: Host vendor provider registration
  10. vSAN host may encounter a purple diagnostic screen during performance statistics updates
  11. Creating or editing a virtual machine Storage Policy to correct a missing vSAN VASA provider fails
  12. “Unexpected status code: 503” error in vSAN health check plug-in
  13. “Unable to register file system <UUID> for APD timeout notifications: Inappropriate ioctl for device” message in vSAN
  14. vSAN disk components are marked ABSENT after enabling CBT
  15. vSAN 6.2 hybrid disk group performance degradation
  16. Local Disks goes into Permanent Device Loss (PDL) state with inbox drivers for mpt2sas
  17. VMware vSAN datastore not listed as a matching datastore when creating a storage based policy
  18. vSAN CLOMD daemon may fail when trying to repair objects with 0 byte components
  19. “Failed to create directory” error when running vSAN 5.5/6.x
  20. “Virtual SAN Disk Balance” warning alarm during vSAN health check

The post Top vSAN product articles for July 2018 appeared first on Support Insider.

Posted in KB Digest, Top 20 | Comments

Top NSX for vSphere product articles for July 2018

  1. Virtual machine in ESXi is unresponsive with a non-paged pool memory leak
  2. VMs running on ESXi 5.5 with vShield endpoint activated fails during snapshot operations
  3. Performing vMotion or powering on a virtual machine being protected by vShield Endpoint fails
  4. When using VMware vShield App Firewall, virtual machines fail to connect to the vSwitch/vDS/network with the error: Failed to connect virtual device Ethernet0
  5. “The pending transaction requires xxx MB free space” error when installing VIBs
  6. Installing VMware vShield App fails with the error: Previous installation of host services encountered an error
  7. ESXi 5.5 and 6.0 hosts fail with a PSOD: VMCIEventDelayedDispatchCB@com
  8. Systems running MOVE Agentless 3.0 on ESXi 5.5 suffer performance issues or become unresponsive
  9. ESX/ESXi 4.1 Update 2 host with vShield Endpoint 1.0 installed fails with a purple diagnostic screen mentioning VFileFilterReconnectWork
  10. Degraded Windows network file copy performance after full ESXi 5 VMware Tools installation
  11. Status of TLSv1.1/1.2 Enablement and TLSv1.0 Disablement across VMware products
  12. Windows virtual machines using the vShield Endpoint TDI Manager or NSX Network Introspection Driver (vnetflt.sys) driver fails with a blue diagnostic screen
  13. Network connectivity issues after upgrade in NSX/VCNS environment
  14. Slow VMs after upgrading VMware Tools in NSX and vCloud Networking and Security
  15. vShield Manager appliance system disk is full in VMware vCloud Networking and Security 5.5.x
  16. NSX Controller disconnected or isolates intermittently
  17. Duplicate VTEPs in ESXi hosts after rebooting vCenter Server
  18. NSX is unavailable from the vSphere Web Client Plug-in
  19. Guest Introspection memory usage spikes to 90+% or you see the error: “Lost communication with ESX module” in NSX-V 6.2.x and 6.3.x
  20. After upgrading to VMware ESXi 5.5 Patch Release ESXi550-201504002, virtual machines using VMware NSX for vSphere 6.x or Cisco Nexus 1000v are unable to communicate across hosts

The post Top NSX for vSphere product articles for July 2018 appeared first on Support Insider.

Posted in KB Digest, Top 20 | Comments

Top vSphere product articles for July 2018

  1. “The transaction log for database ‘VIM_VCDB’ is full” error on a Microsoft SQL DB server
  2. ESXi 5.5 Update 3b and later hosts are not manageable after an upgrade
  3. “Host IPMI system event log status” alarm in vCenter Server
  4. Determining where growth is occurring in the vCenter Server database
  5. ESXi host disconnects intermittently from vCenter Server
  6. Troubleshooting the vCenter Server service
  7. Investigating the health of a vCenter Server database
  8. Required VMware vCenter Converter ports
  9. The vpxd process becomes unresponsive after upgrading to vCenter Server 5.5/6.0
  10. Storage vMotion migration fails with the error: The method is disabled by ‘SYMC-INCR dd-mm-yyyy hh:mm
  11. Troubleshooting checklist for VMware Converter
  12. vCenter Server 5.5 fails to start after reboot with the error: Unable to create SSO facade: Invalid response code: 404 Not Found
  13. Unable to log in to the root account of vCenter Server Appliance
  14. Upgrading to VMware Tools 5.1 reports the message: Error in the RPC receive loop: RpcIn: Unable to send
  15. ESXi hosts fail to mount VMFS5 volumes that are formatted with ATS-only capabilities
  16. After making a change or restarting vCenter Single Sign-On server system, vCenter Server 5.1.x fails to start
  17. “could not open/create change tracking files” error when powering on VM
  18. Failed to verify the SSL certificate” after upgrading to vCenter Server 5.5 U1 or later
  19. vCenter Server 5.x fails to start with the error: Failed to add LDAP entry
  20. Enabling the Update Manager plug-in fails with the error: database unavailable or has network problems

The post Top vSphere product articles for July 2018 appeared first on Support Insider.

Posted in KB Digest, Top 20 | Comments

Must read white paper: Persistent Memory performance with vSphere 6.7

Advertise here with BSA

Today I noticed this whitepaper titled: Persistent Memory Performance on vSphere 6.7. An intriguing topic for sure as it is something “relatively new and something I haven’t encountered too much in the field. Yes, I talk about Persistent Memory, aka NVDIMMs, in my talks usually but then it typically relates to vSAN. I have not seen too many publications from VMware on this topic, so I figured I would share this publication with you:

The paper starts with a brief intro and then explains the different modes in which PMEM can be used, either as a “disk” (vPMEMDisk) or surfaced up to the guest OS as an NVDIMM (vPMEM). With the latter option, there’s also the ability to have some form of application awareness, which is referred to as the 3rd mode (vPMEM-aware).

I am not going to copy and paste the findings, as the paper has a lot of interesting data and you should go through it. One thing I found most interesting is the huge decrease in latency. Anyway, read the paper and get familiar with persistent memory / NVDIMMs, as this technology will start changing the way we design HCI platforms in the future and cater for low latency / high throughput applications in traditional environments.

The post Must read white paper: Persistent Memory performance with vSphere 6.7 appeared first on Yellow Bricks.

Posted in 6.7, memory, nvdimm, performance, persistent memory, Server, vSphere | Comments

What happened to MaxCostPerEsx41DS? It doesn’t seem to work in vSphere 6.x?

Advertise here with BSA

Today I received a question which also caught me by surprise, someone updated from vSphere 5.0 and he noticed that when doing an SDRS Maintenance Mode that the setting MaxCostPerEsx41DS did not work. This setting actually limits the number of active SvMotions on a single datastore. You can imagine that this can be desired when you are “limited” in terms of performance. I was a bit surprised as I had not heard that these settings changed at all. Also, a quick search on internal pages and externally did not deliver any results. After a discussion with some support folks and some more digging, I found a reference to a naming change. Not surprising I guess, but as per vSphere 6.0 the setting is called MaxCostPerEsx6xDS. So if you would like to limit the number of SvMotion’s active at the same time, please note the change in names.

For more background on this topic I would like to refer to Frank’s excellent blog on this topic here.

The post What happened to MaxCostPerEsx41DS? It doesn’t seem to work in vSphere 6.x? appeared first on Yellow Bricks.

Posted in 6.0, 6.5, 6.7, limit, MaxCostPerEsx41DS, MaxCostPerEsx6xDS, Storage, SvMotion, vSphere | Comments

Back in the saddle again

You may have noticed I’ve been fairly quiet the last few months both here and on social media. The reason is I have been re-locating from Colorado to Texas (Houston) and it has been quite a lot of work moving and settling in. I had to sell my house in Colorado which took a ton … Continue reading »

[[ This is a content summary only. Visit my website for full links, other content, and more! ]]
Posted in News | Comments

You asked for it: vSphere 6.7 Clustering Deep Dive ebook, now available!

Advertise here with BSA

We knew when we released the paper version of the book that many would yell: What about an e-book? Although sales numbers of the Host Deep Dive and previous Clustering Deep Dive books have shown that by far most people prefer a printed copy, we decided to go ahead and create an ebook as well. It is not as simple unfortunately as simply uploading a PDF or an MS Word file. We had to spend evenings reformatting the book in an e-book authoring tool, compile it, review it, fix issues, compile again etc. Nevertheless, it is done!

So what we did is we just uploaded it to Amazon, and we made it available for 14,95 USD, or whatever that roughly converts to in your local currency in your local store. We also noticed there was a bundling option, so as soon as the ebook and the paper copy are linked you can buy the ebook alongside the paper copy for only 2,99 USD. (Linking the book may still take a couple of days, we’ve initiated the process with Amazon and are waiting for them to complete it.)

You wanted it, so go out and pick it up, right before the weekend!


The post You asked for it: vSphere 6.7 Clustering Deep Dive ebook, now available! appeared first on Yellow Bricks.

Posted in 6.7, cluster, clustering deep dive, deep dive, Server, VMware, vSphere, vsphere 6.7 clustering deep dive ebook | Comments