Insufficient configured resources to satisfy the desired vSphere HA failover level on the cluster

Advertise here with BSA

I was talking to the HA team this week, specifically about the upcoming HA book. One thing they mentioned is that people still seem to struggle with the concept of admission control. This is the reason I wrote a whole chapter on it years ago, yet there still seems to be a lot of confusion. One thing that is not clear to people is the percentage calculations. We have some customers with VMs with extremely large reservations, in that case instead of using the “slot policy” they typically switch to “percentage based policy”. Simply as the percentage based policy is a lot more flexible.

However, recently we have had some customers that hit the following error message:

Insufficient configured resources to satisfy the desired vSphere HA failover level on the cluster

This error message, in the case of these particular situations (yes there was a bug as well, read this article on that), set the percentage lower than what would equal a full host. In other words, in a 4 host environment, a single host would equal 25%. In some cases, customers would set the percentage to a value lower than 25%. I am personally not sure why anyone would do this as it contradicts the whole essence of admission control. Nevertheless, it happens.

This message indicates that you may not have sufficient resources, in the case of a host failure, to restart all the VMs. This of course is the result of the percentage being set lower than the value that would equal a single host. Note though, this does not stop you from powering on new VMs. You will only be stopped from powering on new VMs when you exceed the available unreserved resources.

So if you are seeing this error message, please verify the configured percentage if you set it manually. Ensure that at a minimum it equals the largest host in the cluster.

** back to finalizing the book **


The post Insufficient configured resources to satisfy the desired vSphere HA failover level on the cluster appeared first on Yellow Bricks.

Posted in 6.5, 6.7, admission control, BC-DR, ha, Server, VMware, vSphere | Comments

vSphere 6.5 and limits, do they still apply to SvMotion?

Advertise here with BSA

I had a question this week and I thought I wrote about this before but apparently, I did not. Hopefully by now most of you the I/O scheduler has changed over the past couple of years. Within ESXi, we moved to a new scheduler, which often is referred to as mClock. This new scheduler, and a new version of SIOC (storage i/o control) also resulted in some behavioral changes. Some which may be obvious, others which are not. I want to explicitly point out two things which I have discussed in the past which changed with vSphere 6.5 (and 6.7 as such) and both are around the use of limits.

First of all: Limits and SvMotion. In the past when a limit was applied to a VM, this would also artificially limit SvMotion. As of vSphere 6.5 (may apply to 6.0 as well but I have not verified this) this is no longer the case. Starting vSphere 6.0 the I/O scheduler creates a queue for every file on the file system (VMFS), this to avoid for instance a VM stalling other types (metadata) of IO. The queues are called SchedQ and briefly described by Cormac Hogan here. Of course, there’s a lot more to it than Cormac or I discuss here, but I am not sure how much I can share so I am not going to go there. Either way, if you were used to limits being applied to SvMotion as well you are warned… this is no longer the case.

Secondly, the normalization of I/Os changed with limits. In the past when a limit was applied IOs were normalized at 32KB, meaning that a 64KB I/O would count as 2 I/Os and a 4KB I/O would count as 1. This was confusing for a lot of people and as of vSphere 6.5 this is no longer the case. When you place a limit of 100 IOPS the VMDKs will be limited at 100 IOPS, regardless of the I/O size. This, by the way, was documented here on storagehub, not sure though how many people realized this.

The post vSphere 6.5 and limits, do they still apply to SvMotion? appeared first on Yellow Bricks.

Posted in 6.5, limit, mclock, scheduler, Server, sioc, Storage, Various, vSphere | Comments

How to start vRealize Orchestrator Workflows from vCloud Director – Part II

vRealize Orchestrator is an important automation component in every vCloud Director environment. The first article of the series showed the different scenarios in which Orchestrator workflow can be used to extend vCloud Director processes:

This article explains in detail how workflows can be started by cloud users, operators or automatically in a vCloud Director context.

What is needed to start a vRealize Orchestrator workflow?

To start a workflow, the vRealize Orchestrator engine needs three pieces of information:

  1. The workflow id. In the workflow library of an orchestrator server there are typically hundreds of workflows, sorted in a tree structure. It is possible (though not recommended) to have multiple workflows with the same name. All workflows however are identified by a unique id, which is shown in the “General” tab of the client.
  2. An authenticated user. No matter what mechanism is used to start the workflow, they all require a user being authenticated against the vRO service. vRealize Orchestrator can be configured to authenticate against the Identity Management Service of a vSphere Platform Service Controller or a vRealize Automation IDM.
  3. The workflow’s input parameter values. The workflow developer can specify input parameters needed within the workflow logic. These parameters contain of a type and a description, the values have to be provided by whoever starts the workflow.

Start a workflow from vRealize Orchestrator Client

vRealize Orchestrator (vRO) comes with a stand-alone java-based client application that can be installed on an operator’s system, or started directly from browser via Java WebStart technology. This client is the main tool to develop the workflows in first place, but it also allows to run and monitor workflows.

When workflows are being started from the vRealize Orchestrator client, the user logged-in to the client has to provide values for the input parameters. All input presentation logic defined for the workflow is processed before the workflow starts.

Starting a workflow from the vRO client allows the workflow developer to run the workflow in debug mode. In this mode the workflow can be interrupted by breakpoints, and manually proceeded element-by-element. In debug mode it’s also possible to watch and even manipulate the values of workflow attributes during runtime.

In a vCloud Director environment the workflows are mostly part of an end-to-end automation, so the option to start workflows from the vRO client is only used during workflow development, or for a few special workflows targeted to a limited group of administrators who have direct access to the vRO client.

Most of the workflow executions are being triggered automatically by one of the following mechanisms.

Starting Workflows from via the REST API

vRealize Orchestrator provides a comprehensive REST API that allows external systems to automate all sorts of tasks. It is possible for example to access the inventory of installed plugins, or import or export workflow packages. And of course it is possible to browse the workflow library and run workflows via the REST API.

The REST API is documented on the vRO server itself with a Swagger based documentation environment, you can access the documentation by browsing to https://your.vroserver:8281/vco/api/docs.

vRealize Orchestrator REST API Documentation

An http POST request to the url https://your.vroserver:8182/vco/api/workflows/<workflow-id>/executions starts the workflow. The request body has to contain the values for the input parameters.

If the call is successful, the API returns with a status 202 ACCEPTED response that contains the workflow execution id in the Location Header.
If there is an error, for instance missing input parameters or a wrong format for them, the API returns with a status 4xx response.

The easiest way to figure out the proper syntax for input parameters is to look at a past workflow execution of the same workflow, by accessing the URL https://your.vroserver:8281/vco/api/workflows/<workflow-id>/executions/<execution-id>, and check the <input-parameters> section of the response there.

Calling vRealize Orchestrator workflows via the REST API is also used under the hood for the workflows that are being published into the vCloud Director Service Library. Here vCloud Director automatically creates a form for the input parameters, and uses the credentials defined by the administrator during registration of the vRO server in vCloud Director to start the workflow.

Scheduling Workflows

Workflow Scheduler in vRealize Orchestrator

For some use-cases, like maintenance or reporting tasks, it is useful to run workflows automatically at certain times. The vRealize Orchestrator server comes with a built-in workflow scheduler, that allows you to schedule workflow executions. When defining the workflow schedule, you also have to specify the input parameters and the user that is used to start the workflow when it’s time.

The workflow scheduler is accessible via the vRO client, or via the REST API with the url https://your.vroserver:8281/vco/api/tasks.

Trigger Workflows via AMQP messages

To use orchestrator workflows as part of the vCloud Director extensibility framework it is needed to start workflow using AMQP messages. As stated in the first article of the series, vCloud Director Blocking Tasks and Service Extensibility both use AMQP as protocol to communicate with external systems. vCloud Director sends AMQP messages to the AMQP broker with routing keys dependent on the situation, the AMQP broker put them into the queues according the configured bindings.

AMQP Subscription in vRO Inventory

In vRealize Orchestrator you then can configure the AMQP Plugin to subscribe to certain queues of the AMQP broker. With that the subscriptions appear in the AMQP inventory of vRO, and can be used in the policy mechanism of vRO to trigger workflows.

A Policy can be used in vRO to either start a workflow, or to run some javascript logic based on events. These events, like a SNMP trap or an AMQP message are provided by the corresponding plugin. In the definition of the policy, you select the AMQP:Subscription as policy element, and then the “onMessage” event. In the policy details you then can either specify the javascript logic or select the workflow that will be executed whenever a message is received on the queue of the subscription.

It is even possible to have some javascript logic that then calls a workflow programmatically. This allows some more flexibility to dynamically select the workflow based on content of the AMQP message, but also makes the architecture more complex and harder to troubleshoot.

Policy Definition to start a workflow by an AMQP Message

Run Workflows within Workflows

One powerful concept of vRealize Orchestrator is the possibility to modularize and reuse existing workflows and actions. So it is absolutely possible to run workflows within a workflow. There are several ways to do this:

Synchronous call to a workflow element

By using the “Workflow Element” in the palette of the schema editor you can add an existing workflow into the logic of your workflow. Alternatively you can browse the workflow in the “All Workflows” section and drag&drop it onto the schema. When adding the workflow, you have to bind the input parameters of the workflow to call to attributes of your workflow, same for the output parameters of the called workflow.

Workflow Elements in Schema Palette

In this scenario the logic of the included workflow will be executed as part of the calling workflow, within the same execution context. This creates no overhead, the calling workflow will continue when the logic of the included workflow has been completed.

Asynchronous start of a workflow

It is also possible to call other workflows asynchronously, by using the “Asynchronous Workflow”. The called workflow will be started in its own execution context, so a new workflow token will be generated for the called workflow. In the calling workflow you get the id of the created workflow token, so you can add some logic to monitor its status, if needed.

Nested workflow element

The “Nested workflows” element from the palette allows you to select one or more workflows, which will be started in their own execution context. However, in contrary to the asynchronous call here the calling workflow will only continue when all the called workflows are completed.

All the options mentioned above require to select the to-be-called workflow at development time.

Run a workflow from a javascript element

If you want to select the workflow to call dynamically at runtime, you need to start it from a scriptable task element using JavaScript. First you get the object representing the workflow to call, either by specifying it in an attribute or input parameter, or by finding it in the workflow library or identifying by its id. Next you create a properties object that contains the input parameters for the workflow. Then you can use the workflowObject.execute(inputParameters) method to start the workflow. This will return a workflow token object that represents the started workflow, which then can be processed for instance to monitor the status of the workflow execution (more details about this in the next article of the series).

The code below shows an example of this logic. It gets the workflow object by id (in this example a workflow from the vCloud Director Plugin Library “Clone a vApp”), creates the needed input parameters in a properties object and starts the workflow.

var cloneVAppWorkflow = Server.getWorkflowWithId("CE8080808080808080808080808080808080808001266312685778e677578588f");
var workflowInputs = new Properties();
workflowInputs.put("vapp", sourceVApp);
workflowInputs.put("vdc", destinationOrgVdc);
workflowInputs.put("name", nameOfClonedVApp);
workflowInputs.put("description", "cloned by vRO workflow");
workflowInputs.put("deleteSource", false);
workflowInputs.put("powerOn", true);
workflowInputs.put("linkedClone", true);

var cloneToken = cloneVAppWorkflow.execute(workflowInputs);

Because vRealize Orchestrator can run multiple workflows (even multiple instances of the same workflow) in parallel, calling workflows asynchronously is a nice strategy to parallelize tasks. But be aware that each creation of a workflow execution creates load on the vRO server itself, and of course it might create a bottleneck on the endpoint the workflow calls.

Using the Multi-node plugin for vRealize Orchestrator it is even possible to start a workflow on a remote vRO host. More on this in a later article of the series.


vRealize Orchestrator provides different mechanisms to start workflows manually and automatically. Manual start of workflow through the vRO Client is mainly used during workflow development. Publishing workflows into the vCloud Director Service Library uses the vRO REST API in background, while vCloud Director Blocking Tasks can trigger workflows using AMQP. The scheduler is useful for repeating workflows, e.g. for reporting or maintenance processes. For a workflow developer there are different ways available to call existing workflows synchronously and asynchronously.

No matter which mechanism is used to start the workflow, always proper authentication is needed, and the workflow input parameters have to be provided. The result of a workflow execution is a workflow token, which contains the runtime information like status, start time and error messages of the specific workflow run.

In next article of the series will dig deeper into workflow executions, and how to access and monitor their information.

The articles of this series:

The post How to start vRealize Orchestrator Workflows from vCloud Director – Part II appeared first on VMware Cloud Provider Blog.

Posted in Cloud Services | Comments

Top 20 vSAN Articles June 2018

  1. Component metadata health check fails with invalid state error
  2. Best practices for vSAN implementations using Dell PERC H730 or FD332-PERC storage controllers
  3. Status of TLSv1.1/1.2 Enablement and TLSv1.0 Disablement across VMware products
  4. Replacing SSD disk on a vSAN cluster
  5. The ramdisk ‘vsantraces’ is full
  6. “The ramdisk “vsantraces” is full” error reports in vSAN logging
  7. On vSAN 6.6 and 6.6.1 cluster, Testing health check fails randomly
  8. VMware vSAN 6.2 on disk upgrade fails due to CBT enabled virtual disks
  9. vSAN CLOMD daemon may fail when trying to repair objects with 0 byte components
  10. Diskgroups fail to mount due to heap exhaustion
  11. “Host cannot communicate with all other nodes in vSAN enabled cluster” error
  12. vSAN Unassociated Object Handling – A Methodology
  13. “Virtual SAN Disk Balance” warning alarm during vSAN health check
  14. Certification of Dell PERC H730 and FD332-PERC Controllers with vSAN 6.x
  15. vCenter Server 6.0 Update 2 displays on non-vSAN enabled ESXi hosts displays the message: Retrieve a ticket to register the vSAN VASA Provider
  16. Initializing vSAN during boot takes a longer time
  17. Virtual Machine with more than 64GB memory fails to Storage vMotion to vSAN cluster
  18. Placing host into maintenance mode with ‘Full data migration’ option fails
  19. vSAN performance diagnostics reports: “vSAN is experiencing congestion in one or more disk group(s)”
  20. Cannot view or add vSAN Storage Providers in the vSphere Web Client

The post Top 20 vSAN Articles June 2018 appeared first on Support Insider.

Posted in KB Digest, Top 20 | Comments

Top 20 NSX Articles June 2018

  1. Virtual machine in ESXi is unresponsive with a non-paged pool memory leak
  2. VMs running on ESXi 5.5 with vShield endpoint activated fails during snapshot operations
  3. Performing vMotion or powering on a virtual machine being protected by vShield Endpoint fails
  4. When using VMware vShield App Firewall, virtual machines fail to connect to the vSwitch/vDS/network with the error: Failed to connect virtual device Ethernet0
  5. “The pending transaction requires xxx MB free space” error when installing VIBs
  6. Installing VMware vShield App fails with the error: Previous installation of host services encountered an error
  7. ESXi 5.5 and 6.0 hosts fail with a PSOD: VMCIEventDelayedDispatchCB@com
  8. Systems running MOVE Agentless 3.0 on ESXi 5.5 suffer performance issues or become unresponsive
  9. ESX/ESXi 4.1 Update 2 host with vShield Endpoint 1.0 installed fails with a purple diagnostic screen mentioning VFileFilterReconnectWork
  10. Degraded Windows network file copy performance after full ESXi 5 VMware Tools installation
  11. Status of TLSv1.1/1.2 Enablement and TLSv1.0 Disablement across VMware products
  12. Windows virtual machines using the vShield Endpoint TDI Manager or NSX Network Introspection Driver (vnetflt.sys) driver fails with a blue diagnostic screen
  13. Network connectivity issues after upgrade in NSX/VCNS environment
  14. Slow VMs after upgrading VMware Tools in NSX and vCloud Networking and Security
  15. vShield Manager appliance system disk is full in VMware vCloud Networking and Security 5.5.x
  16. NSX Controller disconnected or isolates intermittently
  17. Duplicate VTEPs in ESXi hosts after rebooting vCenter Server
  18. NSX is unavailable from the vSphere Web Client Plug-in
  19. After upgrading to VMware ESXi 5.5 Patch Release ESXi550-201504002, virtual machines using VMware NSX for vSphere 6.x or Cisco Nexus 1000v are unable to communicate across hosts
  20. Guest Introspection memory usage spikes to 90+% or you see the error: “Lost communication with ESX module” in NSX-V 6.2.x and 6.3.x

The post Top 20 NSX Articles June 2018 appeared first on Support Insider.

Posted in KB Digest, Top 20 | Comments

Top 20 vSphere Articles June 2018

  1. “The transaction log for database ‘VIM_VCDB’ is full” error on a Microsoft SQL DB server
  2. ESXi 5.5 Update 3b and later hosts are not manageable after an upgrade
  3. “Host IPMI system event log status” alarm in vCenter Server
  4. Determining where growth is occurring in the vCenter Server database
  5. ESXi host disconnects intermittently from vCenter Server
  6. Troubleshooting the vCenter Server service
  7. Investigating the health of a vCenter Server database
  8. Required VMware vCenter Converter ports
  9. The vpxd process becomes unresponsive after upgrading to vCenter Server 5.5/6.0
  10. Storage vMotion migration fails with the error: The method is disabled by ‘SYMC-INCR dd-mm-yyyy hh:mm’
  11. Troubleshooting checklist for VMware Converter
  12. vCenter Server 5.5 fails to start after reboot with the error: Unable to create SSO facade: Invalid response code: 404 Not Found
  13. Unable to log in to the root account of vCenter Server Appliance
  14. Upgrading to VMware Tools 5.1 reports the message: Error in the RPC receive loop: RpcIn: Unable to send
  15. ESXi hosts fail to mount VMFS5 volumes that are formatted with ATS-only capabilities
  16. After making a change or restarting vCenter Single Sign-On server system, vCenter Server 5.1.x fails to start
  17. could not open/create change tracking files? error when powering on VM
  18. Failed to verify the SSL certificate” after upgrading to vCenter Server 5.5 U1 or later
  19. vCenter Server 5.x fails to start with the error: Failed to add LDAP entry
  20. Enabling the Update Manager plug-in fails with the error: database unavailable or has network problems

The post Top 20 vSphere Articles June 2018 appeared first on Support Insider.

Posted in KB Digest, Top 20 | Comments

New KB articles published for week ending 8th July 2018

VMware NSX-T Data Center

PSC 6.7 HA configuration for Resource vCenter Server with NSX-T Load Balancer
Date Published:7/6/2018

VMware NSX for vSphere

PSC 6.7 HA configuration for Management vCenter Server (Green Field deployment) with NSX-T Load Balancer
Date Published:7/7/2018

NSX Manager’s dashboard reports “NSX component NSX Replicator is Unable to retrieve status”
Date Published:7/2/2018

VMware SDDC Manager

Failed or non-responsive NSX Controller VMs can safely be destroyed and redeployed via Web Client with no harm to VMware Cloud Foundation
Date Published:7/3/2018

The VMware Cloud Foundation 2.3.2 health check reports the Uplink port connection status as RED when a custom L3 uplink configuration is used.
Date Published:7/3/2018

Cumulus 3.3 or later causes Network Issues due to the Reserved VLAN Range on the Management Switch in VMware Cloud Foundation 2.3.x
Date Published:7/3/2018

VMware Cloud Foundation components certificates replacement
Date Published:7/4/2018

VMware vCenter Lifecycle Manager

Update check with a default repository type displays the same build information as the previous repository URL check when upgrading from vRealize Suite Lifecycle Manager 1.0
Date Published:7/5/2018

vROPS cluster is not accessible after reverting the snapshot of vROPS taken by LCM
Date Published:7/5/2018

VMware vCloud Availability for Cloud-to-Cloud DR

Protecting vApps to a Storage Policy backed by a datastore cluster fails in vCloud Availability for Cloud-to-Cloud DR 1.0
Date Published:7/6/2018

VMware vRealize Operations Manager

Manual Configuration of liagent.ini on Remote Collector nodes in vRealize Operations Manager 6.7
Date Published:7/7/2018

VMware vRealize Suite Lifecycle Manager

Prerequisites for upgrading vRSLCM from version 1.2 to 1.3
Date Published:7/3/2018

VMware vSphere ESXi

Host Loosing NFS datastore when renamed existing NFS and presented new Share with the old name of an existing share
Date Published:7/5/2018

VMware vSphere Integrated Containers

VIC 1.4.x Could not find any VIC appliance VM
Date Published:7/5/2018

The post New KB articles published for week ending 8th July 2018 appeared first on Support Insider.

Posted in KB Digest, Knowledge Base | Comments

Opvizor Performance Analyzer for vSAN

Advertise here with BSA

At a VMUG a couple of months ago I bumped into my old friend Dennis Zimmer. Dennis told me that he was working on something cool for vSAN but couldn’t reveal what it was just yet. Last week I had a call with Dennis about what that thing was. Dennis is the CEO for Opvizor, and some of you may recall the different tooling that Opvizor has produced over the years, of which the Health Analyzer was probably the most famous one back then. I’ve used it in the past on various occasions and I had various customers using it. During the briefing, Dennis explained to me that Opvizor started focussing on performance monitoring and analytics a while ago as the health analyzer market was overly crowded and had the issue that is was a one-off business (checks once in a while instead of daily use). On top of that, many products now come with some form of health analysis included. (See vSAN for instance.) I have to agree with Dennis, so this pivot towards Performance Monitoring makes much sense to me.

Dennis explained to me how they are seeing more and more customer demand for vSAN performance monitoring especially combined with VMware ESXi, VM and App data. Although vCenter has various metrics, and there’s VROps, he told me that Opvizor has many customers who need more than vCenter or vROPS standard has to offer today and don’t own VROps advanced. This is where Opvizor Performance Analyzer comes in to play and that is why today Opvizor announced they are including vSAN specific dashboards. Now, this isn’t just for vSAN of course. Opvizor Performance Analyzer includes not just vSAN but also vSphere and various other parts of the stack. When talking with Dennis one thing became clear, Opvizor is taking a different approach than most other solutions. Where most focus on simplifying, hiding, and aggregating, the focus for Opvizor is on providing as much relevant detail as possible to fulfill the needs of beginner and professional.

So how does it work? Opvizor provides a virtual appliance. You simply deploy it in your environment and connect it to vCenter and you are ready to go. The appliance collects data every 5 minutes (but 20 seconds intervals of these 5 minutes) and has a retention of up to 5 years. As I said, the focus is on infrastructure statistics and performance analytics and as such Opvizor delivers all the data you ever need.

It doesn’t just provide you with all the info you will ever need. It will also allow you to overlay different metrics, which makes performance troubleshooting a lot easier, and will allow you to correlate and pinpoint particular problems. Opvizor comes with dashboards for various aspects, here are the ones included in the upcoming release for vSAN:

  • Capacity and Balance
  • Storage Diskgroup Stats
  • VM View
  • Physical disk latency breakdown
  • Cache Diskgroup stats
  • vSAN Monitor

Now I said this is the expert´s troubleshooting tool, but Opvizor Performance Analyzer also provided in-depth information about what each metric is / means and provides starter dashboards for beginners. You can simply click on the “i” in the top left corner of the widget and you get all the info about that particular widget.

When you do know what you are looking for you can click, hover, and zoom when needed. Hover over the specific section in the graph and the point in time values of the metrics will pop up. In the case below I was drilling down on a VM in the vSAN cluster and looking at write latency in specific. As you can see we have 3 objects and in particular 2 disks and a “vm name space”.

And this is just a random example, there are many metrics to look at and many different widgets and overviews. Just to give you an idea, here are some of the metrics you can find in the UI:

  • Latency (for all different components of the stack)
  • IOPs (for all different components of the stack)
  • Bandwidth (for all different components of the stack)
  • Congestion (for all different components of the stack)
  • Outstanding I/O (for all different components of the stack)
  • Read Cache Hit rate (for all different components of the stack)\
  • ESXi vSAN host disk usage
  • ESXi vSAN host cpu usage
  • Number of Components
  • Disk Usage
  • Cache Usage

And there;s much more, too many to list in this blog. And again, not just vSAN, but there are many dashboards to chose from. If you don’t have a performance monitoring solution yet and you are evaluating solutions like SolarWinds, Turbunomics and others make sure to add Opvizor to that list. One thing I have to say, I spotted a couple of things that I liked to see changed, and I think within 24hrs the Opvizor guys managed to incorporate the feedback. That was a crazy fast turnaround, good to see how receptive they are.

Oh, one more thing I found in the interface, it is these dashboards that deal with things like NUMA. But also things like the Top 10 VMs in terms of IOPS. Both very useful, especially when doing deep performance troubleshooting and optimizing.

I hope that gives you a sense of what they can do. There’s a fully functional 30-day trial, check it out if you want to find out more about Performance Analyzer or simply just want to play around with it. Opvizor announced this brand new version on their own blog here, make sure to give that a read as well!

The post Opvizor Performance Analyzer for vSAN appeared first on Yellow Bricks.

Posted in Management & Automation, monitoring, opvizor, performance, Server, Software Defined, startup, Storage, Virtual SAN, vsan, vSphere | Comments

Extend and Automate vCloud Director using vRealize Orchestrator

VMware vRealize Orchestrator (vRO) is a powerful workflow engine that can be used in versatile ways to automate tasks and complex processes. Its open APIs and plugins for many protocols and systems also make it a flexible integration tool. vCloud Director (vCD) provides different APIs and mechanisms that can be used to extend the service offerings, integrate it with external systems, and automate configuration and consumption tasks. The complete Extensibility Framework of vCloud Director is layed out in this article:

There are different scenarios in which vRealize Orchestrator can be used in combination with vCloud Director. This article shows these scenarios, and explains the APIs, tools and techniques used for the integration. It is the first of a series of articles, which will explain more details about how to use, operate and develop scalable and reliable vRealize Orchestrator workflows in a vCloud Director context.

Automating vCloud Director using the vCD Plugin for vRealize Orchestrator

vCloud Director provides a comprehensive REST API, on both, provider and tenant level. This API can be used to automate tasks that otherwise would have to be done manually using the web-based UI of vCloud Director. The vCloud Director Plugin for vRealize Orchestrator builds a wrapper around this REST API, and exposes inventory objects and their attributes and methods into the workflows. It also comes with a huge library of workflows for individual tasks, which either can be used as they are, or as building blocks for more complex custom workflows. The images below show the vCloud Director inventory in vRealize Orchestrator and the workflow library.

vCloud Director Workflow LibraryvCloud Director Inventory

Workflows can be built to automate complex processes, like an end-to-end onboarding of a new tenant as provider. It is also possible to use vRealize Orchestrator and its own REST API or other plugins to integrate and consume vCloud Director services automatically. For example, a workflow to automatically provision a new vApp from the catalog could be triggered from a Continous Integration / Continous Delivery pipeline.

The vCD Plugin for vRealize Orchestrator takes care of the authentication against vCD, and the mapping of inventory objects to the REST API. So workflows in vRO can be used to simplify the integration with external systems, to add additional logic, or integrate other external systems into the overall process.

vCloud Director triggers vRO Workflows during Blocking Tasks and Notifications

A powerful extensiblity mechanism of vCloud Director is it’s ability to send and receive AMQP messages, as part of blocking tasks or notifications. Blocking Tasks let vCloud Director pause a process at a certain state, send a message to the AMQP bus, and wait for a response message before continuing. An example could be an additional approval step before deploying a new vApp. Notifications message are just send to the message bus, without any impact on the process in vCD. These can be used for example to call a workflow that adds a newly deployed vApp to a monitoring system.vCloud Director AMQP and Blocking Task settings

vRealize Orchestrator ships with an AMQP Plugin that can send and receive messages, and provides trigger that can start workflows when a message is received. In combination with the vCloud Director plugin this is a powerful solution to use vRO workflows being triggered by vCD, then facilitate external integration, and (in case of blocking tasks), send a response back to vCloud Director.

vRO AMQP Plugin

vCloud Director API Extensions powered by vRO Workflows

With the API Extensions it is possible to publish additional service offerings into the vCloud Director REST API. These services then can be consumed by tenants by calling the REST API. The API call in background will again send AMQP messages to the message bus, which makes vRealize Orchestrator a perfect recipient component. A detailed description of the vCloud Director API Extensions can be found in this whitepaper:

vCloud Director API Extensions

vCloud Director 9 Service Library Extension

vCloud Director 9 introduced a new mechanism to extend the service library for tenant. It’s possible to publish vRealize Orchestrator workflows as services into the library. When a user requests a service backed by a vRO workflow, the UI will automatically create a form based on the input parameters of the workflow. This is a very powerful extensibility mechanism, because it enables service providers to offer all sorts of services, even beyond typical Infrastructure-as-a-Service functionality. A nice example on how to use the service catalog extension to offer tenants a self-service password reset capability can be found here:

vCD Service Library Extension


vRealize Orchestrator is an important component in every vCloud Director environment. Its rich automation capabilities combined with the close out-of-the-box integration with vCloud Director make them a perfect combination to extend and automate vCloud Director with less development effort. The vCloud Director and AMQP Plugins for vRealize Orchestrator make it easy to implement automation of vCloud Director tasks, integrating external systems during blocking tasks and extending the service library.


The post Extend and Automate vCloud Director using vRealize Orchestrator appeared first on VMware Cloud Provider Blog.

Posted in VMware Cloud Provider | Comments

SDDC Delta Force for Cloud Providers – Networking & Security Track (LATAM)

Author – Joe Leo, Sr. Manager, Cloud Provider Systems Engineering

Our mission was to further enable our VCPP Partners with VMware’s Virtual Cloud Network Portfolio featuring Networking and Security solutions for Cloud Providers through virtual and on-site, live sessions. We received strong, positive feedback from both sessions in Mexico City and Bogota, Colombia, leaving partners with a deeper understanding of networking and security benefits for NSX and eager to learn more.


Mexico City, Mexico

Hotel Stara Hamburgo – Calle  Hamburgo 32, Juarez, 06600 Cuidad de Mexico CDMX

Monday June 11th 2018


Leslie Suarez and Teresa Garcia from LOL handled operations for the event with 12 participating partners on the first day. Eduardo Marin kicked off introductions and went over the objectives and desired outcomes for the event. Fernando Escobar and Neimar Oler delivered the morning sessions, including the VCPP Vision and Strategy/Use Cases featuring vCloud Director, vCloud Availability (C2C, DR2C) and much more. We received many questions and interactions during the session, given the small audience.

Neimar continued with the vCloud Director and vCAV discussions, covering NSX integration and vCloud Extender.

We provided a basic NSX 101 presentation for Cloud Providers using traditional vLAN, firewall and load-balancer technologies for hosted private cloud offerings, as well as reviewing the NSX-T Architecture. Ravi led NSX discussions, diving into the overlay virtual network and the portability of layer-2 leveraging the advancements of VxLAN and Geneve encapsulation techniques to accommodate the high-level flexibility and agility NSX-V and T offers, NSX-T Positioning and Use Cases, and NSX Hybrid Cloud Extender. All sessions were well received with high levels of engagement and participation.

Tuesday June 12th 2018

With 19 attendees on Day 2, the NSX/SD-WAN (VeloCloud) session generated a lot of interest and interactions. Ravi explained our fully ‘cloud-delivered’ architecture (SaaS) solution, leveraging the power of the branch and branch network +Cloud, which has rich reporting capabilities and the ability to leverage the power of the edge by enabling IoT delivery. As a result, the VCPP team has an opportunity to engage Kio Networks in Mexico City in a PoC, earn their business and expand on their current offerings.


Daniel Aguirre from NSBU covered VMware’s Cloud Services Strategy and AppDefense. This helped VMware’s Cloud Providers understand how to bolster NSX’s micro-segmentation capabilities with AppDefense’s application-level security awareness for VMs and clouds, leveraging the ‘least privileged’ strategy on the compute stack with AppDefense’s ‘Intended State’ knowledge of applications and their interactions based on known-behavior manifests.

Ravi explained the differences between NSX-V and NSX-T, the supported platforms (vSphere vs other hypervisors), a comparison of the supported encapsulations (vxLAN vs Geneve), as well as the differences between their API stacks. To ensure a full understanding on how our program charges for solutions, Eduardo walked through the overall VCPP program, points system and bundles to our partners.

Fernando closed down Day-2 with a Call-To-Action, reminding our Cloud Providers of our vCAT-SP architectural tools and compiled a repository of very useful, comprehensive information and enablement content. We then thanked everyone for their participation and partnership in the VCPP program, and recognized all participants including our supporting partner/Aggregator Licencia Online (LOL) for overall coordination of the hotel, meals and really cool Delta Force Polo shirts!


Bogota, Colombia

Hotel NH Collection Bogotá Teleport Royal – Cl. 113 #7-65, Bogotá, Colombia

June 14 -15th 2018

The second leg of the SDDC Delta Force training was held in beautiful Bogota, Colombia at the NH Teleport Hotel. Day 1 had over 35 Cloud Provider personnel in attendance with flawless execution by Adriana Triana and the LOL team handling logistics, food, translation-services and event planning.

Fernando kicked off Day 1 with a Vision and Strategy discussion, including a segment on containers for our Cloud Providers interested in our container strategy, diving into our PKS vision and the Developer Ready Infrastructure. Partnering with Pivotal Cloud Foundry and Kubernetes, we have developed a production-ready, highly available container service with deep integration with vSphere and NSX to help our Cloud Providers leverage their existing investments in VMware. This was in addition to the content that was shared in Mexico (vCD, vCAv, DR2C, C2C, platform extensibility, etc.).

Neimar spoke on the benefits of VMware’s vCloud Director 9.1 and the monetization benefits through its rich extensibility capabilities and easy-to-use, clean HTML-5 interface. Further discussion around leveraging integrated vRO workflows and creating customer/tenant-portals generated good interest and dialog among the participants. vCloud Director Extender, DR2C and C2C were also popular since tools to migrate highly-available workloads up into the cloud are top-of-mind.


Seeing as this was so well received in Mexico City, Ravi went back to the whiteboard to go over the concept of NSX’s network virtualization (logical view overlay), the associated supporting protocols and how it all works together to provide the agility and advantage over traditional networking functions. As expected, this was just as well received with the Cloud Providers in Colombia.


With the assistance of Fernando (running point on the VCPP Mobile Demo Lab), Neimar led a demo of vCloud Director showing and explaining the extensive monetization capabilities made possible due to the flexibility and extensibility of our amazing Cloud Provider Platform!


VMworld Pass Winners

Congratulations to Jorge Marquez and Giovanni Zapata, the VMworld pass winners from our Mexico City and Bogota events! We’ll see you both in Vegas!

SDDC Delta Force Training for Cloud Providers (VCPP) Recordings and Presentations:

Password: VCPP+Delta

Virtual Workshop Session 1 – Part 1: & Part 2:

Virtual Workshop Session 2 – Part 3 (Full Recording):

Full Presentations:  SDDC Delta Force Training for Cloud Providers – Mexico CitySDDC Delta Force Training for Cloud Providers – Bogota Colombia


Special thank you to our team members who made this possible:

The Mexico City Delta Force Team

The Bogota Delta Force Team

VCPP America’s Solutions Engineering: Fernando Escobar – Sr. Systems Engineer, Neimar Oler – Sr. Systems Engineer, Ravindra Shankar – Staff Systems Engineer, Wissam Mahmassani – Sr. Systems Engineer

VCCP America’s Sales: Rafael Vidaillet – Business Development Manager, Eduardo Marin – Sales Specialist

VMware America’s Marketing: Johanna Avila – Marketing Manager, Monica Jaramillo – Senior Marketing Manager

VMware America’s Aggregators: Leslie Juárez – Field Marketing México, Teresa Garcia – Field Marketing México, Adriana Triana – Field Marketing Manager Colombia, Marcelo Gardelin – Adistec Strategic Alliances Director, Carlos Mendo – Compusoluciones

The post SDDC Delta Force for Cloud Providers – Networking & Security Track (LATAM) appeared first on VMware Cloud Provider Blog.

Posted in Cloud Services, VMware Cloud Provider | Comments