Tuesday 5 December 2017

Powershell/Powercli: remove disabled NICs.

Hi.

Today I had some interesting task, to remove/modify disabled NICs in Windows OS. Information I had was just list of VMs.

"Easy"- I thought - "I''ll take MAC addresses of disabled adapters, compare them against adapters present in VM, done".
 As it turned out, it's not as straight forward as I assumed. When adapters are disabled in OS, driver is unloaded and no info about NIC's MAC or IP is provided. So no WMI.
There is Get-NetAdapter cmdlet, which can return MAC of disabled NIC, but I had issues with PSremoting, and I didn't have access to OS itself.

So I decided to turn a flaw into a advantage. VMtools grabs info about NIC and present it in
 $currentvm.ExtensionData.Guest.Net ($currentvm is of course variable holding VM object):


We have all nice info about active adapters in OS. So, if adapter is disabled, it's not present here.

I then grabbed list of MACs from Get-Networkadapter cmdlet:





Now it's just comparing the lists, adding some logic in case there are more than one active adapter present in OS, and I'll ended up with this script:

$vms = Get-Content .\vms.txt

foreach ($vm in $vms) {
$currentvm = get-vm $vm
$workingmac = $currentvm.ExtensionData.Guest.net.macaddress
$currentvmadapters = $currentvm|Get-NetworkAdapter
$macstoremove = $currentvmadapters.MacAddress

 if ($workingmac.count -ge 1) {
  $workingmac|foreach {
  $removeme = $_
  $macstoremove = $macstoremove|sls -NotMatch "$removeme"
 
   }

  }
  
 $macstoremove|foreach {
 $mac = $_
 $currentvmadapters|where {$_.macaddress -like "$mac" }|Set-NetworkAdapter -StartConnected:$false -Connected:$false

 }

}



I'm using simple Select-String cmdlet to filter out MAC addresses currently used from list of MACs to be modified. 
I'm just disconnecting NICs here, but of course it can delete them.


Update:

Turned out that Remove-NetworkAdapter cmdlet works only with powered off VMs. Bummer.

But thanks to amazing LucD from vmware forums, it can be done by device change task for each device. Here's the thread.

So final code may look like this:

$vms = Get-Content .\vms.txt

foreach ($vm in $vms) {
$currentvm = get-vm $vm
$workingmac = $currentvm.ExtensionData.Guest.net.macaddress
$vmos = $currentvm.ExtensionData.Config.GuestFullName
$currentvmadapters = $currentvm|Get-NetworkAdapter
$macstoremove = $currentvmadapters.MacAddress
$spec = New-Object VMware.Vim.VirtualMachineConfigSpec
$nicstoremove = @()

if ($vmos -notlike "*windows*") {
$currentvm|select name,@{n="OS";e={$vmos}}|Out-File -Force -Append nonwindows_OS.txt
write-host -ForegroundColor Yellow "$($currentvm.name) is not the Windows VM..."
}
else {


if ($workingmac.count -ge 1) {
$workingmac|foreach {
 $removeme = $_
 $macstoremove = $macstoremove|sls -NotMatch "$removeme"
 
  }
 }

 Write-host -ForegroundColor Green "Processing $($currentvm.name) VM..."
  
 $macstoremove|foreach {
 $mac = $_
 $nicstoremove += $currentvmadapters|where {$_.macaddress -like "$mac" }
 }
 }
 $nicstoremove|select parent, macaddress, type, networkname|Export-Csv NicsToRemove.csv -Append -NoTypeInformation

$spec = New-Object VMware.Vim.VirtualMachineConfigSpec
  foreach($Adapter in $nicstoremove){
      $devSpec = New-Object VMware.Vim.VirtualDeviceConfigSpec
      $devSpec.operation = "remove"
      $devSpec.device += $Adapter.ExtensionData
      $spec.deviceChange += $devSpec
    }
  $currentvm.ExtensionData.ReconfigVM_Task($spec)|Out-Null


 }















Sunday 17 September 2017

Zerto VPG creation failure (vim.fault.CannotCreateFile)

I just solved annoying issue. I'm using Zerto to replicate VM's between sites. Target site has vSAN as storage. Some time ago we had host failure and we replaced host with new one.

When creating new VPG, Zerto throws very generic error:




In tasks I found additional info:

Cannot complete file creation operation.. Fault: Vim25Api.CannotCreateFile.

I started to dig, first steps was to look at /var/log/vpxa.log file. I found this:

2017-09-15T20:27:34.791Z info vpxa[2F85EB70] [Originator@6876 sub=Default opID=326fd094-3b] [VpxLRO] -- ERROR task-internal-108298 -- vpxa -- vpxapi.VpxaService.reserveName: vim.fault.CannotCreateFile:
--> Result:
--> (vim.fault.CannotCreateFile) {
--> faultCause = (vmodl.MethodFault) null,
--> file = "ds:///vmfs/volumes/vsan:a5c518a4ceaa4b9e-8cd24fc5c9c0cad3/44e38a59-5440-a5e4-a8e5-0cc47aa432a8/256_vsanDatastore_vm-393_history_10_134_9_132_log_volume.vmdk",
--> msg = ""
--> }
--> Args:
-->
--> Arg spec:
--> (vpxapi.VmLayoutSpec) {
--> vmLocation = (vpxapi.VmLayoutSpec.Location) null,
--> multipleConfigs = <unset>,
--> basename = "Z-VRA-host2.domain.com",
--> baseStorageProfile = <unset>,
--> disk = (vpxapi.VmLayoutSpec.Location) [
--> (vpxapi.VmLayoutSpec.Location) {
--> url = "ds:///vmfs/volumes/vsan:a5c518a4ceaa4b9e-8cd24fc5c9c0cad3/44e38a59-5440-a5e4-a8e5-0cc47aa432a8/256_vsanDatastore_vm-393_history_10_134_9_132_log _volume.vmdk",
--> key = 16001,
--> sourceUrl = <unset>,
--> urlType = "exactFilePath",
--> storageProfile = <unset>
--> }
--> ],
--> reserveDirOnly = <unset>
--> }
 


Not very helpfull... 'msg' line was empty, so I started to chase my tail to find something. 
 Then I looked into /var/log/hostd.log file and I found this:

2017-09-16T21:44:11.266Z info hostd[69401B70] [Originator@6876 sub=Solo.Vmomi opID=179ca1eb-c-cc73 user=vpxuser:VSPHERE.LOCAL\prod-Zerto-fcbdf1fb-3575-4d99-9fde-0be131222758] Result:
--> (vim.fault.FileAlreadyExists) {
-->    faultCause = (vmodl.MethodFault) null,
-->    faultMessage = (vmodl.LocalizableMessage) [
-->       (vmodl.LocalizableMessage) {
-->          key = "com.vmware.esx.hostctl.default",
-->          arg = (vmodl.KeyAnyValue) [
-->             (vmodl.KeyAnyValue) {
-->                key = "reason",
-->                value = "Failed to create directory 44e38a59-5440-a5e4-a8e5-0cc47aa432a8 (File Already Exists)"
-->             }
-->          ],
-->          message = <unset>
-->       }
-->    ],
-->    file = "44e38a59-5440-a5e4-a8e5-0cc47aa432a8"
-->    msg = ""
--> }


 Ah, it can't create directory. Quick look to vSAN content and I noticed that this directory exists. But deleting it from Web client failed. So I went back to ESXi:


ls: ./44e38a59-5440-a5e4-a8e5-0cc47aa432a8: No such device or address


Hmm, something's wrong here, can I remove it by force? Of course:

 /usr/lib/vmware/osfs/bin/osfs-rmdir 44e38a59-5440-a5e4-a8e5-0cc47aa432a8 -f

Result:

Deleting directory 44e38a59-5440-a5e4-a8e5-0cc47aa432a8 in container id a5c518a4ceaa4b9e8cd24fc5c9c0cad3 backed by vsan (force=True)

 (yes, I used The Force)

Verified in GUI that catalog was deleted (it was) and tried to create VPG again... Voila, it worked! 

So, leftovers of old host/VRA caused this issue. Interesting, vSphere didn't give any information that there is something wrong. Health service reported everything as green. 

Oh, BTW I passed some time ago VCAP6 design exam and now I'm VCIX :P