CBT + DDBoost + BBBackups not equal to “almost zero”

Hello,

Just wondering on the space used on a Data Domain when you do several backup of the same server in a short period of time.

Since a while (exactly from the 7 of June) we experiment an increase of the space used on our DD not in relation with data growing.

DD space used.png

I discover that on the date where we had a capacity increased we had several manual launch of some backup.

What is “bizarre” is the fact that the capacity used on the DD seems to increase accordingly to the number of backups… (which is “acceptable” in case of traditional file level backup)

Capacity report V2.png

If you look at the number of backup version we have for the server “Server one” (as an example) you can see that on the 22 of June we had 5 backup of this server (don’t ask me why 5…) and every backup has a Data Domain copy (which is confirmed by the column AE of the Excel report.

Server one.png

I was expecting that with all CBT + Dedup and whatsoever optimization we will have just few blocks added and not the full disk.

(We are using Networker 9.2.1.4)

Questions:

1) Is CBT an option to be setup on every VM (found an article on that). We are running vSphere 6.5 with vProxy.

2) Even if not activated, a Block Based Backup should do that for you ?

3) Will an Incremental backup not take only modified Blocks ?

3) And what is the role of DDBoost in that “party” ?

Any explanation or answers welcomed because EMC dosen’t want to give me a clear explanation how things are really working here…

Related:

  • No Related Posts

VNX: Replication is not working due to wrong NFS export of the destination filesystem (User Correctable)

Article Number: 499603 Article Version: 3 Article Type: Break Fix



VNX/VNXe Family,VNX1 Series,VNX2 Series

Replication might not work properly due to wrong replication configuration of destination file system:

The customer has a replication session for a certain file system, any changes were done on production were reflected in DR site like expected and the replication session was synced and no reported issues, however the customer isn’t able to find those changes on destination AIX host.

The customer has configured the replication session so that destination file system name is not the same as source file system name.

The NFS export was done on source file system however it has to be done on the destination one.

From DR-host side the customer mounted the NFS export and couldn’t find changes done however those changes are existed on the DR-DM itself in destination file system.

If the NFS export has to be done on the source (production) file system name and it’s not possible to change it to destination file system then following can be done:

  • Delete the replication session.
  • Delete the source file system from DR site
  • Recreate the replication session again with the same source and destination file systems names.

If it’s possible to change the NFS export to be on the destination file system, then no need to change replication configuration and NFS export has to be re-created to the correct destination file system name.

Related:

  • No Related Posts

Re: Hyper-V Disaster Recovery with DELLEMC Unity

VMware uses Storage Replication Adapters, or plugins that specifically can handle using storage-based replication to secure the virtual machines, and handle the failover.

When looking at Hyper-V, the only materials that I could find were these:

https://docs.microsoft.com/en-us/azure/site-recovery/hyper-v-vmm-disaster-recovery

Look at page 38 of this guide:

https://www.emc.com/collateral/white-papers/h12557-storage-ms-hyper-v-virtualization-wp.pdf

Which basically makes it sound like Hyper-V Replica handles the replication on a per-guest/VHD basis. Alternatively you might want to look at Veeam or Zerto for VM-level recovery, but still it’s not storage-level replication.

You could also look at EMC Storage Integrator (ESI):

https://www.emc.com/data-center-management/storage-integrator-for-windows-suite.htm#!interoperability

I’m not saying that using storage-level replication won’t work. Maybe it will, but I can’t find anything that specifically says it will, or that seems to make it easy like it is with VMware.

~Chris

Related:

  • No Related Posts

NVP-vProxy standalone ESXi Host whitout vCenter trouble

Standalone ESXi Host whitout vCenter



NetWorker (NVE) 9.2.1.2

vProxy 2.1.1-41

ESXi 6.0



I try to make a VM Image backu (NVP-vProxy). The VM has 5 virtual disks, after 1-3 disk the backup stops.



Failed to open source VMDK “[VSA_LUN01] “servername/servername_2.vmdk”: VDDK Error: 18000: Cannot connect to the host.



But I can make a backup of every separate disk, without errors

Related:

  • No Related Posts

Microsoft App Agent – customer backed up SQL DBs to “wrong” Data Domain

Working with a customer on their first ever implementation of the Microsoft App Agent, and will hopefully be setting up replication soon (depending on whether I get replies to one of my other posts here about where the documentation is for that setup), but have discovered something else in the process.

The customer has SQL servers that operate at two different campuses, and each campus has a Data Domain local to it. The initial implementation and testing of the Microsoft App Agent was performed with the SQL servers and the Data Domain on the secondary campus before I started my part of the project. Over the course of the customer moving forward with their testing and further implementation of the agent on other SQL servers, I have discovered that they have also implemented the agent on the primary site SQL servers as well – except they have specified the secondary site Data Domain as the target for the SQL backups.

So, at this time, it would appear that the primary Data Domain didn’t even have an MTree created on it yet for the agent SQL backups. However, we do still have to set up replication between the 2 Data Domains for the SQL data.

So – is there any way for the replication to be set up that will allow replication of the existing data in the MTree on the secondary DD to the primary DD in a manner that once the replication is done, the customer can update his primary site SQL backup configurations to point to the primary DD, and the backups will be less “painful”? Also, even if replication can’t help with the initial local primary backup performance, is there anything I need to consider when setting up the local DD for local Microsoft App Agent backups with respect to the replication configuration? Such as – do I need to make sure I have unique MTree names that differentiate between data from primary site backups versus secondary site backups?

Not sure what options I have because I’m not even sure what kind of replication the Microsoft App Agent should use at this point (and I would gladly RTFM if I could find the FM).

All comments/feedback appreciated – thanks.

Related:

  • No Related Posts

Backup Exec 15 Job Failing Recently – Media Mount Failed

I need a solution

I have 4 servers that I’m backing up.  Server A has the Backup Exec 15 software installed, and the 3 others have the remote agent.  I’m backing up all drives on the machines as well as system state, all to tape.  The job has been working great for the past 3+ years.  Just last week it started failing on one of the remote servers.  Here’s the Error from last night.  

Backup- \OURSERVERNAMED:
V-79-57344-41488 - Due to one or more errors in \WisTSK02.wise.k12D:, this backup cannot be used for conversion to virtual machines, Simplified Disaster Recovery (SDR), or a complete online restore.

Because the current incremental or full job failed, subsequent incremental or differential jobs will not include files that were backed up in the failed job. Therefore, the subsequent incremental or differential jobs will not be valid for a Simplified Disaster Recovery restore or a complete online restore. You must run a full backup again to ensure a successful system recovery.

Media mount failed.

User canceled a Physical Volume Library operation.

V-79-57344-33861 - The media operation was terminated by the user.

It has been having the same error every night. Most of the time its on the System?State instead of a drive.  

I have updated the main server software, as well as the remote agents and of course rebooted.  

Please help!

0

Related:

  • No Related Posts

Re: ScaleIO 2.0.1.3 on VMWare – abysmal write performance while uploading to datastore

Hello, guys!

We are experiencing a rather weird behavior of our ScaleIO deployment. Firstly, let me describe the setup:

6 Dell FX2 blocks with two FC630 and FD332 nodes each.

Every node is connected with 4 10G links to an MLAG group of two Arista DCS-7150S-52-CL switches. Two of the uplinks are used for VM data and two – for ScaleIO

In every FX2 block one FC630 node is running ESXi 6.5U1 and one – Windows 2012R2 Core with Hyper-V. Both are contributing to ScaleIO storage and both are in the same and only PD.

In the FD332 storage nodes we are using 10 Dell 1.64 Tb SFF SAS spinning drives with two 800 Gb Mixed Use SSDs for rfcache.

The system runs quite smoothly with pretty much satisfying performance:

Capture.PNG.png

Yet there is a problem, and it’s quite bad.

When uploading a file to the datastore or replicating a VM to the vmware part of the cluster – the speed never exceeds 16 MB/s per one session, i.e. if I were to copy two files simultaneously – the aggregate speed would be 31 MB/s more or less. As you might guess, when moving VMs with disks over 1Tb – such speed becomes a pain. Restoring from a backup (we use veeam B&R) also hits this limit. And all of this only applies to vmware part of the cluster – with Hyper-V we easily exceed 120 MB/s with the same activities. Once again, the ScaleIO cluster is the same on vmware and hyper-v. The hardware is also the same down to each and every P/N.

The most interesting part, however, is that Storage vMotion to the ScaleIO cluster from the same cluster we have been copying the VMs with veeam works fine!

Here is what we have checked:

  • Everything network-related – MLAGs, Arista configs etc.
  • ESXi configuration – LUN depth, vSwitch configs, SVM configuration etc.
  • We tried to avoid using vmk for veeam traffic by using two proxy VMs (Guest OS is windows 2016) – one in source esxi cluster (not scaleio) and one in destination (with scaleio). In such setup veeam directly attaches the snapshot to the proxy VM on both source and destination and thus traffic never gets to vmk – the destination proxy VM writes directly to the scaleio LUN. In such setup we are seeing “Highest active time: 100%” in windows resource monitor

So, any help would be greatly appreciated as we are really stuck with this situation.

Related:

  • No Related Posts

Re: vProxy restore from vSphere web client fails immediately

Hello everyone,

I have a customer who is using a vProxy to backup up all of his VMs.

NetWorker server OS is a RHEL 6.9 and the installed NetWorker version is 9.1.1.4 along with vProxy version 2.0.2-23_1.

From a NMC view everything is working fine. Backup, restore and FLR.

The problem occurs after the login into the vSphere web client. The plugin “VM Backup and Recovery” was successfully installed but after the connect to the NetWorker server no policies/workflows are listed in the backup section (five workflows are configured under a policy named “VMware” in the NMC and groups containing the VMs are assigned). In the recover section all of the backups are listed. A restore attempt of any VM brings up the restore wizard which can be execute all the way but fails immediately after clicking on finish. The failure message is “An unexpected connection error occurred and the cause could not be determined. Please verify the server is running” appears (the NetWorker server runs of course).

The daemon.log does not show any message during the restore attempt.

The restapi.log has some entries:

When a VW is selected for restore in vSphere:

2018-02-02 15:11:08.687 WARN [ ] c.e.n.w.c.j.JsonConvEngine – Unable to register field ‘thinProvisioned’: getter or setter method is not found.

2018-02-02 15:11:08.688 WARN [ ] c.e.n.w.c.j.JsonConvEngine – Discarded unknown JSON key: ‘datacenter’.

2018-02-02 15:11:08.688 WARN [ ] c.e.n.w.c.j.JsonConvEngine – Discarded unknown JSON key: ‘change-version’.

2018-02-02 15:11:08.688 WARN [ ] c.e.n.w.c.j.JsonConvEngine – Discarded unknown JSON key: ‘template’.

2018-02-02 15:11:08.688 WARN [ ] c.e.n.w.c.j.JsonConvEngine – Discarded unknown JSON key: ‘path’.

2018-02-02 15:11:08.688 WARN [ ] c.e.n.w.c.j.JsonConvEngine – Discarded unknown JSON key: ‘host-name’.

2018-02-02 15:11:08.688 WARN [ ] c.e.n.w.c.j.JsonConvEngine – Discarded unknown JSON key: ‘moref-vm-path’.

2018-02-02 15:11:08.688 WARN [ ] c.e.n.w.c.j.JsonConvEngine – Discarded unknown JSON key: ‘ip-address’.

2018-02-02 15:11:08.688 WARN [ ] c.e.n.w.c.j.JsonConvEngine – Discarded unknown JSON key: ‘os-name’.

2018-02-02 15:11:08.688 WARN [ ] c.e.n.w.c.j.JsonConvEngine – Discarded unknown JSON key: ‘vm-path’.

2018-02-02 15:11:08.688 WARN [ ] c.e.n.w.c.j.JsonConvEngine – Discarded unknown JSON key: ‘os-identifier’.

2018-02-02 15:11:08.688 WARN [ ] c.e.n.w.c.j.JsonConvEngine – Discarded unknown JSON key: ‘cluster-compute-resource’.

2018-02-02 15:11:08.688 WARN [ ] c.e.n.w.c.j.JsonConvEngine – Discarded unknown JSON key: ‘esxi-name’.

2018-02-02 15:11:08.688 WARN [ ] c.e.n.w.c.j.JsonConvEngine – Discarded unknown JSON key: ‘datastore’.

2018-02-02 15:11:08.688 WARN [ ] c.e.n.w.c.j.JsonConvEngine – Discarded unknown JSON key: ‘version’.

2018-02-02 15:11:08.688 WARN [ ] c.e.n.w.c.j.JsonConvEngine – Discarded unknown JSON key: ‘compute-resource’.

2018-02-02 15:11:08.688 WARN [ ] c.e.n.w.c.j.JsonConvEngine – Discarded unknown JSON key: ‘networks’.

2018-02-02 15:11:08.688 WARN [ ] c.e.n.w.c.j.JsonConvEngine – Discarded unknown JSON key: ‘thin’.

And after the restore execution:

2018-02-02 15:13:08.960 DEBUG [ ] c.e.n.c.Nsrdb – Query ‘[type: NSR hypervisor, name: vcenter_fqdn]’, show ‘[environment: ]’, numRes ‘1’.

2018-02-02 15:13:08.966 DEBUG [ ] c.e.n.c.Nsrdb – Number of results: ‘1’.

Has anyone experienced something like that?

I would appreciate any help would be more that happy to provide more information if desired.

Cheers

Jan

Related:

  • No Related Posts

Re: Failed to create snapshot for LUN

Hello

I want to go to the Veeam LAN Free Backup

Backup Proxy is Veeam itself

I chose the Direct Storage Access option

But when Run Backup, Error happens

It seems that it can not snapshot LUNs

Failed to create snapshot for LUN Error: CLIProvider : VNX Snapshots feature software is not enabled. Please install the VNX Snapshots feature enabler. (0x716d8033)

z1.PNG

I did not get it right

That means the VNX Snapshots license must be enabled on EMC ?

Related:

  • No Related Posts

Failed to create snapshot for LUN

Hello

I want to go to the Veeam LAN Free Backup

Backup Proxy is Veeam itself

I chose the Direct Storage Access option

But when Run Backup, Error happens

It seems that it can not snapshot LUNs

Failed to create snapshot for LUN Error: CLIProvider : VNX Snapshots feature software is not enabled. Please install the VNX Snapshots feature enabler. (0x716d8033)

z1.PNG

I did not get it right

That means the VNX Snapshots license must be enabled on EMC ?

Related:

  • No Related Posts