Re: Question about Windows processes involved in Avamar backup

Working with a customer on backups for a large Windows file server, and am trying to get an idea of performance, so I have the Windows Resource Monitor open and I’m looking at disk operations.

While I see the expected entries for avtar under both “Processes with Disk Activity” and “Disk Activity”, I am also seeing what appear to be “associated” operations of some kind for “System” with a PID of “4” – and under the “Disk Activity” section, the System entries seem to be working on the same “DeviceHarddiskVolumeShadowCopy…” entries that the avtar processes are also working on.



I understand that the Windows backup uses VSS these days and that Avamar would likely “hit” that shadow copy – I also understand that Windows will likely be working on that shadow copy as well to a degree. What I curious about is that at times, the System processes seem to be a lot more busy with the shadow copy than avtar is – and while this system is being used (as a file server), I wouldn’t expect it to be “hit” as much by users as I would expect it to be “hit” by Avamar during a backup.



(and I also understand that System PID 4 is somewhat of a “catch all” process that takes care of a lot of things – but I still can’t figure out why at some points it would be showing 4x or more the number of I/Os that avtar would)



Can anyone provide some additional perspective on why there are so many System processes “hitting” the shadow copy as well as the avtar ones?



All comments/feedback appreciated – thanks.

Related:

Question about Windows processes involved in Avamar backup

Working with a customer on backups for a large Windows file server, and am trying to get an idea of performance, so I have the Windows Resource Monitor open and I’m looking at disk operations.

While I see the expected entries for avtar under both “Processes with Disk Activity” and “Disk Activity”, I am also seeing what appear to be “associated” operations of some kind for “System” with a PID of “4” – and under the “Disk Activity” section, the System entries seem to be working on the same “DeviceHarddiskVolumeShadowCopy…” entries that the avtar processes are also working on.



I understand that the Windows backup uses VSS these days and that Avamar would likely “hit” that shadow copy – I also understand that Windows will likely be working on that shadow copy as well to a degree. What I curious about is that at times, the System processes seem to be a lot more busy with the shadow copy than avtar is – and while this system is being used (as a file server), I wouldn’t expect it to be “hit” as much by users as I would expect it to be “hit” by Avamar during a backup.



(and I also understand that System PID 4 is somewhat of a “catch all” process that takes care of a lot of things – but I still can’t figure out why at some points it would be showing 4x or more the number of I/Os that avtar would)



Can anyone provide some additional perspective on why there are so many System processes “hitting” the shadow copy as well as the avtar ones?



All comments/feedback appreciated – thanks.

Related:

Re: VSS Backups not working after Windows 10 creators update 1709

Hello everyone,

just wondering whether anyone else has experienced problems with Avamar VSS Backups of Windows 10 clients since the creators update (1709) was rolled out.

I have a relatively new Win10 Client. Backups of Filesystem and VSS were working fine until the creators update was installed a couple of weeks ago. Now the VSS backups fail after 30 seconds. This is happening on all Windows 10 clients (Professional) with this update.

Error code is 10007

Filesystem backups are still working fine. But the System State backups are no longer working.

In the logfiles I see this:

2017-11-28 20:00:25 avvss Info <16480>: EFI system detected. ASR BCD will not be included in shadow copy

2017-11-28 20:00:25 avvss Info <19014>: Determining best VSS provider for volume ‘c:’

2017-11-28 20:00:25 avvss Info <40027>: Volume c: will be frozen by ‘Microsoft Software Shadow Copy provider 1.0’ in Snapshot {a38bcfd5-e1d9-4341-aedb-e9f403391672}

2017-11-28 20:00:25 avvss Info <8776>: Freezing volumes now…

2017-11-28 20:00:25 avvss Info <8780>: Creating the shadow copy set (DoSnapshotSet) …

2017-11-28 20:00:28 avvss Error <10975>: ERROR: Selected writer ‘ASR Writer’ is in failed state!

– Status: 8 (VSS_WS_FAILED_AT_PREPARE_SNAPSHOT)

– Writer Failure code: 0x800423f0 (Der Schattenkopiesatz enthält nur eine Teilmenge der

Volumes, die für eine ordnungsgemäße Sicherung der ausgewählten

Komponenten des Generators notwendig sind.)

– Writer ID: {be000cbe-11fe-4426-9c58-531aa6355fc4}

– Instance ID: {5ee97a67-801b-4754-ab89-98f2f8d093c5}

2017-11-28 20:00:28 avvss Error <10976>: ERROR: Selected writer ‘ASR Writer’ reported an error!

– Status: 8 (VSS_WS_FAILED_AT_PREPARE_SNAPSHOT)

– Writer Failure code: 0x800423f0 (Der Schattenkopiesatz enthält nur eine Teilmenge der

Volumes, die für eine ordnungsgemäße Sicherung der ausgewählten

Komponenten des Generators notwendig sind.)

– Writer ID: {be000cbe-11fe-4426-9c58-531aa6355fc4}

– Instance ID: {5ee97a67-801b-4754-ab89-98f2f8d093c5}

2017-11-28 20:00:28 avvss Error <12077>: VSS returned an error message exiting HR = 0x8000ffff

2017-11-28 20:00:28 avvss Info <0000>: Finished with backup. Performing cleanup.

The text in German above translates to:

The shadow-copy set only contains only a subset of the volumes needed to correctly backup the selected components of the writer

Avamar Version 7.3.1-125 (Server & Client)

I’ve already tried rebooting the clients.

Just after the backup fails, I ran vssadmin list writers. It showed the ASR Writer in a failed state. But after a while the writer returns to a stable state.

Any ideas?

Thanks and regards,

Stephen

Related:

  • No Related Posts

VSS Backups not working after Windows 10 creators update 1709

Hello everyone,

just wondering whether anyone else has experienced problems with Avamar VSS Backups of Windows 10 clients since the creators update (1709) was rolled out.

I have a relatively new Win10 Client. Backups of Filesystem and VSS were working fine until the creators update was installed a couple of weeks ago. Now the VSS backups fail after 30 seconds. This is happening on all Windows 10 clients (Professional) with this update.

Error code is 10007

Filesystem backups are still working fine. But the System State backups are no longer working.

In the logfiles I see this:

2017-11-28 20:00:25 avvss Info <16480>: EFI system detected. ASR BCD will not be included in shadow copy

2017-11-28 20:00:25 avvss Info <19014>: Determining best VSS provider for volume ‘c:’

2017-11-28 20:00:25 avvss Info <40027>: Volume c: will be frozen by ‘Microsoft Software Shadow Copy provider 1.0’ in Snapshot {a38bcfd5-e1d9-4341-aedb-e9f403391672}

2017-11-28 20:00:25 avvss Info <8776>: Freezing volumes now…

2017-11-28 20:00:25 avvss Info <8780>: Creating the shadow copy set (DoSnapshotSet) …

2017-11-28 20:00:28 avvss Error <10975>: ERROR: Selected writer ‘ASR Writer’ is in failed state!

– Status: 8 (VSS_WS_FAILED_AT_PREPARE_SNAPSHOT)

– Writer Failure code: 0x800423f0 (Der Schattenkopiesatz enthält nur eine Teilmenge der

Volumes, die für eine ordnungsgemäße Sicherung der ausgewählten

Komponenten des Generators notwendig sind.)

– Writer ID: {be000cbe-11fe-4426-9c58-531aa6355fc4}

– Instance ID: {5ee97a67-801b-4754-ab89-98f2f8d093c5}

2017-11-28 20:00:28 avvss Error <10976>: ERROR: Selected writer ‘ASR Writer’ reported an error!

– Status: 8 (VSS_WS_FAILED_AT_PREPARE_SNAPSHOT)

– Writer Failure code: 0x800423f0 (Der Schattenkopiesatz enthält nur eine Teilmenge der

Volumes, die für eine ordnungsgemäße Sicherung der ausgewählten

Komponenten des Generators notwendig sind.)

– Writer ID: {be000cbe-11fe-4426-9c58-531aa6355fc4}

– Instance ID: {5ee97a67-801b-4754-ab89-98f2f8d093c5}

2017-11-28 20:00:28 avvss Error <12077>: VSS returned an error message exiting HR = 0x8000ffff

2017-11-28 20:00:28 avvss Info <0000>: Finished with backup. Performing cleanup.

The text in German above translates to:

The shadow-copy set only contains only a subset of the volumes needed to correctly backup the selected components of the writer

Avamar Version 7.3.1-125 (Server & Client)

I’ve already tried rebooting the clients.

Just after the backup fails, I ran vssadmin list writers. It showed the ASR Writer in a failed state. But after a while the writer returns to a stable state.

Any ideas?

Thanks and regards,

Stephen

Related:

  • No Related Posts

7022371: Block Based Transfer Method selection fails with an Error: “VSS snapshots are required in order to perform Block Based transfers”

This document (7022371) is provided subject to the disclaimer at the end of this document.

Environment

PlateSpin Migrate 12.x
PlateSpin Protect 11.x
PlateSpin Forge 11.x

Situation

During the initial job configuration after workload discovery, an error which states: “VSS snapshots are required in order to perform Block Based transfers” is displayed upon selecting the Block Based option in the Transfer Method section of the Replication Settings

Resolution

By default, “EarliestWindowsVersionForCommonInventory” is set to “5.2” which means that for Windows 2003 (whose internal number is “5.2”) and up, PlateSpin will use the new discovery mechanism also known as Common Inventory. Kindly refer below for the Windows versions and their corresponding internal number
– Windows 8.1; Windows Server 2012 R2 = 6.3
– Windows 8; Windows Server 2012 = 6.2
– Windows 7; Windows Server 2008 R2 = 6.1
– Windows Server 2008; Windows Vista = 6.0
– Windows Server 2003 R2; Windows Server 2003; Windows Home Server; Windows XP Professional X64 Edition = 5.2
– Windows XP = 5.1
– Windows 2000 = 5.0
To work around this issue, set the value of the “EarliestWindowsVersionForCommonInventory” parameter in the PlateSpin Configuration to “5.2” and then rediscover the source server
OR
You can edit the XML file of the source machine in the PlateSpin Browser by adding the following lines below the <systemCacheFileList /> tag
<volumeShadowService>
<providers>
<provider>
<id>{b5946137-7b9f-4925-af80-51abd60b20d5}</id>
<name>Microsoft Software Shadow Copy provider 1.0</name>
<version>1.0.0.7</version>
</provider>
</providers>
</volumeShadowService>
There could be some variations in the XML file that you have to be aware of. For instance:
<machines:systemCacheFileList />
Add the lines below including the prefix “machines:”
<machines:volumeShadowService>
<machines:providers>
<machines:provider>
<machines:id>{b5946137-7b9f-4925-af80-51abd60b20d5}</machines:id>
<machines:name>Microsoft Software Shadow Copy provider 1.0</machines:name>
<machines:version>1.0.0.7</machines:version>
</machines:provider>
</machines:providers>
</machines:volumeShadowService>
OR it could also be in this instance:
<q1:systemCacheFileList />
Add the lines below including the prefix “q1:”
<q1:volumeShadowService>
<q1:providers>
<q1:provider>
<q1:id>{b5946137-7b9f-4925-af80-51abd60b20d5}</q1:id>
<q1:name>Microsoft Software Shadow Copy provider 1.0</q1:name>
<q1:version>1.0.0.7</q1:version>
</q1:provider>
</q1:providers>
</q1:volumeShadowService>

Cause

In some cases, using the old discovery mechanism can cause the snapshot transfer method provider to be not recognized, thus throwing VSS snapshot error during job configuration.

Disclaimer

This Support Knowledgebase provides a valuable tool for NetIQ/Novell/SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented “AS IS” WITHOUT WARRANTY OF ANY KIND.

Related:

Re: Re: SQL Backup Configured as NSRSQLSV suddenly failing – error attached

Your output indicates that the shadow storage is not set up.

Did you look at the event logs, and there were no VSS issues?

On my system, I deleted all my shadows, bit it still shows the shadow storage association:

C:windowssystem32>vssadmin List volumes

vssadmin 1.1 – Volume Shadow Copy Service administrative command-line tool

(C) Copyright 2001-2005 Microsoft Corp.

Volume path: C:

Volume name: \?Volume{1c425a42-db08-11e5-93c5-806e6f6e6963}

C:windowssystem32>vssadmin List shadows

vssadmin 1.1 – Volume Shadow Copy Service administrative command-line tool

(C) Copyright 2001-2005 Microsoft Corp.

No items found that satisfy the query.

C:windowssystem32>vssadmin List Shadowstorage

vssadmin 1.1 – Volume Shadow Copy Service administrative command-line tool

(C) Copyright 2001-2005 Microsoft Corp.

Shadow Copy Storage association

For volume: (C:)\?Volume{1c425a42-db08-11e5-93c5-806e6f6e6963}

Shadow Copy Storage volume: (C:)\?Volume{1c425a42-db08-11e5-93c5-806e6f6e6963}

Used Shadow Copy Storage space: 0 B (0%)

Allocated Shadow Copy Storage space: 0 B (0%)

Maximum Shadow Copy Storage space: 46.576 GB (5%)

Refer to the following article on how to recreate the shadow storage area, and then see that changes the backup results.

https://www.petri.com/forums/forum/server-operating-systems/windows-server-2008-2008-r2/58571-vss-error-0x8004230f-on-a-brand-new-system

Related:

Event ID 1 — Volume Shadow Copy Service Initialization

Event ID 1 — Volume Shadow Copy Service Initialization

Updated: January 27, 2011

Applies To: Windows Server 2008 R2

The Volume Shadow Copy Service (VSS) provides the ability to create a point in time image (shadow copy) of one or more volumes that can be used to perform backups. The service is also used during restores of applications. After the VSS service is initialized, VSS requester applications can perform backups and restores in cooperation with VSS writers that manage application data to be backed up and VSS providers that create and manage shadow copies.

Event Details

Product: Windows Operating System
ID: 1
Source: VSS
Version: 6.1
Symbolic Name: VSS_ERROR_STARTING_SERVICE_CTRL_DISPATCHER
Message: Volume Shadow Copy Service initialization error: the control dispatcher cannot be started [%1]. %2

Resolve
Review Service Control Manager events in Event Viewer and take appropriate action

This event indicates that the Volume Shadow Copy Service (VSS) could not register its dispatcher function with the Service Control Manager during VSS service initialization.

To perform this procedure, you must have membership in Administrators, or you must have been delegated the appropriate authority.

To view the VSS events in the Windows System event log:

  1. Open Event Viewer. Click Start, click Run, type eventvwr.msc, and then click OK. If the User Account Control dialog box appears, ensure that the action it displays is what you want, and then click Continue.
  2. In Event Viewer, expand Windows Logs, and then click System.
  3. To create a filtered view of just the Service Control Manager events, click Action, click Create Custom View, choose By source, and in the Event sources drop-down list, select Service Control Manager, and then click OK.
  4. In Save Filter to Custom View, provide a name and description.
  5. In the left pane, expand Custom View, right-click the name of the custom view that you created, and then click Find.
  6. To find all the Service Control Manager Eventlog Provider events for VSS, in Find what, type: Shadow Copy.
  7. To examine an event, double-click the event, and then click Event Log Online Help for more information about how to resolve the issue.
  8. If you cannot resolve the issue, contact Microsoft Customer Service and Support. For more information, see http://go.microsoft.com/fwlink/?LinkId=102491. You should provide the entire event log message as it appears in the Event Viewer.

Verify

To verify that the Volume Shadow Copy Service (VSS service) is available, perform the following procedure, and then retry the previous Volume Shadow Copy Service (VSS) operation.

To perform this procedure, you must have membership in Administrators, or you must have been delegated the appropriate authority.

To verify that the VSS service is available:

  1. Open an elevated Command Prompt window by clicking Start, pointing to All Programs, clicking Accessories, right-clicking Command Prompt, and then clicking Run as administrator.
  2. At the prompt, type: net start vss.
  3. View the text message to verify that the service starts or is already started.

Related Management Information

Volume Shadow Copy Service Initialization

File Services

Related:

Event ID 8225 — Volume Shadow Copy Service Operations

Event ID 8225 — Volume Shadow Copy Service Operations

Updated: January 27, 2011

Applies To: Windows Server 2008 R2

The Volume Shadow Copy Service (VSS) provides the ability to create a point in time image (shadow copy) of one or more volumes that can be used to perform backups. The service is also used during restores of applications.

Event Details

Product: Windows Operating System
ID: 8225
Source: VSS
Version: 6.1
Symbolic Name: VSS_INFO_SHUTDOWN_SIGNAL
Message: The VSS service is shutting down due to shutdown event from the Service Control Manager. %1

Resolve

This is a normal condition. No further action is required.

Related Management Information

Volume Shadow Copy Service Operations

File Services

Related:

Event ID 8228 — Volume Shadow Copy Service Operations

Event ID 8228 — Volume Shadow Copy Service Operations

Updated: January 27, 2011

Applies To: Windows Server 2008 R2

The Volume Shadow Copy Service (VSS) provides the ability to create a point in time image (shadow copy) of one or more volumes that can be used to perform backups. The service is also used during restores of applications.

Event Details

Product: Windows Operating System
ID: 8228
Source: VSS
Version: 6.1
Symbolic Name: VSS_ERROR_IXMLDOMDOCUMENT_PARSEERROR
Message: Fail to parse XML file. Reason %1 Line %2 Position %3 Errorcode %4 Problem text %5 %6

Resolve
Contact the backup software vendor

This event is usually caused by restoring from a backup that was created on a more recent Windows operating system version.

To resolve this issue, contact the backup software vendor for further support.

Verify

To verify that the Volume Shadow Copy Service (VSS) is operating correctly, retry the previous VSS operation.

Related Management Information

Volume Shadow Copy Service Operations

File Services

Related: