OneFS: Upgrade fails with error “UnicodeDecodeError: ‘utf8’ codec can’t decode byte”

Article Number: 524285 Article Version: 5 Article Type: Break Fix



Isilon,Isilon OneFS,Isilon OneFS 8.0,Isilon OneFS 8.1

When running a OneFS upgrade the upgrade might fail during the isi_version_checker phase stating that it is not able to decode a file which is not encoded with UTF-8.

UnicodeDecodeError: ‘utf8’ codec can’t decode byte

You can verify the same by performing an upgrade assessment which will give out same results when viewing the upgrade summary report.

isi upgrade nodes list displays the following error :

Node Upgrade State: error

Error Details: Failed Step: HookPreUpgradeOptional

The pre-upgrade check from 8.1.0.2 to 8.1.0.4 failed in UID_GID_Check with the following error:
The user id or group id of [/tmp/abc5cc_3056986] is too large and may cause an error in the upgrade process, please change the user/group id (maximum of 2097151) or remove the file before continuing

The failure is likely to occur when running the uid_gid_check script which checks for any files under ‘/’ with a large UID/GID as this could impact the upgrade. If one of the files contains non-UTF-8 characters in its name. The script looks for any characters with non UTF encoding under / on all nodes.

OneFS 8.0.0.7, CLI parser can handle characters that were not valid UTF-8 avoiding such issues.

New pre-upgrade check introduced in 8.1.0.3

[A] If there is an OneFS upgrade which has failed follow the below steps:

(If there is an upgrade assessment which is giving an error jump to [B])

  1. Check the upgrade state first:

# isi upgrade view

Upgrade Status:

Cluster Upgrade State: error >> error state in upgrade

(see output of `isi upgrade nodes list` to know which node failed)

Current Upgrade Activity: Upgrade

Upgrade Install Image: /ifs/data/Isilon_Support/OneFS_v8.0.0.7_Install.tar.gz

Upgrade Start Time: 2018-07-26T10:59:55

Current OS Version: 8.0.0.1_build(131)style(5)

Upgrade OS Version: 8.0.0.7_build(240)style(5)

Nodes Progress:

Total Cluster Nodes: 5

Nodes On Older OS: 5

Nodes Upgraded: 0

Nodes Transitioning/Down: 0

If there are any errors, run “isi_upgrade_logs” to gather more information
  1. Check on which node it fails

# isi upgrade nodes list

Node LNN: 2

Node Upgrade State: error

Error Details: Failed Step: HookPreUpgradeOptional

Error Log: /ifs/.ifsvar/upgrade/Agent-32-000e1ea67780a95c3c57e400d1411b40b520.log

Last Upgrade Action: –

Last Action Result: –

Node Upgrade Progress: None

Node OS Version: 8.0.0.1_build(131)style(5)

  1. You will notice that the upgrade is in an error state, run “# isi_upgrade_logs” to generate the log files to check the error.
  2. View the log files generated from the nodes in the previous step, and you will see the below message at the start of the file :

isi_version_checker encountered an unexpected error.

‘utf8’ codec can’t decode bytes in position 0-2: invalid data

failed

This means there is a file under “/” [except /ifs]on the node that throws the upgrade error which is not encoded with UTF-8 or there are files under ‘/’ which have a large UID and GID.

Check directories, say “/tmp” “/var/tmp“. A simple “ls -al” under these directories should help check files with large UID and GID’s.

# ls -al /tmp

-rw——- 1 0 0 – 0 Apr 7 10:55 apache2.py.lock

-rw-r–r– 1 0 0 – 0 Apr 7 09:32 cpool_stat_cpool_query_stat_map_

-rw-r–r– 1 0 0 – 0 Apr 7 09:32 cpool_stat_cpool_stat_map_

-rw——- 1 0 0 – 0 Apr 7 10:55 generic_copy.py.lock

-rw——- 1 0 0 – 0 Apr 7 11:14 hostupdate.py

-rw-r–r– 1 0 0 – 0 Apr 7 10:55 isi_flexnet_d_sem_agg_update.key

-rw-r–r– 1 0 0 – 0 Apr 7 10:53 isi_pkg_outcmdck1

-rw-r–r– 1 0 0 – 0 Apr 7 10:49 isi_pkg_outcmderr

-rw——- 1 1000002 1000000 – 5045 Jul 23 08:09 krb5cc_1000002 >> Files with large UID and GID

-rw——- 1 1000376 1000000 – 4677 Jul 13 07:44 krb5cc_1000376

-rw——- 1 0 0 – 0 Apr 7 09:32 metadata-generator

-rw——- 1 0 0 – 0 Apr 7 11:10 nsswitch.py.lock

-rw——- 1 0 0 – 0 Jul 21 07:24 ntpd.py.lock

-rw——- 1 0 0 – 524 Apr 7 10:54 portmap.

To check if there are files with non printable characters (files not encoded with UTF-8) use the below command, the command will take time to time to return an output :

# isi_for_array -s “find / -type f -not -path ‘/ifs/*’| grep -E ‘.*[^[:print:]].*'”

Once you identify the files move it under “/ifs/data/Isilon_Support” with user permission and retry the upgrade.

# isi upgrade retry-last-action –nodes=all

[B]If there is an assessment which is giving an error, follow the below steps:

  1. Generate the assessment report by running ” # isi_upgrade_logs -a “ :

# isi_upgrade_logs -a

For Assessment with Upgrade ID [2018-07-26T09:52:19.000Z]

Assessment errors found attempting to find and extract logs from archive

Assessment Archive File found here [/ifs/.ifsvar/upgrade/archive/UpgradeArchive-2018-07-26T09:57:30.000Z.tgz]

Extracting [UpgradeArchive-2018-07-26T09:57:30.000Z/Agent-32-000e1ea67780a95c3c57e400d1411b40b520.log] to [/ifs/upgrade_error_logs/Agent-32-000e1ea67780a95c3c57e400d1411b40b520.log]

Extracting [UpgradeArchive-2018-07-26T09:57:30.000Z/32-000e1ea67780a95c3c57e400d1411b40b520/logs/hooklogs/pre-upgrade-optional/isi_upgrade_checker.log] to [/ifs/upgrade_error_logs/32-000e1ea67780a95c3c57e400d1411b40b520/logs/hooklogs/pre-upgrade-optional/isi_upgrade_checker.log]

Extracting [UpgradeArchive-2018-07-26T09:57:30.000Z/Assessment-summary.report] to [/ifs/upgrade_error_logs/Assessment-summary.report]

Suggest running ‘grep -ilr error /ifs/upgrade_error_logs/*’ to find the log files with errors

  1. Follow the suggestion in the assessment report, run ‘grep -ilr error /ifs/upgrade_error_logs/*‘ to find the log files with errors, and you can view the assessment summary report [cat /ifs/upgrade_error_logs/Assessment-summary.report] to verify which node the upgrade assessment is pointing the error.
  2. While viewing the log files if you see the error “utf8′ codec can’t decode bytes“, follow the steps mentioned in [A] on the node on which the assessment fails.

UTF8 (U from Universal Character Set + Transformation Format—8-bit) is a character encoding capable of encoding all possible characters (called code points) in Unicode.

Related:

  • No Related Posts

Error when migrating 15.1 “Failed to run migration action “UpgradeReadinessTool””

I need a solution

We get below error during migrating 14.5 to 15.1 ,  please let us know what we are missing.
We have placed the Readiness extract in URT folder (without any sub folders). all seems fine in Readiness tool as well.
 
 
 
SQLPlus standard output end
Aug 29, 2018 9:06:36 AM com.symantec.dlp.migrationcommon.MigrationActionsExecutor executeMigrationActions
SEVERE: Failed to execute migration actions
com.symantec.dlp.migrationcommon.MigrationException: Failed to run migration action “UpgradeReadinessTool”
at com.symantec.dlp.migrationcommon.MigrationActionsExecutor.runMigrationAction(MigrationActionsExecutor.java:54)
at com.symantec.dlp.migrationcommon.MigrationActionsExecutor.executeMigrationActions(MigrationActionsExecutor.java:100)
at com.symantec.dlp.migrationcommon.MigrationUtility.runMigrationUtility(MigrationUtility.java:115)
at com.symantec.dlp.migrationcommon.MigrationUtility.runMigrationUtility(MigrationUtility.java:70)
at com.symantec.dlp.enforceservermigrationutility.EnforceServerMigrationUtility.runMigration(EnforceServerMigrationUtility.java:17)
Caused by: com.symantec.dlp.migrationcommon.MigrationException: Failed to execute prechecker SQL script
at com.symantec.databasemigration.UpdateReadinessProvider.precheck(UpdateReadinessProvider.java:104)
at com.symantec.dlp.enforceservermigrationutility.actions.UpgradeReadinessToolMigrationAction.runAction(UpgradeReadinessToolMigrationAction.java:70)
at com.symantec.dlp.migrationcommon.MigrationActionsExecutor.runMigrationAction(MigrationActionsExecutor.java:47)
… 4 more
Caused by: java.lang.Exception: SQLPlus error: 
no rows selected
0

Related:

  • No Related Posts

ViPR SRM 4.0: “TooLongFrameException” SEVERE errors in Load Balancer collector log on host with XtremIO collector

Article Number: 495886 Article Version: 2 Article Type: Break Fix



ViPR SRM 4.0,ViPR SRM 4.0.1,ViPR SRM 4.0.2

On a collector host that has an XtremIO collector service installed, the collecting-0-0 log of the Load Balancer Connector (LBC) may log the following repeated error messages:

SEVERE — [2017-01-18 12:27:41 CET] — ApplicationDataForwarder::unhandledExceptionCaught(): An unhandled error occured on channel [id: 0xf2cdf353, /127.0.0.1:47796 => /127.0.0.1:2020]

io.netty.handler.codec.TooLongFrameException: frame length (4861) exceeds the allowed maximum (4096)

There is no data missing in any XtremIO reports and all reports populate fully. There are no error messages in XtremIO collection logs.

The messages in the Load Balancer Connector logs may be caused by any collector on the same host. In one instance, the XtremIO collector was identified as the source. Within the XtremIO collector, a metric called ‘NoOfVol’ is collected which refers to the number of volumes contained within Consistency Groups and Snapshots. A property ‘vollist’ contains details (WWN’s) of all volumes in these Consistency Groups and Snapshots. Where the number of volumes is unusually large (typically over 100), the excessive length of the ‘vollist’ property causes the length of the overall metric to exceed default limits in the environment. The affected metrics will be dropped at the Load Balancer level and cause the SEVERE error messages.

Dell EMC is evaluating the requirements to address this issue in an upcoming product release.

Until a fix is available, environments encountering this issue can have a workaround applied by Dell EMC Customer Service (CS) for ViPR SRM. To request this workaround, open a Service Request (SR), and reference this article number in your SR.

Related:

  • No Related Posts

“error 2381:directory does not exist” while upgrading SEP Version/ installing SEP client

I need a solution

Getting error “error 2381:directory does not exist: C:WindowsSystem32driversSEPE000F3944D.105×64” while installing Symantec Endpoint protection on one of our machine with server 2008 OS.

eARLIER sep vERSION INSTALLED WAS 14 MP1. Now we are trying to upgrade the same to 14RU1 MP1. It is not getting successful. Hence uninstalled older version using cleanwipe, rebooted server. Tried installing Unmanaged setup, MSI Setup, but every time getting above error.

0

Related:

  • No Related Posts

Heath check ciritical- “No Input”

I need a solution

Hi Team,

In that proxy all the health check parameter showing “No Input” for most of the health check parameter.

I have refered the below KB articles but the error message still there and proxy status critical.

https://support.symantec.com/en_US/article.TECH245…

https://support.symantec.com/en_US/article.TECH248…

Please find the attached error screenshot.

Please advise.

Thanks,

Ram.

0

Related:

  • No Related Posts

“NotifyUser object” action generates a CPL error

I need a solution

Hi,

On my proxySG I needed to define a new “NotifyUser object” action.

I used only by default parametres (see attached)

Even before using the new object, I get always this error when compiling CPL :

Policy installation
Compiling new configuration file: Inline configuration
Tue, 21 Aug 2018 09:54:30 UTC
Warning: Empty layer removed: ‘
<Proxy>  condition=!__is_notify_internal

Then in The CPL code I can see :

define condition __is_notify_internal
    url=http://notify.bluecoat.com
    url=’/notified-NotifyUser1?’
    url=’/verify-NotifyUser1?’
end

Any help please ?

0

Related:

  • No Related Posts

VIPR Controller 3.0: ViPR Backup Fails to Upload to the FTP Server “Error 30006: Error occurred while retrieving object Upload status”[1]

Article Number: 493163 Article Version: 3 Article Type: Break Fix



ViPR Controller Controller 3.0,ViPR Controller Controller 2.4 SP1,ViPR Controller Controller 3.5

Reviewed the ViPR Logs and verified the following error below from the Portsvc.log:

Error from Portsvc.log:

<?xml version=”1.0″ encoding=”UTF-8″ standalone=”yes”?><error><code>30006</code><description>Error occurred while retrieving object</description><details>Error occurred while retrieving object Upload status</details><retryable>false</retryable></error>

vipr1 XXXX-XXXX.XXXX.XXXX.com portalsvc 2016-11-12 18:20:33,213 [play-thread-6] ERROR ExceptionOnErrorFilter.java (line 53) Error 30006: Error occurred while retrieving object Upload status

Internal Server Error (500) for request GET /backup/itemsjson?ids=EMC-Backup-1-3.0,EMC-Backup2-1-3.0-2016,EMC-Backup-1-3.0-testbackup,EMCLAB4-Backup-5-3.0,EMCProductio-Backup-1.3.4,EMC-Customer-Backup-1-5,viprEMC-Backup-9-3.0

Execution exception

ServiceErrorException occured : Error 30006: Error occurred while retrieving object Upload status

play.exceptions.JavaExecutionException: Error 30006: Error occurred while retrieving object Upload status

at play.mvc.ActionInvoker.invoke(ActionInvoker.java:228)

at Invocation.HTTP Request(Play!)

Caused by: com.emc.vipr.client.exceptions.ServiceErrorException: Error 30006: Error occurred while retrieving object Upload status

at com.emc.vipr.client.impl.jersey.ExceptionOnErrorFilter.handle(ExceptionOnErrorFilter.java:41)

at com.emc.vipr.client.impl.jersey.ProxyTokenFilter.handle(ProxyTokenFilter.java:27)

at com.emc.vipr.client.impl.jersey.AuthTokenFilter.handle(AuthTokenFilter.java:34)

at com.sun.jersey.api.client.Client.handle(Client.java:648)

at com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)

at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)

at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)

at com.emc.vipr.client.impl.RestClient.getURI(RestClient.java:229)

at com.emc.vipr.client.core.Backup.getBackup(Backup.java:33)

at util.BackupUtils.getBackup(BackupUtils.java:48)

at controllers.Backup.itemsJson(Backup.java:61)

at play.mvc.ActionInvoker.invokeWithContinuation(ActionInvoker.java:524)

at play.mvc.ActionInvoker.invoke(ActionInvoker.java:475)

at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:451)

at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:446)

at play.mvc.ActionInvoker.invoke(ActionInvoker.java:160)

… 1 more

When clicking Upload, the Upload Status shows refreshes and shows “Not Started”

CSH

The FTP User Account Password and Rights was changed.

Resolution:

1. Please have the FTP Admin verify the correct Username and password.

SSH into the ViPR andTest SFTP to the FTP Server:

Example: sftp vipr@emc.ftp.com

Enter the Password and verify connection.

2. Verify on the FTP side if the User has Rights to the Folder and Directory from the

FTP Server side.

This article has been promoted as HVC on DELL EMC Community Network (ECN): https://community.emc.com/docs/DOC-60074

Related:

  • No Related Posts

Unable to deploy custom receiver from Store front. Getting ” An error occurred while saving changes on the “Deploy Citrix Receiver” property dialog. Please check the log in event viewer and try again.”

While trying to add customized “receiver.exe” to deploy receiver option in Storefront, you might get following error while saving it.

“An error occurred while saving changes on the “Deploy Citrix Receiver” property dialog. Please check the log in event viewer and try again”.

It works fine with default receiver.

This was the event:

Log Name: Citrix Delivery Services

Source: Citrix Delivery Services Admin

Event ID: 1

Description:

An error occurred running the command: ‘Update-DSWebReceiverHTML5Config’

Filepath ‘C:Program FilesCitrixReceiver StoreFrontReceiver ClientsWindowsFLExternalTest.EXE’ does not contains a filename.

At C:Program FilesCitrixReceiver StoreFrontManagementCmdletsWebReceiverModule.psm1:1658 char:41

+ $SourceTypeForInstallerForWindows = GetReceiverInstallerSourceType -Installe …

+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Filepath ‘C:Program FilesCitrixReceiver StoreFrontReceiver ClientsWindowsFLExternalTest.EXE’ does not contains a filename.

Citrix.DeliveryServices.PowerShell.Command.RunnerInterfaces.Exceptions.PowerShellExecutionException, Citrix.DeliveryServices.PowerShell.Command.RunnerInterfaces, Version=3.12.0.0, Culture=neutral, PublicKeyToken=e8b77d454fa2a856

An error occurred running the command: ‘Update-DSWebReceiverHTML5Config’

Filepath ‘C:Program FilesCitrixReceiver StoreFrontReceiver ClientsWindowsFLExternalTest.EXE’ does not contains a filename.

At C:Program FilesCitrixReceiver StoreFrontManagementCmdletsWebReceiverModule.psm1:1658 char:41

+ $SourceTypeForInstallerForWindows = GetReceiverInstallerSourceType -Installe …

+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

System.Management.Automation.ActionPreferenceStopException, System.Management.Automation, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35

The running command stopped because the preference variable “ErrorActionPreference” or common parameter is set to Stop: Filepath ‘C:Program FilesCitrixReceiver StoreFrontReceiver ClientsWindowsFLExternalTest.EXE’ does not contains a filename.

System.Management.Automation.Interpreter.InterpretedFrameInfo: System.Management.Automation.Interpreter.InterpretedFrameInfo[]

at System.Management.Automation.Internal.PipelineProcessor.SynchronousExecuteEnumerate(Object input, Hashtable errorResults, Boolean enumerate)

at System.Management.Automation.PipelineOps.InvokePipeline(Object input, Boolean ignoreInput, CommandParameterInternal[][] pipeElements, CommandBaseAst[] pipeElementAsts, CommandRedirection[][] commandRedirections, FunctionContext funcContext)

at System.Management.Automation.Interpreter.ActionCallInstruction`6.Run(InterpretedFrame frame)

at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)

Related:

  • No Related Posts