Isilon OneFS 8.1 Error enabling ESRS gateway: “Error response code from gateway: 401, Unauthorized”[1]

Article Number: 511087 Article Version: 7 Article Type: Break Fix



EMC Secure Remote Services Virtual Edition,EMC Secure Remote Services Virtual Edition 3.02,EMC Secure Remote Services Virtual Edition 3.04,EMC Secure Remote Services Virtual Edition 3.06,EMC Secure Remote Services Virtual Edition 3.08

Isilon OneFS 8.1 has two possible ESRS configurations, one for ESRS v2, and one for ESRS v3. The v3 requires a username and password. Attempting to enable the v3 configuration with the wrong user name and password will result in the ESRS gateway sending ‘401’ errors back to the cluster.

Example of the error when enabling the ESRS v3 configuration on the cluster:

cluster-1# isi esrs modify –enabled=true –username=user –password=EMC123

Internal error: ESRSBadCodeError: ESRS Add Device failed. Error response code from gateway: 401, Unauthorized. Response dictionary was: {{‘http_response_code’: 401, u’responseCode’: 401, u’serialNumber’: u’ELMISL######’, u’gatewaySerialNumber’: u’########’, u’veType’: u’Connected’, u’message’: u’Invalid username and password.’, u’model’: u’ISILON-GW’}}

messages log

cluster-1: 2017-10-03T11:30:42-05:00 <3.3> cluster-1 isi_rsapi_d[5314]: [0x800704000]: Proxy response contained errors, ESRS was not enabled.

cluster-1: 2017-10-03T11:30:43-05:00 <3.3> cluster-1 isi_esrs_d[75337]: ERROR: tardis ipc error: TARDIS_IPC_ERR_CONSTRAINT_VIOLATION; error while collecting response: UNIQUE constraint failed: kv_table.key, kv_table.os_version (19)

cluster-2: 2017-10-03T11:30:41-05:00 <3.3> cluster-2 isi_esrs_d[48808]: ERROR: tardis ipc error: TARDIS_IPC_ERR_CONSTRAINT_VIOLATION; error while collecting response: UNIQUE constraint failed: kv_table.key, kv_table.os_version (19)

cluster-2: 2017-10-03T11:30:42-05:00 <3.3> cluster-2 isi_rsapi_d[9470]: [0x800704400]: pcm_add_device python call failed: Traceback (most recent call last): File “/usr/local/lib/python2.7/site-packages/isi/esrs/restclient.py”, line 877, in pcm_add_device File “/usr/local/lib/python2.7/site-packages/isi/esrs/restclient.py”, line 842, in _gateway_failover File “/usr/local/lib/python2.7/site-packages/isi/esrs/restclient.py”, line 396, in _pcm_post File “/usr/local/lib/python2.7/site-packages/isi/esrs/restclient.py”, line 106, in _check_return_code ESRSBadCodeError: ESRS Add Device failed. Error response code from gateway: 401, Unauthorized. Response dictionary was: {{‘http_response_code’: 401, u’responseCode’: 401, u’serialNumber’: u’ELMISL#######’, u’gatewaySerialNumber’: u’############’, u’veType’: u’Connected’, u’message’: u’Invalid username and password.’, u’model’: u’ISILON-GW’}}

isi_esrs_d.log

cluster-1: 2017-10-03T11:30:43-05:00 <3.6> cluster-1 isi_esrs_d[75337]: INFO: Node 1 updating registered node tardis variables

cluster-1: 2017-10-03T11:30:43-05:00 <3.3> cluster-1 isi_esrs_d[75337]: ERROR: tardis ipc error: TARDIS_IPC_ERR_CONSTRAINT_VIOLATION; error while collecting response: UNIQUE constraint failed: kv_table.key, kv_table.os_version (19)

Attempting to enable with debug command

cluster-1# isi –debug esrs modify –enabled=true –username=user –password=EMC123

2017-10-03 11:37:46,000 DEBUG rest.py:70: >>>GET [‘remote-service’, ‘5’, ‘esrs’, ‘status’]

2017-10-03 11:37:46,000 DEBUG rest.py:72: args={}

body={}

2017-10-03 11:37:46,001 DEBUG rest.py:92: <<<(200, {‘status’: ‘200 Ok’, ‘content-type’: ‘application/json’, ‘allow’: ‘GET, PUT, HEAD’}, ‘n{n”esrs” : n{n”connectivity_status” : “Disconnected”,n”enabled” : falsen}n}n’)

2017-10-03 11:37:46,001 DEBUG rest.py:70: >>>PUT [‘remote-service’, ‘5’, ‘esrs’, ‘status’]

2017-10-03 11:37:46,001 DEBUG rest.py:72: args={}

body={‘username’: ‘root’, ‘password’: ‘********’, ‘enabled’: True}

2017-10-03 11:37:48,691 DEBUG rest.py:92: <<<(500, {‘status’: ‘500 Internal Server Error’, ‘content-type’: ‘application/json’, ‘allow’: ‘GET, PUT, HEAD’}, ‘n{n”errors” : n[nn{n”code” : “AEC_EXCEPTION”,n”id” : 3,n”lnn” : 3,n”message” : “Internal error: ESRSBadCodeError: ESRS Add Device failed. Error response code from gateway: 401, Unauthorized. Response dictionary was: {{‘http_response_code’: 401, u’responseCode’: 401, u’serialNumber’: u’ELMISL#######’, u’gatewaySerialNumber’: u’##########’, u’veType’: u’Connected’, u’message’: u’Invalid username and password.’, u’model’: u’ISILON-GW’}}\n”n}n]n}n’)

Internal error: ESRSBadCodeError: ESRS Add Device failed. Error response code from gateway: 401, Unauthorized. Response dictionary was: {{‘http_response_code’: 401, u’responseCode’: 401, u’serialNumber’: u’ELMISL0817J7V3′, u’gatewaySerialNumber’: u’ELMESR0617VH56′, u’veType’: u’Connected’, u’message’: u’Invalid username and password.’, u’model’: u’ISILON-GW’}}

2017-10-03 11:37:48,691 DEBUG isi:375: Internal error: ESRSBadCodeError: ESRS Add Device failed. Error response code from gateway: 401, Unauthorized. Response dictionary was: {{‘http_response_code’: 401, u’responseCode’: 401, u’serialNumber’: u’ELMISL########’, u’gatewaySerialNumber’: u’#############’, u’veType’: u’Connected’, u’message’: u’Invalid username and password.’, u’model’: u’ISILON-GW’}}

An associated traceback may also be seen

Traceback (most recent call last):

File “/usr/bin/isi”, line 383, in <module>

run_cli(controllers.isi)

File “/usr/bin/isi”, line 318, in run_cli

level = execute(module, argv, False)

File “/usr/bin/isi”, line 232, in execute

level.execute(data, optvals, **optvals)

File “/usr/local/share/CLI/decorators.py”, line 95, in __call__

super(modify_controller, self).__call__(data, optvals, **kwargs)

File “/usr/local/share/CLI/decorators.py”, line 57, in __call__

self.execute(data, **kwargs)

File “/usr/local/share/CLI/controllers/isi/esrs/modify.py”, line 78, in execute

status_data.put(d)

File “/usr/local/share/CLI/papi/container.py”, line 200, in put

self.last_result = self.conn.transact(papi.PUT, self.uri, args=args, body=value)

File “/usr/local/share/CLI/papi/rest.py”, line 124, in transact

raise RestError(response, message, error_fields)

papi.rest.RestError: Internal error: ESRSBadCodeError: ESRS Add Device failed. Error response code from gateway: 401, Unauthorized. Response dictionary was: {{‘http_response_code’: 401, u’responseCode’: 401, u’serialNumber’: u’ELMISL########’, u’gatewaySerialNumber’: u’##########’, u’veType’: u’Connected’, u’message’: u’Invalid username and password.’, u’model’: u’ISILON-GW’}}

Using incorrect credential when attempting to enable the ESRS configuration on the cluster.

Refer to https://support.emc.com/kb/510767 – Isilon: How To set up ESRS for OneFS v8.1+. It also includes information on troubleshooting new installs.

The issue is outside of Isilon’s control and is dependent upon configuration between ESRS and the Support EMC site. The error showing in the cluster logs, are actually errors the ESRS gateway is sending back to Isilon.

Enabling the ESRS v3 configuration on the cluster requires using the same Support EMC site credentials. The Support EMC credentials are the same ones used when logging into the website to manager Service Requests, search KBs, or download files.

If the credentials being used do not match that of the site, or if the credentials aren’t working or unknown. Engage with the Account team responsible for the site that is using the ESRS gateway. Changes may be required or related to ESRS itself, updates to the Customer Site information, the Install Base, or even changes to the account privileges associated with the Support EMC site.

Also, for the ESRS v3 configuration to work on Isilon, requires a signed and valid license.

Related: