OneFS: Process httpd of service isi_webui has failed to restart after multiple attempts

Article Number: 499006 Article Version: 8 Article Type: Break Fix



Isilon OneFS

> The onefs WebUI will be inaccessible if the webui_httpd.conf file under /var/run/apache2/ became stale on the problematic node

isilon-1# ps axuww | grep httpd | grep -v grep

root 43633 0.0 0.5 170644 10188 – Ss 2:49PM 0:00.06 /usr/local/apache2/bin/httpd -k start

daemon 43634 0.0 0.5 170332 9840 – I 2:49PM 0:00.00 /usr/local/apache2/bin/httpd -k start

daemon 43638 0.0 0.6 174360 11652 – I 2:49PM 0:00.00 /usr/local/apache2/bin/httpd -k start

<<< the output is missing “/usr/local/apache2/bin/httpd -f /usr/local/apache2/conf/webui_httpd.conf -k start” thread. There should be 3 or more threads on each node

> Attempting to manually start httpd of isi_webui service would result in the following message:

isilon-1# /usr/local/sbin/webui_httpd_ctl start

httpd: Could not reliably determine the server’s fully qualified domain name, using 27.27.27.102 for ServerName

httpd (pid 72923) already running

isilon-1# cat /var/apache2/run/webui_httpd.pid

72923

If a node is restarted unexpectedly or if the webui process is killed in certain ways, the pid files are left. When webui is restarted either via MCP or via a node reboot, webui may not start correctly, emitting the error:> httpd (pid 72923) already running(where 72923 was the pid of the previously-running webui daemon)

> SSH to the problematic node with “root” account.

> Verify if webui_httpd.pid exists:

# ls -l /var/apache2/run/webui_httpd.pid

– If it does, remove that .pid file with the below command:

# cd /var/apache2/run/; rm -f webui_httpd.pid

Attempt to start httpd again with

# /usr/local/sbin/webui_httpd_ctl start

> After following the steps above, the output should be similar to the below where there should be 3 or more httpd of webui threads.

jv8004-1# ps axuww | grep “webui_httpd.conf” | grep -v grep

root 3135 0.0 0.6 261704 13280 – Ss 4Apr18 0:18.62 /usr/local/apache2/bin/httpd -f /usr/local/apache2/conf/webui_httpd.conf -k start

daemon 3140 0.0 0.6 261668 13268 – I@ 4Apr18 0:00.23 /usr/local/apache2/bin/httpd -f /usr/local/apache2/conf/webui_httpd.conf -k start

daemon 3145 0.0 1.5 307568 30924 – I 4Apr18 0:10.58 /usr/local/apache2/bin/httpd -f /usr/local/apache2/conf/webui_httpd.conf -k start

– Try log in to the web administration on problematic node after the above steps are taken.

Related:

Leave a Reply