Restart PHP does not work

[kpapst]

Hi all!

I have a similar problem like the one mentioned in the last forum thread: Cluster deploy applications

Whe n I deploy an application on our CM (with two nodes), none of the Apache configs is reloaded, so I still see the old application version.
I am pretty sure that it has to be a problem with our setup, but I researched for two days now and couldn't find a hint on whats going wrong.
None of the nodes refreshes its Apache configs, neither after deployment nor when I hit the "Restart (at Cluster Setup > Servers screen).

When I either use

Code: Select all

/etc/init.d/httpd reload

or

Code: Select all

/etc/init.d/httpd graceful

I instantly see the new application version.

What I did so far:

I checked the zdd.log (this is from one of the nodes)

Code: Select all

[20.11.2012 15:05:34 DBG1] ZDDeploymentManager: upgrading app http://customersdomain/servertools
[20.11.2012 15:05:34 DBG1] Extracted package will be at /usr/local/zend/tmp/zdd/.TJ5920
[20.11.2012 15:05:34 DBG1] Extracting /usr/local/zend/tmp/qt_temp.XM5920 into /usr/local/zend/tmp/zdd/.TJ5920
[20.11.2012 15:05:34 DBG1] Updating http://customersdomain/servertools status to DEACTIVATING with the message Deactivating current application (version 0.2.122)
[20.11.2012 15:05:34 DBG1] Removing alias /usr/local/zend/etc/sites.d/http/customersdomain/80/98_servertools.conf
[20.11.2012 15:05:34 DBG1] Updating http://customersdomain/servertools status to UPLOADING
[20.11.2012 15:05:34 DBG1] Extracted package will be at /usr/local/zend/tmp/zdd/.nS5920
[20.11.2012 15:05:34 DBG1] Extracting /usr/local/zend/tmp/qt_temp.kn5920 into /usr/local/zend/tmp/zdd/.nS5920
[20.11.2012 15:05:34 DBG1] Updating http://customersdomain/servertools status to STAGING
[20.11.2012 15:05:34 DBG1] Zend Params: baseUrl='http://customersdomain/servertools', createVhost='1', ignoreFailures='0', userApplicationName='Tools'
[20.11.2012 15:05:34 DBG1] User Params: domain='preprod', environment='development'
[20.11.2012 15:05:34 DBG1] Checking for global script /pre_stage.php
[20.11.2012 15:05:34 DBG1] Extracted package scripts are in /usr/local/zend/tmp/zdd/.Ti5920
[20.11.2012 15:05:34 DBG1] Executing pre_stage.php
[20.11.2012 15:05:34 DBG1] Execution of  pre_stage.php  was successful
[20.11.2012 15:05:34 DBG1] Application version is 0.2.123
[20.11.2012 15:05:34 DBG1] Checking for global script /post_stage.php
[20.11.2012 15:05:34 DBG1] Extracted package scripts are in /usr/local/zend/tmp/zdd/.Ti5920
[20.11.2012 15:05:34 DBG1] Executing post_stage.php
[20.11.2012 15:05:34 DBG1] Execution of  post_stage.php  was successful
[20.11.2012 15:05:34 DBG1] Updating http://customersdomain/servertools status to ACTIVATING
[20.11.2012 15:05:34 DBG1] ZDDeploymentManager::activate called with baseUrl= http://customersdomain/servertools execRunOnceNodeID= 13
[20.11.2012 15:05:34 DBG1] Checking for global script /pre_activate.php
[20.11.2012 15:05:34 DBG1] Extracted package scripts are in /usr/local/zend/tmp/zdd/.Ti5920
[20.11.2012 15:05:34 DBG1] Executing pre_activate.php
[20.11.2012 15:05:34 DBG1] Execution of  pre_activate.php  was successful
[20.11.2012 15:05:34 DBG1] modifying Apache configuration for deployment of base url http://customersdomain/servertools
[20.11.2012 15:05:34 DBG1] ZDDeploymentManager::doRestartWebserver() Restarting the web server
[20.11.2012 15:05:34 DBG1] ZDDeploymentManager::doRestartWebserver() Restarted the web server succesfully
[20.11.2012 15:05:34 DBG1] Checking for global script /post_activate.php
[20.11.2012 15:05:34 DBG1] Extracted package scripts are in /usr/local/zend/tmp/zdd/.Ti5920
[20.11.2012 15:05:34 DBG1] Executing post_activate.php
[20.11.2012 15:05:35 DBG1] Execution of  post_activate.php  was successful
[20.11.2012 15:05:35 DBG1] Updating http://customersdomain/servertools status to ACTIVE
[20.11.2012 15:05:35 DBG1] Found 0 persistent resources
[20.11.2012 15:05:35 DBG1] ZMEventRules::loadFromXml
[20.11.2012 15:05:35 DBG1] Removing application install dir for http://customersdomain/servertools version 0.2.121
[20.11.2012 15:05:35 DBG1] Removing directory /usr/local/zend/var/apps/http/customersdomain/80/servertools/0.2.121
[20.11.2012 15:05:35 NOTICE] Application on http://customersdomain/servertools was successfully upgraded


As you can see it mentions:

Code: Select all

[20.11.2012 15:05:34 DBG1] ZDDeploymentManager::doRestartWebserver() Restarting the web server
[20.11.2012 15:05:34 DBG1] ZDDeploymentManager::doRestartWebserver() Restarted the web server succesfully

but I can't see any Apache log entry regarding this restart.

When I use the init.d script directly (as mentioned above) I see major changes in the process list (less apache processes and changed IDs). But when I check the process list before and after the deployment (and also before and after "Restart All") via

Code: Select all

ps aux | grep httpd

or as mentioned in your online FAQs

Code: Select all

ps aux | grep -e zend -e apache -e httpd

they do not change in any way.

So I am pretty sure that the Webserver is not reacting on the ZDDeploymentManager::doRestartWebserver() call.

I also checked the Zend global directive config (again from one of the nodes) at /usr/local/zend/etc/conf.d/ZendGlobalDirectives.ini

Code: Select all

; UID of httpd process
zend.httpd_uid=48

; GID of httpd process
zend.httpd_gid=48


the values match:

Code: Select all

[user@server zend]$ id apache
uid=48(apache) gid=48(apache) groups=48(apache)

[user@server zend]$ id zend
uid=101(zend) gid=103(zend) groups=48(apache),103(zend)


As I do not know what ZDDeploymentManager::doRestartWebserver() does niternally I checked some file permissions:

Code: Select all

[user@server zend]$ ls -la /etc/init.d/httpd
-rwxr-xr-x 1 root root 3335 Dec 30  2011 /etc/init.d/httpd

[user@server zend]$ ls -la /etc/init.d/zend-server
lrwxrwxrwx 1 root root 30 Sep 21 14:40 /etc/init.d/zend-server -> /usr/local/zend/bin/zendctl .sh

[user@server zend]$ ls -la /usr/local/zend/bin/zendctl .sh
-rwxr-xr-x 1 zend zend 6973 Jul 24 13:47 /usr/local/zend/bin/zendctl .sh


(I had to add the space between zendctl and .sh to be able to post this entry - forum told me "You are not authorized to post url links, please remove or rename:zendctl .sh").

I am kind of lost here and hope someone can't point me into the right direction to find out why the restart does not happen.

Thanks in advance
Kevin

[kpapst]

Forgot to append a process list snippet:

Code: Select all

[user@server zend]$ ps aux | grep -e zend -e apache -e httpd
zend      5771  0.0  0.0  21844  1080 ?        Ss   Nov20   0:00 /usr/local/zend/bin/watchdog -c /usr/local/zend/etc/watchdog-monitor.ini -u 101 -g 103 -s monitor
zend      5773  0.0  0.0 119592  3448 ?        Ssl  Nov20   0:00 /usr/local/zend/bin/MonitorNode /usr/local/zend/etc/monitor_node.ini /usr/local/zend/etc/conf.d -e
root      5805  0.0  0.1 518108 38324 ?        Ss   Nov20   0:00 /usr/sbin/httpd
zend      5852  0.0  0.0  21848  1072 ?        Ss   Nov20   0:00 /usr/local/zend/bin/watchdog -c /usr/local/zend/etc/watchdog-sc.ini -u 101 -g 103 -s scd
zend      5853  0.0  0.0 299076  4760 ?        Ssl  Nov20   0:08 /usr/local/zend/bin/scd -p /usr/local/zend/etc/scd.ini -e
zend      5918  0.0  0.0  21844  1076 ?        Ss   Nov20   0:00 /usr/local/zend/bin/watchdog -c /usr/local/zend/etc/watchdog-deployment.ini -u 101 -g 103 -s zdd
zend      5920  0.0  0.0 166052  7020 ?        Ssl  Nov20   0:11 /usr/local/zend/bin/zdd /usr/local/zend/etc/zdd.ini -e
zend      5968  0.0  0.0  21844  1080 ?        Ss   Nov20   0:00 /usr/local/zend/bin/watchdog -c /usr/local/zend/etc/watchdog-jq.ini -u 101 -g 103 -s jqd
zend      5970  0.0  0.0  79000  4044 ?        Ss   Nov20   0:04 /usr/local/zend/bin/jqd -f /usr/local/zend/etc/jqd.ini -e
zend      6011  0.0  0.0 205784 13508 ?        Ss   Nov20   0:00 /usr/local/zend/gui/lighttpd/sbin/php -c /usr/local/zend/gui/lighttpd/etc/php-fcgi.ini
zend      6013  0.0  0.0 199988  5316 ?        S    Nov20   0:00 /usr/local/zend/gui/lighttpd/sbin/php -c /usr/local/zend/gui/lighttpd/etc/php-fcgi.ini
zend      6014  0.1  0.1 217204 35856 ?        S    Nov20   1:43 /usr/local/zend/gui/lighttpd/sbin/php -c /usr/local/zend/gui/lighttpd/etc/php-fcgi.ini
zend      6015  0.1  0.0 215216 30304 ?        S    Nov20   1:44 /usr/local/zend/gui/lighttpd/sbin/php -c /usr/local/zend/gui/lighttpd/etc/php-fcgi.ini
zend      6016  0.1  0.0 215220 30548 ?        S    Nov20   1:41 /usr/local/zend/gui/lighttpd/sbin/php -c /usr/local/zend/gui/lighttpd/etc/php-fcgi.ini
zend      6017  0.1  0.0 215252 29888 ?        S    Nov20   1:44 /usr/local/zend/gui/lighttpd/sbin/php -c /usr/local/zend/gui/lighttpd/etc/php-fcgi.ini
zend      6018  0.1  0.0 215216 29636 ?        S    Nov20   1:42 /usr/local/zend/gui/lighttpd/sbin/php -c /usr/local/zend/gui/lighttpd/etc/php-fcgi.ini
zend      6019  0.1  0.0 215224 29856 ?        S    Nov20   1:43 /usr/local/zend/gui/lighttpd/sbin/php -c /usr/local/zend/gui/lighttpd/etc/php-fcgi.ini
root      6032  0.0  0.0  21844  1076 ?        Ss   Nov20   0:00 /usr/local/zend/bin/watchdog -c /usr/local/zend/etc/watchdog-lighttpd.ini -u zend -g zend -s lighttpd
zend      6034  0.0  0.0  25200  2284 ?        Ss   Nov20   0:00 /usr/local/zend/gui/lighttpd/sbin/lighttpd -m /usr/local/zend/gui/lighttpd/lib -f /usr/local/zend/gui/lighttpd/etc/lighttpd.conf -D
apache    6169  0.0  0.2 762488 75896 ?        S    Nov20   0:09 /usr/sbin/httpd
apache    6170  0.0  0.2 762488 75296 ?        S    Nov20   0:09 /usr/sbin/httpd
apache    6171  0.0  0.2 762488 75252 ?        S    Nov20   0:09 /usr/sbin/httpd
apache    6172  0.0  0.2 762488 75600 ?        S    Nov20   0:09 /usr/sbin/httpd
apache    6173  0.0  0.2 762488 75984 ?        S    Nov20   0:09 /usr/sbin/httpd
apache    6174  0.0  0.2 762488 84592 ?        S    Nov20   0:10 /usr/sbin/httpd
... more httpd worker here ....


Any help is appreciated!