Failover / HA For Zend Job Queue

[rampog]

Hi,

currently we have planing for migrating our technology to be windows / microsoft base technology
prior new regulation we are pretty familiar with unix/linux environment such us for doing offline process using cron job.
but i'm not familiar with task scheduler that windows provide.
i've do some research zend job queue are potensial our way for doing this. & suitable solution that we doesn't have to aware of OS
(applicaition transparancy) if something happen that made us to migrate again on other operating systems.
which is zend server run on of those.


i've several question related zend job queue.


if i see the documentation of shahar architecture the daemon (zend job queue daemon) will trigger execution running on http request to backend server (url)
that could be distribute procces as equal as our web application server using NLB & cluster

does zend joq queue daemon it self could be HA or failover. ? if daemon / server fail.
how to recover this situation ?


Please More Advice

[zvika]

Hi

Within a Zend Cluster, there are several Jobqueue daemons / services which pulls jobs from a centralized database.
This gives you both multiple executors and target multiple target URLs when landing on Load Balanced HTTP job target.

Jobqueue daemons have verbose configuration to limit/expand capacity of each executor (jqd.ini):

; The maximum number of HTTP based jobs which can be executed simultaneously by single back-end server
zend_jobqueue.max_http_jobs=4

; Number of seconds while daemon trying to establish a connection with back-end server (units: seconds)
zend_jobqueue.connection_timeout=30

; Number of seconds while URL based job must complete. (units: seconds)
zend_jobqueue.http_job_timeout=120

; The minimal number of milliseconds between job startups.
zend_jobqueue.job_restart_timeout=200

; Number of retries in case of HTTP job failure.
zend_jobqueue.http_job_retry_count=10

; The number of seconds between retries of failed HTTP jobs. (units: seconds)
zend_jobqueue.http_job_retry_timeout=1

Check Zend Server Online Reference - Jobs Component for more information.
Let me know if this is the information you seek.

[rampog]

Hi Zvika,

oke thanks for great explanation related job queue. so actually it's already aware about failover.
there is centralize database for queue management in MySQL DB & it's already support ACID compliance transactional database.
so multiple daemon of each node server will not both execute same job.

& the challenge is how if MySQL fail which this is out side Zend Concern it is take concern on Database Vendor.
how to mitigate this, it might be to setup cluster of MySQL.

oke lastly how to set predefined configuration for job.
is there any GUI could be use.
& how to know existing job that already configure.
add & remove configuration of job.
i could not see in zend server gui
my poor knowledge is we have to execute php program like below

Code: Select all

$queue = new ZendJobQueue();
$params = array("data" => "Test");
        $options = array("priority" => ZendJobQueue::PRIORITY_HIGH);

        // execute the job in two minutes unless the priority is urgent
       
            $options["schedule"] = "*/1 * * * *";
       

        $jobID = $queue->createHttpJob("http://localhost/test/execQueue.php?testcron", $params, $options);


but it seem not proper in my opinion..


please more advice

[zvika]

Hi

I believe both your questions can be answered by using Jobqueue API,
the GUI shows the jobs list and you can set/re-queue jobs, but when you add jobs programmatically,
you can also query them and modify / re-queue using the Jobqueue API.
Zend Job Queue - PHP API

[rampog]

HI Zvika,
i got clarification that zend gui does not provide for add job scheduler / set add,delete schedule.
it can be done only programatically by using ZendJobQueue PHP API.
ok tks for support

[zvika]

Hi

The ZS GUI (in version 5) can add recurring jobs, which are normally made for maintenance / cleanup DB or transactions etc.
Normal jobs offloaded from your application needs to be inserted programmatically of course, it is intended for this purpose.

You might like to check Zend Server 6 beta or when it is becomes GA, those features are enhanced.