Queue Workers not starting immediately?

For programming and general questions on Zend Framework

Queue Workers not starting immediately?

Postby mart1982 on Fri Jun 16, 2017 11:29 am

Hi all, quite new to this, so apologies in advance if I'm looking/asking in the wrong place.
I'm performing QA on an application that uses Zend for processing work.

In the 'real' application, we have up to 3 long running workers to process work (though the test system is configured to allow workers to live for a max of 10 seconds).

In our automated tests, we check to see if there are any future or current tasks, and if there are, start up workers 'on demand' that will die immediately when there is no more work to do (so that workers are fresh and new for each test - '/queue/worker?retire=true').

When there is only a small amount of work to do, we are seeing a delay between (what I think is) Zend knowing there is work to do, and work actually starting.

The output below is what we get when we query the server to see if there is any work to do on a 0.5 second retry interval (starting an on-demand worker each time until there is no work to do).
You can see there are 25 future tasks that work is not started on for several seconds, but once it's started it gets done quickly.
Code: Select all
      │    {"tasks":0,"futureTasks":25,"workers":0,"maxWorkers":3,"workerLifetime":"10s","pingError":false}
      │    {"tasks":0,"futureTasks":25,"workers":0,"maxWorkers":3,"workerLifetime":"10s","pingError":false}
      │    {"tasks":0,"futureTasks":25,"workers":0,"maxWorkers":3,"workerLifetime":"10s","pingError":false}
      │    {"tasks":0,"futureTasks":25,"workers":0,"maxWorkers":3,"workerLifetime":"10s","pingError":false}
      │    {"tasks":0,"futureTasks":25,"workers":0,"maxWorkers":3,"workerLifetime":"10s","pingError":false}
      │    {"tasks":0,"futureTasks":25,"workers":0,"maxWorkers":3,"workerLifetime":"10s","pingError":false}
      │    {"tasks":0,"futureTasks":25,"workers":0,"maxWorkers":3,"workerLifetime":"10s","pingError":false}
      │    {"tasks":25,"futureTasks":0,"workers":0,"maxWorkers":3,"workerLifetime":"10s","pingError":false}
      │    {"tasks":0,"futureTasks":0,"workers":0,"maxWorkers":3,"workerLifetime":"10s","pingError":false}



On one of our other tests, we generate a lot more work, and we can see that the work is started faster (there aren't several seconds of no work being processed)

Code: Select all
      │    {"tasks":19,"futureTasks":231,"workers":0,"maxWorkers":3,"workerLifetime":"10s","pingError":false}
      │    {"tasks":0,"futureTasks":231,"workers":0,"maxWorkers":3,"workerLifetime":"10s","pingError":false}
      │    {"tasks":46,"futureTasks":185,"workers":0,"maxWorkers":3,"workerLifetime":"10s","pingError":false}
      │    {"tasks":0,"futureTasks":185,"workers":0,"maxWorkers":3,"workerLifetime":"10s","pingError":false}
      │    {"tasks":38,"futureTasks":147,"workers":0,"maxWorkers":3,"workerLifetime":"10s","pingError":false}
      │    {"tasks":36,"futureTasks":111,"workers":0,"maxWorkers":3,"workerLifetime":"10s","pingError":false}
      │    {"tasks":0,"futureTasks":111,"workers":0,"maxWorkers":3,"workerLifetime":"10s","pingError":false}
      │    {"tasks":111,"futureTasks":0,"workers":0,"maxWorkers":3,"workerLifetime":"10s","pingError":false}
      │    {"tasks":0,"futureTasks":0,"workers":0,"maxWorkers":3,"workerLifetime":"10s","pingError":false}


I'm trying to work out why this is happening, and how to make work start faster. We have over 500 tests and this 4-5 second wait before work is processed is adding 45 minutes to the test run that we'd like to avoid.

I'm wondering if workers don't start working until [work has been in the queue for 5 seconds or there are more than 50 work items to do], or something similar. If this is the case, how would we configure things to reduce the delay?

Hope this all makes sense!

Martin.
mart1982
 
Posts: 2
Joined: Fri Jun 16, 2017 11:09 am

Re: Queue Workers not starting immediately?

Postby mart1982 on Fri Jun 16, 2017 12:08 pm

I just tried updating the tests to get them to start long running workers at the start of the test to see if that helped, and it hasn't - we still seem to have a pause where we can see there is future work to do, but nothing is happening.

Note here we have 3 workers for the entire duration, but we sit with 25 tasks not being worked on for a while.

Code: Select all
      │    {"tasks":0,"futureTasks":25,"workers":3,"maxWorkers":3,"workerLifetime":"30s","pingError":false}
      │    {"tasks":0,"futureTasks":25,"workers":3,"maxWorkers":3,"workerLifetime":"30s","pingError":false}
      │    {"tasks":0,"futureTasks":25,"workers":3,"maxWorkers":3,"workerLifetime":"30s","pingError":false}
      │    {"tasks":0,"futureTasks":25,"workers":3,"maxWorkers":3,"workerLifetime":"30s","pingError":false}
      │    {"tasks":0,"futureTasks":25,"workers":3,"maxWorkers":3,"workerLifetime":"30s","pingError":false}
      │    {"tasks":0,"futureTasks":25,"workers":3,"maxWorkers":3,"workerLifetime":"30s","pingError":false}
      │    {"tasks":0,"futureTasks":25,"workers":3,"maxWorkers":3,"workerLifetime":"30s","pingError":false}
      │    {"tasks":0,"futureTasks":6,"workers":3,"maxWorkers":3,"workerLifetime":"30s","pingError":false}
      │    {"tasks":0,"futureTasks":6,"workers":3,"maxWorkers":3,"workerLifetime":"30s","pingError":false}
      │    {"tasks":0,"futureTasks":0,"workers":3,"maxWorkers":3,"workerLifetime":"30s","pingError":false}
mart1982
 
Posts: 2
Joined: Fri Jun 16, 2017 11:09 am


Return to Zend Framework

Who is online

Users browsing this forum: No registered users and 2 guests