Distributed wrapper for yii for davidpersson's beanstalkd client
This project differs from other yii extensions in that it provides a very trivial method of managing multiple, distributed beanstalkd servers. Other projects, or classes, differ in that they build upon existing implementations but do not offer any method of failover or recovery from a server failure.
Since a single server going down would cause complete failure this must, at the very least:
- be aware of multiple servers/connections
- be able to choose between them based upon a simple weighted algorithm
- be able to use a different server if one is unavailable In the event of a server failure, this class makes no attempt to ensure enqueued items are not lost since. This will allow the user to enqueue in another server and keep going.
Requirements ¶
- davidpersson's beanstalkd client to be added to your yii project
- modifications to config files to support these additions
Add davidpersson's beanstalkd client ¶
Clone davidpersson's library and place it in the extensions folder of your yii application, in a folder named 'beanstalk'.
Modify config files to support these additions ¶
Ensure that the Beanstalk.php class from this project has been placed in your components folder.
Ensure this line has been added under the aliases section of your yii config file:
'aliases'=>array(
// davidpersson's library has been installed to extensions/beanstalk folder
'Beanstalk'=> 'application.extensions.beanstalk.src',
),
Ensure these lines have been added to the components section of your yii config file:
'components'=>array(
'beanstalk'=>array(
'class'=>'application.components.Beanstalk',
'servers'=>array(
'server1'=>array(
'host'=>'192.168.0.10',
'port'=>11300,
'weight'=>50,
// array of connections/tubes
'connections'=>array(),
),
'server2'=>array(
'host'=>'192.168.0.11',
'port'=>11300,
'weight'=>50,
// array of connections/tubes
'connections'=>array(),
),
),
),
),
Usage ¶
Producer example ¶
/*
* connect to a server (determines by weight or selects only existing server::
*/
$client = Yii::app()->beanstalk->getClient();
$client->connect();
/*
* alternatively, one can connect to a server by name:
*/
//$client = Yii::app()->beanstalk->getClient('server1');
//$client->connect();
$client->useTube('default');
//echo "Tube used: {$client->listTubeUsed()}.\n";
//print_r($client->stats());
$jobDetails = [
'application'=>'beanstalk test',
'payload'=>'performing a test on: '.date('Y-m-d H:i:s'),
];
$jobDetailString = json_encode($jobDetails);
$ret = $client->put(
0, // priority
0, // do not wait, put in immediately
90, // will run within n seconds
$jobDetailString // job body
);
echo "Added $jobDetailString to queue.\n";
echo "Return was: $ret.\n";
$client->disconnect();
Consumer Examples ¶
Peeking ¶
$peek = Yii::app()->beanstalk->getClient();
$peek->connect();
$peek->watch('default');
$job = $peek->reserve(5);
print_r($job);
$result = touch($job['body']);
print_r($job);
print_r($result);
$peek->disconnect();
Consuming ¶
$consumer = Yii::app()->beanstalk->getClient();
$consumer->connect();
$consumer->watch('default');
while(true)
{
$job = $consumer->reserve();
$result = touch($job['body']);
if( $result )
{
// do something with the job request
echo "Done...\n";
$consumer->delete($job['id']);
}
else
{
// handle failure here
echo "Burying...\n";
$consumer->bury($job['id']);
}
}
$consumer->disconnect();
Resources ¶
Inspiration ¶
- davidpersson's beanstalk project
- Yiinstalk, a yii extension for beanstalkd
- yii2-beanstalk, a yii2 extension for beanstalkd
running the worker ?
Thanks for your extension, but I don't understand how can I run the worker ? It should be called one time, wright ? When the server starts ?
RE: running the worker?
Clem,
It has been a few years since I developed this and I ultimately decided to use a different solution which resolved a few more of my concerns, but I will do my best to explain.
To run the worker you can put the consumer code in a yii command, which you can run regularly (using cron, for example) which will check for jobs in the beanstalkd servers. The problem that I encountered that ultimately caused me to abandon this methodology was that since I had two beanstalkd servers I had to check both of them for jobs. This may not be an issue for you, but the solution I came up with involved having two workers (consumers) which ran on a schedule--each one was specifically to check a certain beanstalkd server.
So, create a command, for instance: QueueWorkerCommand, which accepts an argument for the name of the beanstalkd server and then have two entries in cron, one for each beanstalkd server. That will result in both of them getting checked so you can ensure all jobs are processed.
The reasons I was unsatisfied with my efforts were the following (these might cause you to find another solution):
An improvement which might make this whole approach worthwhile:
Alternative solutions:
I know this was a bit long and kind of discourages the use of this extension but it might be the best option for you.
RE RE: running the worker?
Thanks for your reply cottonaf, I finally understood that I need to install beanstalk packages on a server and run it, next run the worker from a command line or cronjob.
It does the job for me today, but I have read your advices and will investigate on the others solutions you expose here for other projects.
Cheers !
If you have any questions, please ask in the forum instead.
Signup or Login in order to comment.