NYCPHP Meetup

[Anthony Papillion]

Hello Everyone,

I'm designing a system that will work on a schedule. Users will submit data 
for processing into the database and then, every minute, a PHP script will 
pass through the db looking for unprocessed rows (marked pending) and 
process them.

The problem is, I may eventually have a few million records to process at a 
time. Each record could take anywhere from a few seconds to a few minutes to 
perform the required operations on. My concern is making sure that the 
script, on the next scheduled pass, doesn't grab the records currently being 
processed and start processing them again.

Right now, I'm thinking of accomplishing this by updating a 'status' field 
in the database. So unprocessed records would have a status of 'pending', 
records being processed would have a status of 'processing' and completly 
processed record will have a status of 'complete'.

For some reason, I see this as ugly but that's the only way I can think of 
making sure that records aren't duplicatly processed. So when I select 
records to process, I'm ONLY selecting one's with the status of 'pending' 
which means they are new, unprocessed.

Is there a better, more eleqent way of doing this or is this pretty much it?

Thanks!
Anthony Papillion 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nyphp.org/pipermail/talk/attachments/20100214/ebaf12be/attachment.html>