NYCPHP Meetup

[Jesse Callaway]

On Fri, Feb 13, 2009 at 3:31 PM, Nicholas Hart <nhart at partsauthority.com> wrote:
> I have a php 'cron job' script which is inserting some records.  It uses
> arrays to store 12 field records along with several nested loops in which it
> tests various currency values from a separate 4 field table and sorts them
> to get the lowest one for insert into a new 4 field table record.  It has
> about 640,000 Product records per 4 Customers or apporx. 2.4 million records
> to insert.  It takes several hours+ to complete and I am hoping there is a
> way to speed it up.
>
> So far, I've tried updating instead of inserting records and changing a few
> functions but nothing seems to make much difference.  Am I being
> unreasonable for the size of the files here?  I'm considering using a
> Derived Table in mySQL and will also consider another language such as
> python.  Please let me know if you have any suggestions.  Thanks.
>
> Nicholas Hart
>
>
>
>
>
> _______________________________________________
> New York PHP User Group Community Talk Mailing List
> http://lists.nyphp.org/mailman/listinfo/talk
>
> http://www.nyphp.org/show_participation.php
>

Do your inserts in chunks. If you have the RAM, do one big insert.
Otherwise you have to hit the disk for each record which can be slow
especially if you have table locks going on elsewhere.

-jesse