NYCPHP Meetup

[Brian Dailey]

I'm already looking at 100,000+ records, and I am somewhat nervous as to 
how it will perform as the load begins to get a little bit higher. I 
suppose I will have to do a little tweaking to make it work properly, 
but it seems like this approach would lend itself a little more to 
indexing rather than using the 2 layer approach.

Which leads me to my next question - Chris, in your 2 layer approach how 
do you handle indexing? If you have a fairly sizable chunk of 
YAML/XML/etc data, how do you handle searching it?

Rolan Yang wrote:
> Brian Dailey wrote:
>> This looks like what I'll ultimately be doing. I played around with 
>> some SQL queries and what I ended up with was similar to this - 
>> joining the table for each data value that I wanted to look up (and 
>> since I expect a matching data value, I don't even have to LEFT JOIN 
>> everything, which speeds it up significantly).
>>
>> Thanks for all of your help and suggestions!
>>
> 
> You may eventually reach a point where your data volume becomes 
> monstrous and the queries start to take an unacceptably long time to 
> run, or consume too much CPU, or maybe even lock up the tables from 
> being written to.  At that point (or before) you may want to consider 
> setting up a second mysql machine as as replicated slave - dedicated for 
> generating reports. Once you have that you can create triggers or run 
> periodic batch processes to index your data in more optimized tables so 
> the reports can be generated on the fly or at least more quickly.
> 
> ~Rolan
> _______________________________________________
> New York PHP Community Talk Mailing List
> http://lists.nyphp.org/mailman/listinfo/talk
> 
> NYPHPCon 2006 Presentations Online
> http://www.nyphpcon.com
> 
> Show Your Participation in New York PHP
> http://www.nyphp.org/show_participation.php
> 
> 

-- 

Thanks!
- Brian Dailey
Software Developer
New York, NY
www.dailytechnology.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: support.vcf
Type: text/x-vcard
Size: 264 bytes
Desc: not available
URL: <http://lists.nyphp.org/pipermail/talk/attachments/20070515/705cb500/attachment.vcf>