NYCPHP Meetup

[Paul A Houle]

The fastest way to get data into mysql is like this:

http://dev.mysql.com/doc/refman/5.1/en/load-data.html

I needed to load 2.5M RDF triples into mysql to look up some 
identifiers.  My RDF store balked at the turtle file,  so I hacked away 
the header and treated it like a space-separated file.  LOAD DATA INFILE 
was scary fast...  It loaded the data in 3 seconds!

Once you've loaded raw data into mysql you can do all kinds of tricks like

INSERT INTO ... SELECT

to process the information into the desired form.  If you can do this 
processing w/ clever mysql coding,  you can probably do it in 1/10 the 
LOC and more than 10 times the speed that it would take you in a 
procedural language.  The main thing is that you'll need to be 
comfortable with joins,  subselects and writing extreme queries.