NYCPHP Meetup

NYPHP.org

[nycphp-talk] Question: mysql and large results

Hans Zaunere zaunere at yahoo.com
Mon Jul 15 15:24:57 EDT 2002


--- cesar <can at andrew.cmu.edu> wrote:
> my currently working on a c++ program that takes a large set of
> position
> data (300,000 floats) and calculates using a statistical criteria
> selects
> a certain set.

I've only done MySQL in C, so this might not be much help.

> 
> my program works fine when i filter out some data before my final
> selection (eg: selecting a certain date range in the query), but when
> i
> select the whole table (which i have to do often)
> mysql_store_result(mysql); seems not to able to give me everything i
> want
> since the first entry in the first row is null. i can access some
> data
> after that but its a little strange, possibly its not all there. Can
> anyone suggest a good way to get all the data?

Take a look at http://www.mysql.com/doc/C/_/C_API_problems.html,
specifically the first one.  You could be running out of memory, but
use some of the diag functions (like mysql_field_count(),
mysql_error(), etc) and some system utils. to see if your result set is
what you're expecting and weather you're running out of memory.  I
suppose there could be a bug in the API, but doubtful, especially if
you're using a modern version.  Also use the mysql client and run the
same query - does that work?

> 
> i'm using the LIMIT word in mysql to break it into parts, i need to
> make

For testing purposes, uses a LIMIT, increasing it everytime... where
does it start to fail?

> it fast and to select the whole table right now, takes a few seconds
> (4
> or so, keep in mind its still doesn't give me everything right) so i
> don't
> want to make to many queries, since the result get feed back to a
> webpage
> and user don't want to wait, since they need to run the program
> often.

Webpage?  Shouldn't you be doing this in PHP?  :)  Regardless, if you
have a large number of records, you really should do some kind of
caching.  MySQL's heap tables can be great for this.


HZ


__________________________________________________
Do You Yahoo!?
Yahoo! Autos - Get free new car price quotes
http://autos.yahoo.com



More information about the talk mailing list