NYCPHP Meetup

NYPHP.org

[nycphp-talk] Anybody doing anything cool w/ the tokenizer?

Paul A Houle paul at devonianfarm.com
Mon Oct 25 13:14:02 EDT 2010


  Anybody using this?

http://php.net/manual/en/book.tokenizer.php

For a while I was using an autoloader that uses this tokenizer to make a 
list what classes are in what file

http://gen5.info/q/2009/01/09/an-awesome-autoloader-for-php/

I had the idea of trying to "dynamically profile" what the autoloader 
loads after running a few pages,  then gather up those files into one 
big file,  which APC ought to be able to cache.

My first draft of this didn't work because,  sometimes there would be a 
situation where class A would extends on class B,  but class A would 
come earlier in the file.  It seems like you can get away with this some 
of the time,  but not all of the time.

I've rewritten the autoloader so it now keeps track of the "A extends B" 
and "A implements B" relationships,  and I'm planning to topological 
sort the classes so they get put together in the right order.  Hopefully 
this will work,  and let me have both the benefits of autoloading and APC.

Anyway,  in the middle of all this I came to the feeling that people 
aren't really using the power of the tokenizer.  The code would be a 
little more complicated,  but it ought to be possible to build something 
that extracts all the methods and properties of functions or even build 
something that can take functions apart and put them back together 
again,  which would have all sorts of fun applications.





More information about the talk mailing list