[nycphp-talk] XML files
ian at plusfour.org
Thu May 8 14:29:45 EDT 2003
set up a class to handle the specific db requests. set up the soap
server.. (there is a soap server available via pear). the client can
make request to your server via http, or https using a soap api for
On Wednesday, May 7, 2003, at 11:59 AM, Larry Chuon wrote:
> Unfortunately OSSuite is a LAMP based product, not Python. It is
> CMS/workflow engine. You and I have worked with Zope for a while. We
> understand the importance behind workflow. Hopefully, my team can
> re-architect a framework that will incorporate a workflow engine for
> OSSuite. We are writing a spec for that as we speak. I just want to
> add XML
> into the mix. Anirudh and I chatted about this awhile back. I think
> it is
> time to take a closer look at XML and other best of breed technology.
> Whatever we come up with, resource is still an issue. Hopefully, some
> you can help us out. Anybody?
> -----Original Message-----
> From: Chalu Kim [mailto:chalu at egenius.com]
> Sent: Wednesday, May 07, 2003 2:46 PM
> To: talk at nyphp.org; Larry Chuon
> Subject: Re: [nycphp-talk] XML files
> This is the kind of stuff you need the workflow for.
> I am speaking in pseudo.
> For example, XML files are published. Then, somebody comes along and
> it. That pushes the state to say "retracted" or "changed". This event
> in a
> workflow system can trigger an action.
> Before the state change, you could have an opportunity to execute
> This is not a simple arena as you need a way to define rules. There
> several attempts such as ZPatterns (available in Zope or Java).
> This enables you to define at the interface level as in receiving data
> trigger actions. In ZPatterns, you can use ZPatterns language
> to define behaviors of data reception. Unfortunately, this approach is
> for average for it is difficult to work with. Just get into this
> Pattern language and all.
> None of XML related stuff will address workflow or aspect-oriented
> because XML files themselves need to remember what state they are in.
> If it is a possibility to mix and match, you can publish XML files into
> That gets you workflow. XML files will be available through web or
> FTP. When
> it gets changed, it can do some processing.
> Good luck
> On Wednesday 07 May 2003 02:17 pm, Larry Chuon wrote:
>> Hi Anirudh,
>> Long time no hear. Thank you and everyone at NYPHP
>> <mailto:everyone at NYPHP>
>> for the solution. I learn a great deal in the last day and half. As
>> had noted, I need to rethink through the solution again. Pumping out
>> nearly a million XML files is not the best practice. Last night, I
>> accepted Gary Malcolm's invitation to look at Xindice. Xindice uses
>> and Xupdate, it addresses some of my concerns (not certain how many
>> it can support though, but I can categorize). Unfortunately, Xindice
>> not have a PHP client for their API. I need to figure out something
>> Your sample codes resolve my large volume of files should I ever need
>> spit out more XML files.
>> Now, I have a whole new set of problems. Lets say, I need to share
>> with some companies. The XML files get changed and must be updated
>> on my
>> system or their system depending on who is/are the recipient(s) of the
>> changes. Since I have nearly a million records, how do I keep track
>> changes? Importing everything back into Xindice or whatever
>> repository in
>> a multiple of X times will generate a lot unnecessary traffic. On the
>> other hand, updating only the changes will require a lot of tracking
>> (timestamp) as well. Finally, giving appropriate hook to the partners
>> require an extensive predefined business rules (I don't know it all
>> and I
>> don't assume to know my partners/users business workflow either) and
>> sort of version control for rollback and approval process similarly to
>> Thought anybody?
>> Thanks for all the suggestions guys.
>> -----Original Message-----
>> From: Anirudh Zala [mailto:anirudh at aumcomputers.com]
>> Sent: Wednesday, May 07, 2003 2:17 AM
>> To: talk at nyphp.org
>> Cc: LarryC at indexstock.com
>> Subject: Re: [nycphp-talk] XML files
>> Let's have look about your problems 1 by 1
>> 1: Problem of storing and accessing large volume of files in 1
>> A: In such cases splitted directory structure solution is best like
>> "Rick Seeger" has suggested. Hence you can design a special directory
>> structure that can generate new directory as and when total amout of
>> execeeds the limit say... 100 files per directory. Below sample
>> does that (it generate new directory as soon as "id" exceeds value
>> // CRATES A DIRECTORY ACCORDING TO GIVEN ID AND PARENT DIR PATH
>> function createDir($id,$parent)
>> $nd_dirlimit = 100;
>> $mod = ceil($id/$nd_dirlimit) - 1;
>> $dirs = $mod * $nd_dirlimit + 1;
>> $dire = $dirs + $nd_dirlimit - 1;
>> $tempDir = $dirs."_".$dire;
>> if(! file_exists("$parent/$tempDir"))
>> return "$tempDir";
>> By above method dirctory structure will be like ../1_100/,
>> ./201_300/ and in each directory file name with id say "1.xml" to
>> (or with any name) will reside. Once above directory structure is
>> implemented no script will crawl like now :D.
>> 2: Importing Data into DB by parsing XML files
>> A: Use php's file "XPath.class.php" this is very nice OO file to
>> parse XML
>> documents at greater speed. It has lot of easy to use method and
>> like $xml->evaluate, $xml->setAttributes, $xml->appendChild,
>> $xml->removeAttribute, $xml->exportToFile etc...by which you can
>> parse lot
>> of XML documents like playing Zigsaw puzzles. It is very handy, class
>> php file. Check example at
>> Some times it is useful to use Utilities of Perl and executing your
>> perl/cgi scripts from console level. This very fastest and reliable
>> IF such doucmens are to be generated, read or write in batch mode.
>> XML::DOM package helps here. Other advantage of such mechanism is
>> that you
>> can run such scripts seperately from other servers that has connection
>> your DB server, hence reduction of overhaed on your main server that
>> php scripts running.
>> Another method is XSQL that is powerful but still new to us and PHP,
>> your DB SQL is written directly into XML file that syncronises (update
>> XML document from DB by executing XSQL) your XML and DB from 2 ways.
>> tell much here :( )
>> 3: Is there another way to index and query large volumes of documents?
>> A: I am not getting here exactly your problem, But if you need faster
>> document access from directory or file system (serching 1 file from
>> files) then answer of 1 st question can help here where you just need
>> search particular document from 100 files from documnet diretory,
>> given from DB.
>> And if question is related to Data then use of indexes (Indexing
>> used fields in Db like primary keys and foreigh keys) in mysql can
>> "Access of record" faster.
>> Anirudh Zala ( azala at lechuon.com <mailto:azala at lechuon.com> )
>> - --------------------------
>> Anirudh Zala (Project Manager), Tel: +91 281 2451894
>> AUM Computers, Gsm: +91 98981 37727
>> 317, Star Plaza,
>> <mailto:anirudh at aumcomputers.com> anirudh at aumcomputers.com
>> Rajkot-360001, Gujarat, INDIA,
>> - --------------------------
>> ----- Original Message -----
>> From: "Larry Chuon" < <mailto:LarryC at indexstock.com>
>> LarryC at indexstock.com> To: "NYPHP Talk" < <mailto:talk at nyphp.org>
>> talk at nyphp.org>
>> Sent: Wednesday, 07 May, 2003 4:01 AM
>> Subject: RE: [nycphp-talk] XML files
>>> Thank you all for your quick responses. I'll consider generating
>>> XML on
>>> fly (still new at this). Here's another question. Besides importing
>>> files to dB, is there another way to index and query large volumes of
>>> documents. I'm going out of tangent a bit here. Let say, I'm
>>> tons of document for a public library. They want to scan and
>>> millions of articles. What is the best way to index and search for
>>> say by keywords or captions?
>>> -----Original Message-----
>>> From: Malcolm, Gary [mailto:gmalcolm at professionalcredit.com]
>>> Sent: Tuesday, May 06, 2003 5:53 PM
>>> To: NYPHP Talk
>>> Subject: RE: [nycphp-talk] XML files
>>> <http://www.phpwebhosting.com/> http://www.phpwebhosting.com/
>>> <http://phphosts.codewalkers.com> http://phphosts.codewalkers.com
>>> <http://www.oinko.net/freephp> http://www.oinko.net/freephp
>>> <http://www.free-php-hosting.com> www.free-php-hosting.com
>>> cheap hosting... cheap db access... i love (hearts in eyes) mysql
>>>> -----Original Message-----
>>>> From: <mailto:soazine at pop.erols.com> soazine at pop.erols.com
>> [mailto:soazine at pop.mail.rcn.net]
>>>> Sent: Tuesday, 06 May, 2003 2:41 PM
>>>> To: NYPHP Talk
>>>> Subject: Re: [nycphp-talk] XML files
>>>> Importing the XML files into a database is an ideal solution,
>>>> unfortunately, not always an available one, such as in my
>>>> case. I have
>>>> space on a remote server where database access is very
>>>> expensive (it's my
>>>> own site and out of my price range to afford db access), so I have
>>>> resort to XML as well. PHP parses XML extremely fast and
>>>> efficiently; I
>>>> highly recommend it.
>>>> I'd use PHP's available XML parsers along with grouping them into
>>>> directories sorted by a date or some other delimiter to allow
>>>> for smaller
>>>> amount of files per directory.
>>>> Original Message:
>>>> From: Analysis & Solutions
>>>> <mailto:danielc at analysisandsolutions.com>
>> danielc at analysisandsolutions.com
>>>> Date: Tue, 6 May 2003 17:35:16 -0400
>>>> To: <mailto:talk at nyphp.org> talk at nyphp.org
>>>> Subject: Re: [nycphp-talk] XML files
>>>> On Tue, May 06, 2003 at 04:49:21PM -0400, Larry Chuon wrote:
>>>>> I'm doing everything that you mention below.
>>>> So, import the files into a database and get rid of the XML files.
>>>> Here's a quick tutorial on how to parse XML in PHP:
>>>> FREE scripts that make web and database programming easier
>>>> T H E A N A L Y S I S A N D S O L U T I O N S C O M P A N Y
>>>> 4015 7th Ave #4AJ, Brooklyn NY v: 718-854-0335 f: 718-854-0409
>>>> mail2web - Check your email from the web at
>>>> <http://mail2web.com/> http://mail2web.com/ .
>>> --- Unsubscribe at <http://nyphp.org/list/> http://nyphp.org/list/
> Chalu Kim
> eGenius Inc
> Technology Partner
> chalu at egenius.com, (718) 858-0142
> --- Unsubscribe at http://nyphp.org/list/ ---
More information about the talk