[nycphp-talk] XML files
Ian Forsyth
ian at plusfour.org
Thu May 8 14:29:45 EDT 2003
SOAP SERVER.
set up a class to handle the specific db requests. set up the soap
server.. (there is a soap server available via pear). the client can
make request to your server via http, or https using a soap api for
their language.
http://developer.apple.com/internet/webservices/soapphp.html
On Wednesday, May 7, 2003, at 11:59 AM, Larry Chuon wrote:
> Chalu,
> Unfortunately OSSuite is a LAMP based product, not Python. It is
> missing
> CMS/workflow engine. You and I have worked with Zope for a while. We
> understand the importance behind workflow. Hopefully, my team can
> re-architect a framework that will incorporate a workflow engine for
> OSSuite. We are writing a spec for that as we speak. I just want to
> add XML
> into the mix. Anirudh and I chatted about this awhile back. I think
> it is
> time to take a closer look at XML and other best of breed technology.
> Whatever we come up with, resource is still an issue. Hopefully, some
> of
> you can help us out. Anybody?
>
>
> -----Original Message-----
> From: Chalu Kim [mailto:chalu at egenius.com]
> Sent: Wednesday, May 07, 2003 2:46 PM
> To: talk at nyphp.org; Larry Chuon
> Subject: Re: [nycphp-talk] XML files
>
>
> This is the kind of stuff you need the workflow for.
>
> I am speaking in pseudo.
>
> For example, XML files are published. Then, somebody comes along and
> changes
> it. That pushes the state to say "retracted" or "changed". This event
> in a
> workflow system can trigger an action.
>
> Before the state change, you could have an opportunity to execute
> rules.
>
> This is not a simple arena as you need a way to define rules. There
> have
> been
> several attempts such as ZPatterns (available in Zope or Java).
>
> This enables you to define at the interface level as in receiving data
> to
> trigger actions. In ZPatterns, you can use ZPatterns language
> (declarative)
> to define behaviors of data reception. Unfortunately, this approach is
> not
> for average for it is difficult to work with. Just get into this
> rarefied
> Pattern language and all.
>
> None of XML related stuff will address workflow or aspect-oriented
> programming
> because XML files themselves need to remember what state they are in.
>
> If it is a possibility to mix and match, you can publish XML files into
> Zope.
> That gets you workflow. XML files will be available through web or
> FTP. When
> it gets changed, it can do some processing.
>
> Good luck
>
>
> On Wednesday 07 May 2003 02:17 pm, Larry Chuon wrote:
>> Hi Anirudh,
>>
>> Long time no hear. Thank you and everyone at NYPHP
>> <mailto:everyone at NYPHP>
>> for the solution. I learn a great deal in the last day and half. As
> Chalu
>> had noted, I need to rethink through the solution again. Pumping out
>> nearly a million XML files is not the best practice. Last night, I
>> accepted Gary Malcolm's invitation to look at Xindice. Xindice uses
>> Xpath
>> and Xupdate, it addresses some of my concerns (not certain how many
> records
>> it can support though, but I can categorize). Unfortunately, Xindice
>> does
>> not have a PHP client for their API. I need to figure out something
>> for
>> this.
>>
>> Your sample codes resolve my large volume of files should I ever need
>> to
>> spit out more XML files.
>>
>> Now, I have a whole new set of problems. Lets say, I need to share
>> data
>> with some companies. The XML files get changed and must be updated
>> on my
>> system or their system depending on who is/are the recipient(s) of the
>> changes. Since I have nearly a million records, how do I keep track
>> of
>> changes? Importing everything back into Xindice or whatever
>> repository in
>> a multiple of X times will generate a lot unnecessary traffic. On the
>> other hand, updating only the changes will require a lot of tracking
>> (timestamp) as well. Finally, giving appropriate hook to the partners
> will
>> require an extensive predefined business rules (I don't know it all
>> and I
>> don't assume to know my partners/users business workflow either) and
>> some
>> sort of version control for rollback and approval process similarly to
> CMS.
>> Thought anybody?
>>
>> Thanks for all the suggestions guys.
>> Larry
>>
>> -----Original Message-----
>> From: Anirudh Zala [mailto:anirudh at aumcomputers.com]
>> Sent: Wednesday, May 07, 2003 2:17 AM
>> To: talk at nyphp.org
>> Cc: LarryC at indexstock.com
>> Subject: Re: [nycphp-talk] XML files
>>
>> Larry
>>
>> Let's have look about your problems 1 by 1
>>
>> 1: Problem of storing and accessing large volume of files in 1
>> container
>> directory.
>>
>> A: In such cases splitted directory structure solution is best like
>> what
>> "Rick Seeger" has suggested. Hence you can design a special directory
>> structure that can generate new directory as and when total amout of
>> files
>> execeeds the limit say... 100 files per directory. Below sample
>> function
>> does that (it generate new directory as soon as "id" exceeds value
>> 100)
>>
>> // CRATES A DIRECTORY ACCORDING TO GIVEN ID AND PARENT DIR PATH
>> function createDir($id,$parent)
>> {
>> $nd_dirlimit = 100;
>>
>> $mod = ceil($id/$nd_dirlimit) - 1;
>>
>> $dirs = $mod * $nd_dirlimit + 1;
>> $dire = $dirs + $nd_dirlimit - 1;
>>
>> $tempDir = $dirs."_".$dire;
>>
>> if(! file_exists("$parent/$tempDir"))
>> mkdir("$parent/$tempDir",0777);
>>
>> return "$tempDir";
>> }
>> By above method dirctory structure will be like ../1_100/,
>> ../101_200/,
>> ./201_300/ and in each directory file name with id say "1.xml" to
> "100.xml"
>> (or with any name) will reside. Once above directory structure is
>> implemented no script will crawl like now :D.
>>
>> 2: Importing Data into DB by parsing XML files
>>
>> A: Use php's file "XPath.class.php" this is very nice OO file to
>> parse XML
>> documents at greater speed. It has lot of easy to use method and
> properties
>> like $xml->evaluate, $xml->setAttributes, $xml->appendChild,
>> $xml->removeAttribute, $xml->exportToFile etc...by which you can
>> parse lot
>> of XML documents like playing Zigsaw puzzles. It is very handy, class
> based
>> php file. Check example at
>> http://www.soi.city.ac.uk/~sa386/php/XPath_V2/testBench/useCases/
>> <http://www.soi.city.ac.uk/~sa386/php/XPath_V2/testBench/useCases/>
>>
>> Some times it is useful to use Utilities of Perl and executing your
>> perl/cgi scripts from console level. This very fastest and reliable
>> method
>> IF such doucmens are to be generated, read or write in batch mode.
>> Perl's
>> XML::DOM package helps here. Other advantage of such mechanism is
>> that you
>> can run such scripts seperately from other servers that has connection
> with
>> your DB server, hence reduction of overhaed on your main server that
>> has
>> php scripts running.
>>
>> Another method is XSQL that is powerful but still new to us and PHP,
>> where
>> your DB SQL is written directly into XML file that syncronises (update
> your
>> XML document from DB by executing XSQL) your XML and DB from 2 ways.
> (Can't
>> tell much here :( )
>>
>> 3: Is there another way to index and query large volumes of documents?
>>
>> A: I am not getting here exactly your problem, But if you need faster
>> document access from directory or file system (serching 1 file from
> million
>> files) then answer of 1 st question can help here where you just need
>> to
>> search particular document from 100 files from documnet diretory,
>> already
>> given from DB.
>>
>> And if question is related to Data then use of indexes (Indexing
>> commonly
>> used fields in Db like primary keys and foreigh keys) in mysql can
>> make
>> "Access of record" faster.
>>
>> Thanks,
>>
>> Anirudh Zala ( azala at lechuon.com <mailto:azala at lechuon.com> )
>>
>>
> -----------------------------------------------------------------------
> ----
>> - --------------------------
>> Anirudh Zala (Project Manager), Tel: +91 281 2451894
>> AUM Computers, Gsm: +91 98981 37727
>> 317, Star Plaza,
>> <mailto:anirudh at aumcomputers.com> anirudh at aumcomputers.com
>> Rajkot-360001, Gujarat, INDIA,
>> <http://www.aumcomputers.com>
>> http://www.aumcomputers.com
>>
> -----------------------------------------------------------------------
> ----
>> - --------------------------
>>
>> ----- Original Message -----
>> From: "Larry Chuon" < <mailto:LarryC at indexstock.com>
>> LarryC at indexstock.com> To: "NYPHP Talk" < <mailto:talk at nyphp.org>
>> talk at nyphp.org>
>> Sent: Wednesday, 07 May, 2003 4:01 AM
>> Subject: RE: [nycphp-talk] XML files
>>
>>> Thank you all for your quick responses. I'll consider generating
>>> XML on
>>
>> the
>>
>>> fly (still new at this). Here's another question. Besides importing
> the
>>> files to dB, is there another way to index and query large volumes of
>>> documents. I'm going out of tangent a bit here. Let say, I'm
>>> archiving
>>> tons of document for a public library. They want to scan and
>>> digitize
>>> millions of articles. What is the best way to index and search for
>>
>> articles
>>
>>> say by keywords or captions?
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: Malcolm, Gary [mailto:gmalcolm at professionalcredit.com]
>>> Sent: Tuesday, May 06, 2003 5:53 PM
>>> To: NYPHP Talk
>>> Subject: RE: [nycphp-talk] XML files
>>>
>>> <http://www.phpwebhosting.com/> http://www.phpwebhosting.com/
>>> <http://phphosts.codewalkers.com> http://phphosts.codewalkers.com
>>> <http://www.oinko.net/freephp> http://www.oinko.net/freephp
>>> <http://www.free-php-hosting.com> www.free-php-hosting.com
>>>
>>> cheap hosting... cheap db access... i love (hearts in eyes) mysql
>>>
>>>> -----Original Message-----
>>>> From: <mailto:soazine at pop.erols.com> soazine at pop.erols.com
>>
>> [mailto:soazine at pop.mail.rcn.net]
>>
>>>> Sent: Tuesday, 06 May, 2003 2:41 PM
>>>> To: NYPHP Talk
>>>> Subject: Re: [nycphp-talk] XML files
>>>>
>>>>
>>>> Importing the XML files into a database is an ideal solution,
>>>> unfortunately, not always an available one, such as in my
>>>> case. I have
>>>> space on a remote server where database access is very
>>>> expensive (it's my
>>>> own site and out of my price range to afford db access), so I have
>>>> to
>>>> resort to XML as well. PHP parses XML extremely fast and
>>>> efficiently; I
>>>> highly recommend it.
>>>>
>>>> I'd use PHP's available XML parsers along with grouping them into
>>>> directories sorted by a date or some other delimiter to allow
>>>> for smaller
>>>> amount of files per directory.
>>>>
>>>> Phil
>>>>
>>>> Original Message:
>>>> -----------------
>>>> From: Analysis & Solutions
>>>> <mailto:danielc at analysisandsolutions.com>
>>
>> danielc at analysisandsolutions.com
>>
>>>> Date: Tue, 6 May 2003 17:35:16 -0400
>>>> To: <mailto:talk at nyphp.org> talk at nyphp.org
>>>> Subject: Re: [nycphp-talk] XML files
>>>>
>>>> On Tue, May 06, 2003 at 04:49:21PM -0400, Larry Chuon wrote:
>>>>> I'm doing everything that you mention below.
>>>>
>>>> So, import the files into a database and get rid of the XML files.
>>>>
>>>> Here's a quick tutorial on how to parse XML in PHP:
>>>> http:www.analysisandsolutions.com/code/phpxml.htm
>>>>
>>>> Enjoy,
>>>>
>>>> --Dan
>>>>
>>>> --
>>>> FREE scripts that make web and database programming easier
>>>> <http://www.analysisandsolutions.com/software/>
>>
>> http://www.analysisandsolutions.com/software/
>>
>>>> T H E A N A L Y S I S A N D S O L U T I O N S C O M P A N Y
>>>> 4015 7th Ave #4AJ, Brooklyn NY v: 718-854-0335 f: 718-854-0409
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --------------------------------------------------------------------
>>>> mail2web - Check your email from the web at
>>>> <http://mail2web.com/> http://mail2web.com/ .
>>>
>>> --- Unsubscribe at <http://nyphp.org/list/> http://nyphp.org/list/
>>> ---
>>
>>
>
> --
>
> Chalu Kim
> eGenius Inc
> Technology Partner
> chalu at egenius.com, (718) 858-0142
>
>
> --- Unsubscribe at http://nyphp.org/list/ ---
>
>
More information about the talk
mailing list