Tokenizing and parsing files

Thread view:

Enable EMail Alerts

Start New Thread

Thread rating:

James Reynolds - 06 Mar 2006 22:55 GMT

Over the weekend I tried to find the most elegant solution to
tokenizing and parsing POV-Ray scene files, which uses a c-like
scripting language. I came to the conclusion that there are many
solutions out there, but in trying to understand them it seems
easiest (and take as much time to implement) if I just wrote my own.
(I've never studied lexical analysis, but I understand the basic
concepts).

Does anyone have any opinions on tokenizing and parsing files?

--

Thanks,

James Reynolds
University of Utah
Student Computing Labs
james@scl.utah.edu
801-585-9811

Reply to this Message

David Ledger - 07 Mar 2006 06:27 GMT

>Over the weekend I tried to find the most elegant solution to
>tokenizing and parsing POV-Ray scene files, which uses a c-like
[quoted text clipped - 5 lines]
>
>Does anyone have any opinions on tokenizing and parsing files?

OSX includes the Unix tools 'lex' and 'yacc' for building compilers
for your own language. You define the structure of the language you
want to work with, and 'lex' generates an analyser for it.

David

Signature

David Ledger - Freelance Unix Sysadmin in the UK.
Chair of HPUX SysAdmin SIG of hpUG technical user group (www.hpug.org.uk)
david.ledger@ivdcs.co.uk
www.ivdcs.co.uk

Reply to this Message

Dominic Dunlop - 07 Mar 2006 15:20 GMT

> Over the weekend I tried to find the most elegant solution to
> tokenizing and parsing POV-Ray scene files, which uses a c-like
[quoted text clipped - 5 lines]
>
> Does anyone have any opinions on tokenizing and parsing files?

There's more than one way to do it, of course. But a popular Perl
module is Parse::RecDescent. You'll have to install it yourself, as
it's not part of the Perl core distribution. If you have not done
this before, see the perlmodinstall man page.

There's a tutorial for Parse::RecDescent at <http://www.perl.com/pub/
a/2001/06/13/recdecent.html>.

Signature

Dominic Dunlop

Reply to this Message

Mac Forum / Programming / Perl / March 2006

Tokenizing and parsing files