pyperlish

PyPerl User Manual

 

Table of Contents

Overview

Perl is an extremely effective scripting language, with convenient idioms for many practical problems. We want to make those idioms available to python programmers. We do so by providing enough of the perl basic functionality to support the idioms. There is no effort to fully re-implement perl.

test/hello.py gives the general approach. Here is the meat of that script:

    import pyperlish
    p=pyperlish.PyPerl()
    ...
    #---inline expansion---
    a="apple"; b="banana"; c="cantaloupe"
    fruit=[a,b,c]
    line=p("$a is a type of fruit: @fruit")
    print line
    #---regular expressions---
    if p.m(r'type (of)',line):
        print p("perl-style: prematch=$PREMATCH, g1=$1")
        print "python-style: prematch=%s, g1=%s" % (p.PREMATCH,p.S1)
    #---file operators---
    fname="hello.py"
    if p.e(fname) and p.f('_') and p.r('_'):
        file=p.open("< $fname")
        p.input(file)          #note: <file>  autoloads @_
        p.LIST_SEPARATOR='\n' 
        p.prt("@_")

Key points to note:

  • Import pyperlish, and then instantiate it, e.g., p=pyperlish.PyPerl().
  • p(...) does inline expansion.
  • p.m returns T/F, so can be used idiomatically in an if statement.
  • Regular expressions and the resulting global scalars are available.
  • File operators are available -- much less verbose than python.
  • Most functions know about assumed $_ and @_ inputs and outputs, so you can do streams of functions without mentioning the parameters.
  • "S" is used to represent "$". "A" is used to represent "@".
  • PyPerl is a flat namespace. You don't have to remember which module exports which particular piece of functionality. This is dangerous for general purpose programming, but is valuable for a selected set of functions.

Supported Features

The primary documentation is of course Perl v5.00. We just give a few examples and brief description here. Deltas from Perl semantics are noted. See also the test scripts.

Inline Expansion

  p("this is a scalar: $myscalar, and this is an array: @myarray")

  • Lookup in caller's local dict, caller's global dict, PyPerl's dict, and finally in specially named scalars and arrays
  • If a name is not found, just write it as-is
  • Allow "a..zA..Z0..9_" in names
  • Use LIST_SEPARATOR for building strings from arrays

Strings

-
Example Result
chop("abc")"ab"
chomp("abc\n")"abc"
chomp("abc")"abc"
lc("ABc")"abc"
uc("ABc")"ABC"

Regular Expressions

-
Example Result
m(r'([a-f]+)\s+:',line) PREMATCH, MATCH, POSTMATCH, S1,S2...S9 are filled in and return 1 if match, else 0
s("e(ll)",r'---\1---',"Hello, world")"H---ll---o, world"
tr("el","EL","Hello, world")"HELLo, worLd"

  • m is an alias for match; s is an alias for substitute; tr is an alias for translate
  • m and s detect (and use) precompiled patterns if given
  • If the replacement string in tr is shorter than the original, the last char is reused to fill it out.

Arrays

-
Example Result
grep(r'an',['apple','banana','orange'])['banana','orange']
join() join A_ using ':' , load S_ and return S_
join('##') join A_ using '##' , load S_ and return S_
join(arr=['apple','banana','orange']) 'apple:banana:orange' , load S_ and return S_
join('##',['apple','banana','orange']) 'apple##banana##orange' , load S_ and return S_
pop() pop from A_
pop(['banana','orange']) 'orange' (anon arr is modified but lost)
push(['banana','orange'],'apple') ['banana','orange','apple']
push(['banana','orange'],['apple','grape'])['banana','orange','apple','grape']
push(['banana','orange'],3.14159) ['banana','orange','3.14159']
shift() shift A_, return val, A_
shift(myarray) shift myarray, set A_ and return val,A_
split() split S_ using ':', load A_ and return A_
split('##') split S_ using '##', load A_ and return A_
split(':','apple:orange') ['apple','orange']
unshift(['apple','orange'],'banana') ['banana','apple','orange']

The grep pattern can be precompiled.

Execution

-
Example Result
bq('run_this_program') array of lines output by program
NOTE: 'bq' stands for 'backquote', i.e., the perl `...` feature
caller() functionname,filename,first lineno
system('run_this_program') run program silently

File Operations

-
Example Result
b(filename) if blocktype then 1, else 0
c(filename) if chartype then 1, else 0
d(filename) if dirtype then 1, else 0
e(filename) if exists then 1, else 0
f(filename) if regular file then 1, else 0
p(filename) if fifo pipe then 1, else 0
s(filename) if socket then 1, else 0
r(filename) if readable by effective uid/gid then 1, else 0
R(filename) if readable by real uid/gid then 1, else 0
w(filename) if writable by effective uid/gid then 1, else 0
W(filename) if writable by real uid/gid then 1, else 0
x(filename) if executable by effective uid/gid then 1, else 0
X(filename) if executable by real uid/gid then 1, else 0
z(filename) if zero length then 1, else 0

Each does:

  • Use S_ if no path given
  • Inline expansion of pathname before use.
  • Use cached stat if path='_'
  • Return None if file not found

Files and Dirs

IMHO, Python has better idioms for dirs than Perl, but to keep the package complete, we implement the perl approach. First, upon "opendir", we generate a Dir object and get the listdir for it. Then readdir returns those lines, and closedir deletes the object:

  mydir=p.opendir("xyz/abc")
  lines=p.readdir(mydir)
  p.closedir(mydir)

Files are trickier, because of perl's treatment of input/output separators. For input, a File object is created and its buffer is filled upon instantiation. Then read the line or lines from the buffer, modulated by the separator. For output, a File object is created and each writeline or writelines is again modulated by the separator.

The open command is fairly sophisticated, because it must do inline expansion, then parse for the type of file and its direction, and finally generate the File object. Since python does not have output parameters, we return the File object. Thus:

  infile=p.open("< $myfilename")
  p.INPUT_RECORD_SEPARATOR=''  # paragraph mode, equiv of '\n\n'
  lines=infile.readlines()
  myfile.close()
  ...
  while not p.eof(myfile):
    line=myfile.readline()
  ...
  outfile=p.open("> $myfilename")
  p.OUTPUT_RECORD_SEPARATOR='</P>\n<P>'
  p.OUTPUT_AUTOFLUSH=1
  outfile.writelines(lines)
  outfile.close()
  ...

Input and Output

Perl has an angle bracket input operator, which cannot be replicated in python. So we use the function input:

-
Perl Python Comment
<> input() read from stdin, load A_ and return A_
<myfile > input(myfile) read from myfile, load A_ and return A_
@myarr= <>input(arr=myarr)read from stdin, load myarr and return myarr
$myline=<>input1() read a line from stdin, load S_ and return S_
Increment p.INPUT_LINE_NUMBER

The perl print statement respects the select decision, and the OUTPUT_SEPARATOR. We can get those effects but cannot call the function print, because that is a python reserved word. So we use p.prt.

  oldoutput=p.select(myfile)
  for line in lines:
      p.prt(line)
  p.select(oldoutput)

File Functions

The usual suspects:
-
Example Result
chdir('xyz') if ok return 1 else 0
chmod(0755,'file1','file2') if ok return 1 else 0
chown(myuid,mygid,'file1','file2') if ok return 1 else 0
cwd() return os.getcwd()
mkdir('xyz')
p.mkdir('xyz',0755)
default mode=0777
rmdir('xyz') os.rmdir
unlink('xyz') if ok return 1 else 0

  • Paths are inline expanded first
  • chmod and chown support multiple paths, e.g.:
      p.chown(myuid,mygid,'file1','file2','file3')
    
  • On exception, p.ERRORNO is loaded with the error text

Recommendations

When I first started using python, after years of perl, I desperately needed this module. But as I become more python-ized, I use python idioms more and more. These days I use pyperl for:

  1. Regular expression match with boolean result for use in a branch. E.g.:
       if p.m(pattern,data):
         #now use the PREMATCH,MATCH,POSTMATCH, and S1...S9
    

    We need that kind of idiom in python. This is my main use of pyperl these days.

  2. File operators. Python is just too cumbersome. It isn't a matter of readability, because these operators are well known from both shells and perl.

  3. Help in porting perl scripts. Auto-translation isn't realistic, but using pyperl, you can write the python code as fast as you read the perl code. Actually, I'm finding I just rewrite the whole app with python-ized idioms, so not much need for pyperl on this basis.

  4. Inline expansion, e.g. p("$greeting, world"). Technically this was a cool thing to be able to do, but in fact I find I just use python idioms:
      "%s, world" % (greeting)
    or
      "%(greeting)s, world" % some_dict
    

    So expansion mainly shows up in porting perl code.

 
Creator: Harry George
Updated/Created: 2001-12-28