[Summary view] [Print] [Text view]

   1  =head1 NAME
   2  
   3  perlpacktut - tutorial on C<pack> and C<unpack>
   4  
   5  =head1 DESCRIPTION
   6  
   7  C<pack> and C<unpack> are two functions for transforming data according
   8  to a user-defined template, between the guarded way Perl stores values
   9  and some well-defined representation as might be required in the 
  10  environment of a Perl program. Unfortunately, they're also two of 
  11  the most misunderstood and most often overlooked functions that Perl
  12  provides. This tutorial will demystify them for you.
  13  
  14  
  15  =head1 The Basic Principle
  16  
  17  Most programming languages don't shelter the memory where variables are
  18  stored. In C, for instance, you can take the address of some variable,
  19  and the C<sizeof> operator tells you how many bytes are allocated to
  20  the variable. Using the address and the size, you may access the storage
  21  to your heart's content.
  22  
  23  In Perl, you just can't access memory at random, but the structural and
  24  representational conversion provided by C<pack> and C<unpack> is an
  25  excellent alternative. The C<pack> function converts values to a byte
  26  sequence containing representations according to a given specification,
  27  the so-called "template" argument. C<unpack> is the reverse process,
  28  deriving some values from the contents of a string of bytes. (Be cautioned,
  29  however, that not all that has been packed together can be neatly unpacked - 
  30  a very common experience as seasoned travellers are likely to confirm.)
  31  
  32  Why, you may ask, would you need a chunk of memory containing some values
  33  in binary representation? One good reason is input and output accessing
  34  some file, a device, or a network connection, whereby this binary
  35  representation is either forced on you or will give you some benefit
  36  in processing. Another cause is passing data to some system call that
  37  is not available as a Perl function: C<syscall> requires you to provide
  38  parameters stored in the way it happens in a C program. Even text processing 
  39  (as shown in the next section) may be simplified with judicious usage 
  40  of these two functions.
  41  
  42  To see how (un)packing works, we'll start with a simple template
  43  code where the conversion is in low gear: between the contents of a byte
  44  sequence and a string of hexadecimal digits. Let's use C<unpack>, since
  45  this is likely to remind you of a dump program, or some desperate last
  46  message unfortunate programs are wont to throw at you before they expire
  47  into the wild blue yonder. Assuming that the variable C<$mem> holds a 
  48  sequence of bytes that we'd like to inspect without assuming anything 
  49  about its meaning, we can write
  50  
  51     my( $hex ) = unpack( 'H*', $mem );
  52     print "$hex\n";
  53  
  54  whereupon we might see something like this, with each pair of hex digits
  55  corresponding to a byte:
  56  
  57     41204d414e204120504c414e20412043414e414c2050414e414d41
  58  
  59  What was in this chunk of memory? Numbers, characters, or a mixture of
  60  both? Assuming that we're on a computer where ASCII (or some similar)
  61  encoding is used: hexadecimal values in the range C<0x40> - C<0x5A>
  62  indicate an uppercase letter, and C<0x20> encodes a space. So we might
  63  assume it is a piece of text, which some are able to read like a tabloid;
  64  but others will have to get hold of an ASCII table and relive that
  65  firstgrader feeling. Not caring too much about which way to read this,
  66  we note that C<unpack> with the template code C<H> converts the contents
  67  of a sequence of bytes into the customary hexadecimal notation. Since
  68  "a sequence of" is a pretty vague indication of quantity, C<H> has been
  69  defined to convert just a single hexadecimal digit unless it is followed
  70  by a repeat count. An asterisk for the repeat count means to use whatever
  71  remains.
  72  
  73  The inverse operation - packing byte contents from a string of hexadecimal
  74  digits - is just as easily written. For instance:
  75  
  76     my $s = pack( 'H2' x 10, map { "3$_" } ( 0..9 ) );
  77     print "$s\n";
  78  
  79  Since we feed a list of ten 2-digit hexadecimal strings to C<pack>, the
  80  pack template should contain ten pack codes. If this is run on a computer
  81  with ASCII character coding, it will print C<0123456789>.
  82  
  83  
  84  =head1 Packing Text
  85  
  86  Let's suppose you've got to read in a data file like this:
  87  
  88      Date      |Description                | Income|Expenditure
  89      01/24/2001 Ahmed's Camel Emporium                  1147.99
  90      01/28/2001 Flea spray                                24.99
  91      01/29/2001 Camel rides to tourists      235.00
  92  
  93  How do we do it? You might think first to use C<split>; however, since
  94  C<split> collapses blank fields, you'll never know whether a record was
  95  income or expenditure. Oops. Well, you could always use C<substr>:
  96  
  97      while (<>) { 
  98          my $date   = substr($_,  0, 11);
  99          my $desc   = substr($_, 12, 27);
 100          my $income = substr($_, 40,  7);
 101          my $expend = substr($_, 52,  7);
 102          ...
 103      }
 104  
 105  It's not really a barrel of laughs, is it? In fact, it's worse than it
 106  may seem; the eagle-eyed may notice that the first field should only be
 107  10 characters wide, and the error has propagated right through the other
 108  numbers - which we've had to count by hand. So it's error-prone as well
 109  as horribly unfriendly.
 110  
 111  Or maybe we could use regular expressions:
 112  
 113      while (<>) { 
 114          my($date, $desc, $income, $expend) = 
 115              m|(\d\d/\d\d/\d{4}) (.{27}) (.{7})(.*)|;
 116          ...
 117      }
 118  
 119  Urgh. Well, it's a bit better, but - well, would you want to maintain
 120  that?
 121  
 122  Hey, isn't Perl supposed to make this sort of thing easy? Well, it does,
 123  if you use the right tools. C<pack> and C<unpack> are designed to help
 124  you out when dealing with fixed-width data like the above. Let's have a
 125  look at a solution with C<unpack>:
 126  
 127      while (<>) { 
 128          my($date, $desc, $income, $expend) = unpack("A10xA27xA7A*", $_);
 129          ...
 130      }
 131  
 132  That looks a bit nicer; but we've got to take apart that weird template.
 133  Where did I pull that out of? 
 134  
 135  OK, let's have a look at some of our data again; in fact, we'll include
 136  the headers, and a handy ruler so we can keep track of where we are.
 137  
 138               1         2         3         4         5        
 139      1234567890123456789012345678901234567890123456789012345678
 140      Date      |Description                | Income|Expenditure
 141      01/28/2001 Flea spray                                24.99
 142      01/29/2001 Camel rides to tourists      235.00
 143  
 144  From this, we can see that the date column stretches from column 1 to
 145  column 10 - ten characters wide. The C<pack>-ese for "character" is
 146  C<A>, and ten of them are C<A10>. So if we just wanted to extract the
 147  dates, we could say this:
 148  
 149      my($date) = unpack("A10", $_);
 150  
 151  OK, what's next? Between the date and the description is a blank column;
 152  we want to skip over that. The C<x> template means "skip forward", so we
 153  want one of those. Next, we have another batch of characters, from 12 to
 154  38. That's 27 more characters, hence C<A27>. (Don't make the fencepost
 155  error - there are 27 characters between 12 and 38, not 26. Count 'em!)
 156  
 157  Now we skip another character and pick up the next 7 characters:
 158  
 159      my($date,$description,$income) = unpack("A10xA27xA7", $_);
 160  
 161  Now comes the clever bit. Lines in our ledger which are just income and
 162  not expenditure might end at column 46. Hence, we don't want to tell our
 163  C<unpack> pattern that we B<need> to find another 12 characters; we'll
 164  just say "if there's anything left, take it". As you might guess from
 165  regular expressions, that's what the C<*> means: "use everything
 166  remaining".
 167  
 168  =over 3
 169  
 170  =item *
 171  
 172  Be warned, though, that unlike regular expressions, if the C<unpack>
 173  template doesn't match the incoming data, Perl will scream and die.
 174  
 175  =back
 176  
 177  
 178  Hence, putting it all together:
 179  
 180      my($date,$description,$income,$expend) = unpack("A10xA27xA7xA*", $_);
 181  
 182  Now, that's our data parsed. I suppose what we might want to do now is
 183  total up our income and expenditure, and add another line to the end of
 184  our ledger - in the same format - saying how much we've brought in and
 185  how much we've spent:
 186  
 187      while (<>) {
 188          my($date, $desc, $income, $expend) = unpack("A10xA27xA7xA*", $_);
 189          $tot_income += $income;
 190          $tot_expend += $expend;
 191      }
 192  
 193      $tot_income = sprintf("%.2f", $tot_income); # Get them into 
 194      $tot_expend = sprintf("%.2f", $tot_expend); # "financial" format
 195  
 196      $date = POSIX::strftime("%m/%d/%Y", localtime); 
 197  
 198      # OK, let's go:
 199  
 200      print pack("A10xA27xA7xA*", $date, "Totals", $tot_income, $tot_expend);
 201  
 202  Oh, hmm. That didn't quite work. Let's see what happened:
 203  
 204      01/24/2001 Ahmed's Camel Emporium                   1147.99
 205      01/28/2001 Flea spray                                 24.99
 206      01/29/2001 Camel rides to tourists     1235.00
 207      03/23/2001Totals                     1235.001172.98
 208  
 209  OK, it's a start, but what happened to the spaces? We put C<x>, didn't
 210  we? Shouldn't it skip forward? Let's look at what L<perlfunc/pack> says:
 211  
 212      x   A null byte.
 213  
 214  Urgh. No wonder. There's a big difference between "a null byte",
 215  character zero, and "a space", character 32. Perl's put something
 216  between the date and the description - but unfortunately, we can't see
 217  it! 
 218  
 219  What we actually need to do is expand the width of the fields. The C<A>
 220  format pads any non-existent characters with spaces, so we can use the
 221  additional spaces to line up our fields, like this:
 222  
 223      print pack("A11 A28 A8 A*", $date, "Totals", $tot_income, $tot_expend);
 224  
 225  (Note that you can put spaces in the template to make it more readable,
 226  but they don't translate to spaces in the output.) Here's what we got
 227  this time:
 228  
 229      01/24/2001 Ahmed's Camel Emporium                   1147.99
 230      01/28/2001 Flea spray                                 24.99
 231      01/29/2001 Camel rides to tourists     1235.00
 232      03/23/2001 Totals                      1235.00 1172.98
 233  
 234  That's a bit better, but we still have that last column which needs to
 235  be moved further over. There's an easy way to fix this up:
 236  unfortunately, we can't get C<pack> to right-justify our fields, but we
 237  can get C<sprintf> to do it:
 238  
 239      $tot_income = sprintf("%.2f", $tot_income); 
 240      $tot_expend = sprintf("%12.2f", $tot_expend);
 241      $date = POSIX::strftime("%m/%d/%Y", localtime); 
 242      print pack("A11 A28 A8 A*", $date, "Totals", $tot_income, $tot_expend);
 243  
 244  This time we get the right answer:
 245  
 246      01/28/2001 Flea spray                                 24.99
 247      01/29/2001 Camel rides to tourists     1235.00
 248      03/23/2001 Totals                      1235.00      1172.98
 249  
 250  So that's how we consume and produce fixed-width data. Let's recap what
 251  we've seen of C<pack> and C<unpack> so far:
 252  
 253  =over 3
 254  
 255  =item *
 256  
 257  Use C<pack> to go from several pieces of data to one fixed-width
 258  version; use C<unpack> to turn a fixed-width-format string into several
 259  pieces of data. 
 260  
 261  =item *
 262  
 263  The pack format C<A> means "any character"; if you're C<pack>ing and
 264  you've run out of things to pack, C<pack> will fill the rest up with
 265  spaces.
 266  
 267  =item *
 268  
 269  C<x> means "skip a byte" when C<unpack>ing; when C<pack>ing, it means
 270  "introduce a null byte" - that's probably not what you mean if you're
 271  dealing with plain text.
 272  
 273  =item *
 274  
 275  You can follow the formats with numbers to say how many characters
 276  should be affected by that format: C<A12> means "take 12 characters";
 277  C<x6> means "skip 6 bytes" or "character 0, 6 times".
 278  
 279  =item *
 280  
 281  Instead of a number, you can use C<*> to mean "consume everything else
 282  left". 
 283  
 284  B<Warning>: when packing multiple pieces of data, C<*> only means
 285  "consume all of the current piece of data". That's to say
 286  
 287      pack("A*A*", $one, $two)
 288  
 289  packs all of C<$one> into the first C<A*> and then all of C<$two> into
 290  the second. This is a general principle: each format character
 291  corresponds to one piece of data to be C<pack>ed.
 292  
 293  =back
 294  
 295  
 296  
 297  =head1 Packing Numbers
 298  
 299  So much for textual data. Let's get onto the meaty stuff that C<pack>
 300  and C<unpack> are best at: handling binary formats for numbers. There is,
 301  of course, not just one binary format  - life would be too simple - but
 302  Perl will do all the finicky labor for you.
 303  
 304  
 305  =head2 Integers
 306  
 307  Packing and unpacking numbers implies conversion to and from some
 308  I<specific> binary representation. Leaving floating point numbers
 309  aside for the moment, the salient properties of any such representation
 310  are:
 311  
 312  =over 4
 313  
 314  =item *
 315  
 316  the number of bytes used for storing the integer,
 317  
 318  =item *
 319  
 320  whether the contents are interpreted as a signed or unsigned number,
 321  
 322  =item *
 323  
 324  the byte ordering: whether the first byte is the least or most
 325  significant byte (or: little-endian or big-endian, respectively).
 326  
 327  =back
 328  
 329  So, for instance, to pack 20302 to a signed 16 bit integer in your
 330  computer's representation you write
 331  
 332     my $ps = pack( 's', 20302 );
 333  
 334  Again, the result is a string, now containing 2 bytes. If you print 
 335  this string (which is, generally, not recommended) you might see
 336  C<ON> or C<NO> (depending on your system's byte ordering) - or something
 337  entirely different if your computer doesn't use ASCII character encoding.
 338  Unpacking C<$ps> with the same template returns the original integer value:
 339  
 340     my( $s ) = unpack( 's', $ps );
 341  
 342  This is true for all numeric template codes. But don't expect miracles:
 343  if the packed value exceeds the allotted byte capacity, high order bits
 344  are silently discarded, and unpack certainly won't be able to pull them
 345  back out of some magic hat. And, when you pack using a signed template
 346  code such as C<s>, an excess value may result in the sign bit
 347  getting set, and unpacking this will smartly return a negative value.
 348  
 349  16 bits won't get you too far with integers, but there is C<l> and C<L>
 350  for signed and unsigned 32-bit integers. And if this is not enough and
 351  your system supports 64 bit integers you can push the limits much closer
 352  to infinity with pack codes C<q> and C<Q>. A notable exception is provided
 353  by pack codes C<i> and C<I> for signed and unsigned integers of the 
 354  "local custom" variety: Such an integer will take up as many bytes as
 355  a local C compiler returns for C<sizeof(int)>, but it'll use I<at least>
 356  32 bits.
 357  
 358  Each of the integer pack codes C<sSlLqQ> results in a fixed number of bytes,
 359  no matter where you execute your program. This may be useful for some 
 360  applications, but it does not provide for a portable way to pass data 
 361  structures between Perl and C programs (bound to happen when you call 
 362  XS extensions or the Perl function C<syscall>), or when you read or
 363  write binary files. What you'll need in this case are template codes that
 364  depend on what your local C compiler compiles when you code C<short> or
 365  C<unsigned long>, for instance. These codes and their corresponding
 366  byte lengths are shown in the table below.  Since the C standard leaves
 367  much leeway with respect to the relative sizes of these data types, actual
 368  values may vary, and that's why the values are given as expressions in
 369  C and Perl. (If you'd like to use values from C<%Config> in your program
 370  you have to import it with C<use Config>.)
 371  
 372     signed unsigned  byte length in C   byte length in Perl       
 373       s!     S!      sizeof(short)      $Config{shortsize}
 374       i!     I!      sizeof(int)        $Config{intsize}
 375       l!     L!      sizeof(long)       $Config{longsize}
 376       q!     Q!      sizeof(long long)  $Config{longlongsize}
 377  
 378  The C<i!> and C<I!> codes aren't different from C<i> and C<I>; they are
 379  tolerated for completeness' sake.
 380  
 381  
 382  =head2 Unpacking a Stack Frame
 383  
 384  Requesting a particular byte ordering may be necessary when you work with
 385  binary data coming from some specific architecture whereas your program could
 386  run on a totally different system. As an example, assume you have 24 bytes
 387  containing a stack frame as it happens on an Intel 8086:
 388  
 389        +---------+        +----+----+               +---------+
 390   TOS: |   IP    |  TOS+4:| FL | FH | FLAGS  TOS+14:|   SI    |
 391        +---------+        +----+----+               +---------+
 392        |   CS    |        | AL | AH | AX            |   DI    |
 393        +---------+        +----+----+               +---------+
 394                           | BL | BH | BX            |   BP    |
 395                           +----+----+               +---------+
 396                           | CL | CH | CX            |   DS    |
 397                           +----+----+               +---------+
 398                           | DL | DH | DX            |   ES    |
 399                           +----+----+               +---------+
 400  
 401  First, we note that this time-honored 16-bit CPU uses little-endian order,
 402  and that's why the low order byte is stored at the lower address. To
 403  unpack such a (unsigned) short we'll have to use code C<v>. A repeat
 404  count unpacks all 12 shorts:
 405  
 406     my( $ip, $cs, $flags, $ax, $bx, $cd, $dx, $si, $di, $bp, $ds, $es ) =
 407       unpack( 'v12', $frame );
 408  
 409  Alternatively, we could have used C<C> to unpack the individually
 410  accessible byte registers FL, FH, AL, AH, etc.:
 411  
 412     my( $fl, $fh, $al, $ah, $bl, $bh, $cl, $ch, $dl, $dh ) =
 413       unpack( 'C10', substr( $frame, 4, 10 ) );
 414  
 415  It would be nice if we could do this in one fell swoop: unpack a short,
 416  back up a little, and then unpack 2 bytes. Since Perl I<is> nice, it
 417  proffers the template code C<X> to back up one byte. Putting this all
 418  together, we may now write:
 419  
 420     my( $ip, $cs,
 421         $flags,$fl,$fh,
 422         $ax,$al,$ah, $bx,$bl,$bh, $cx,$cl,$ch, $dx,$dl,$dh, 
 423         $si, $di, $bp, $ds, $es ) =
 424     unpack( 'v2' . ('vXXCC' x 5) . 'v5', $frame );
 425  
 426  (The clumsy construction of the template can be avoided - just read on!)  
 427  
 428  We've taken some pains to construct the template so that it matches
 429  the contents of our frame buffer. Otherwise we'd either get undefined values,
 430  or C<unpack> could not unpack all. If C<pack> runs out of items, it will
 431  supply null strings (which are coerced into zeroes whenever the pack code
 432  says so).
 433  
 434  
 435  =head2 How to Eat an Egg on a Net
 436  
 437  The pack code for big-endian (high order byte at the lowest address) is
 438  C<n> for 16 bit and C<N> for 32 bit integers. You use these codes
 439  if you know that your data comes from a compliant architecture, but,
 440  surprisingly enough, you should also use these pack codes if you
 441  exchange binary data, across the network, with some system that you
 442  know next to nothing about. The simple reason is that this
 443  order has been chosen as the I<network order>, and all standard-fearing
 444  programs ought to follow this convention. (This is, of course, a stern
 445  backing for one of the Lilliputian parties and may well influence the
 446  political development there.) So, if the protocol expects you to send
 447  a message by sending the length first, followed by just so many bytes,
 448  you could write:
 449  
 450     my $buf = pack( 'N', length( $msg ) ) . $msg;
 451  
 452  or even:
 453  
 454     my $buf = pack( 'NA*', length( $msg ), $msg );
 455  
 456  and pass C<$buf> to your send routine. Some protocols demand that the
 457  count should include the length of the count itself: then just add 4
 458  to the data length. (But make sure to read L<"Lengths and Widths"> before
 459  you really code this!)
 460  
 461  
 462  =head2 Byte-order modifiers
 463  
 464  In the previous sections we've learned how to use C<n>, C<N>, C<v> and
 465  C<V> to pack and unpack integers with big- or little-endian byte-order.
 466  While this is nice, it's still rather limited because it leaves out all
 467  kinds of signed integers as well as 64-bit integers. For example, if you
 468  wanted to unpack a sequence of signed big-endian 16-bit integers in a
 469  platform-independent way, you would have to write:
 470  
 471     my @data = unpack 's*', pack 'S*', unpack 'n*', $buf;
 472  
 473  This is ugly. As of Perl 5.9.2, there's a much nicer way to express your
 474  desire for a certain byte-order: the C<E<gt>> and C<E<lt>> modifiers.
 475  C<E<gt>> is the big-endian modifier, while C<E<lt>> is the little-endian
 476  modifier. Using them, we could rewrite the above code as:
 477  
 478     my @data = unpack 's>*', $buf;
 479  
 480  As you can see, the "big end" of the arrow touches the C<s>, which is a
 481  nice way to remember that C<E<gt>> is the big-endian modifier. The same
 482  obviously works for C<E<lt>>, where the "little end" touches the code.
 483  
 484  You will probably find these modifiers even more useful if you have
 485  to deal with big- or little-endian C structures. Be sure to read
 486  L<"Packing and Unpacking C Structures"> for more on that.
 487  
 488  
 489  =head2 Floating point Numbers
 490  
 491  For packing floating point numbers you have the choice between the
 492  pack codes C<f>, C<d>, C<F> and C<D>. C<f> and C<d> pack into (or unpack
 493  from) single-precision or double-precision representation as it is provided
 494  by your system. If your systems supports it, C<D> can be used to pack and
 495  unpack extended-precision floating point values (C<long double>), which
 496  can offer even more resolution than C<f> or C<d>. C<F> packs an C<NV>,
 497  which is the floating point type used by Perl internally. (There
 498  is no such thing as a network representation for reals, so if you want
 499  to send your real numbers across computer boundaries, you'd better stick
 500  to ASCII representation, unless you're absolutely sure what's on the other
 501  end of the line. For the even more adventuresome, you can use the byte-order
 502  modifiers from the previous section also on floating point codes.)
 503  
 504  
 505  
 506  =head1 Exotic Templates
 507  
 508  
 509  =head2 Bit Strings
 510  
 511  Bits are the atoms in the memory world. Access to individual bits may
 512  have to be used either as a last resort or because it is the most
 513  convenient way to handle your data. Bit string (un)packing converts
 514  between strings containing a series of C<0> and C<1> characters and
 515  a sequence of bytes each containing a group of 8 bits. This is almost
 516  as simple as it sounds, except that there are two ways the contents of
 517  a byte may be written as a bit string. Let's have a look at an annotated
 518  byte:
 519  
 520       7 6 5 4 3 2 1 0
 521     +-----------------+
 522     | 1 0 0 0 1 1 0 0 |
 523     +-----------------+
 524      MSB           LSB
 525  
 526  It's egg-eating all over again: Some think that as a bit string this should
 527  be written "10001100" i.e. beginning with the most significant bit, others
 528  insist on "00110001". Well, Perl isn't biased, so that's why we have two bit
 529  string codes:
 530  
 531     $byte = pack( 'B8', '10001100' ); # start with MSB
 532     $byte = pack( 'b8', '00110001' ); # start with LSB
 533  
 534  It is not possible to pack or unpack bit fields - just integral bytes.
 535  C<pack> always starts at the next byte boundary and "rounds up" to the
 536  next multiple of 8 by adding zero bits as required. (If you do want bit
 537  fields, there is L<perlfunc/vec>. Or you could implement bit field 
 538  handling at the character string level, using split, substr, and
 539  concatenation on unpacked bit strings.)
 540  
 541  To illustrate unpacking for bit strings, we'll decompose a simple
 542  status register (a "-" stands for a "reserved" bit):
 543  
 544     +-----------------+-----------------+
 545     | S Z - A - P - C | - - - - O D I T |
 546     +-----------------+-----------------+
 547      MSB           LSB MSB           LSB
 548  
 549  Converting these two bytes to a string can be done with the unpack 
 550  template C<'b16'>. To obtain the individual bit values from the bit
 551  string we use C<split> with the "empty" separator pattern which dissects
 552  into individual characters. Bit values from the "reserved" positions are
 553  simply assigned to C<undef>, a convenient notation for "I don't care where
 554  this goes".
 555  
 556     ($carry, undef, $parity, undef, $auxcarry, undef, $zero, $sign,
 557      $trace, $interrupt, $direction, $overflow) =
 558        split( //, unpack( 'b16', $status ) );
 559  
 560  We could have used an unpack template C<'b12'> just as well, since the
 561  last 4 bits can be ignored anyway. 
 562  
 563  
 564  =head2 Uuencoding
 565  
 566  Another odd-man-out in the template alphabet is C<u>, which packs an
 567  "uuencoded string". ("uu" is short for Unix-to-Unix.) Chances are that
 568  you won't ever need this encoding technique which was invented to overcome
 569  the shortcomings of old-fashioned transmission mediums that do not support
 570  other than simple ASCII data. The essential recipe is simple: Take three 
 571  bytes, or 24 bits. Split them into 4 six-packs, adding a space (0x20) to 
 572  each. Repeat until all of the data is blended. Fold groups of 4 bytes into 
 573  lines no longer than 60 and garnish them in front with the original byte count 
 574  (incremented by 0x20) and a C<"\n"> at the end. - The C<pack> chef will
 575  prepare this for you, a la minute, when you select pack code C<u> on the menu:
 576  
 577     my $uubuf = pack( 'u', $bindat );
 578  
 579  A repeat count after C<u> sets the number of bytes to put into an
 580  uuencoded line, which is the maximum of 45 by default, but could be
 581  set to some (smaller) integer multiple of three. C<unpack> simply ignores
 582  the repeat count.
 583  
 584  
 585  =head2 Doing Sums
 586  
 587  An even stranger template code is C<%>E<lt>I<number>E<gt>. First, because 
 588  it's used as a prefix to some other template code. Second, because it
 589  cannot be used in C<pack> at all, and third, in C<unpack>, doesn't return the
 590  data as defined by the template code it precedes. Instead it'll give you an
 591  integer of I<number> bits that is computed from the data value by 
 592  doing sums. For numeric unpack codes, no big feat is achieved:
 593  
 594      my $buf = pack( 'iii', 100, 20, 3 );
 595      print unpack( '%32i3', $buf ), "\n";  # prints 123
 596  
 597  For string values, C<%> returns the sum of the byte values saving
 598  you the trouble of a sum loop with C<substr> and C<ord>:
 599  
 600      print unpack( '%32A*', "\x01\x10" ), "\n";  # prints 17
 601  
 602  Although the C<%> code is documented as returning a "checksum":
 603  don't put your trust in such values! Even when applied to a small number
 604  of bytes, they won't guarantee a noticeable Hamming distance.
 605  
 606  In connection with C<b> or C<B>, C<%> simply adds bits, and this can be put
 607  to good use to count set bits efficiently:
 608  
 609      my $bitcount = unpack( '%32b*', $mask );
 610  
 611  And an even parity bit can be determined like this:
 612  
 613      my $evenparity = unpack( '%1b*', $mask );
 614  
 615  
 616  =head2  Unicode
 617  
 618  Unicode is a character set that can represent most characters in most of
 619  the world's languages, providing room for over one million different
 620  characters. Unicode 3.1 specifies 94,140 characters: The Basic Latin
 621  characters are assigned to the numbers 0 - 127. The Latin-1 Supplement with
 622  characters that are used in several European languages is in the next
 623  range, up to 255. After some more Latin extensions we find the character
 624  sets from languages using non-Roman alphabets, interspersed with a
 625  variety of symbol sets such as currency symbols, Zapf Dingbats or Braille.
 626  (You might want to visit L<http://www.unicode.org/> for a look at some of
 627  them - my personal favourites are Telugu and Kannada.)
 628  
 629  The Unicode character sets associates characters with integers. Encoding
 630  these numbers in an equal number of bytes would more than double the
 631  requirements for storing texts written in Latin alphabets.
 632  The UTF-8 encoding avoids this by storing the most common (from a western
 633  point of view) characters in a single byte while encoding the rarer
 634  ones in three or more bytes.
 635  
 636  Perl uses UTF-8, internally, for most Unicode strings.
 637  
 638  So what has this got to do with C<pack>? Well, if you want to compose a
 639  Unicode string (that is internally encoded as UTF-8), you can do so by
 640  using template code C<U>. As an example, let's produce the Euro currency
 641  symbol (code number 0x20AC):
 642  
 643     $UTF8{Euro} = pack( 'U', 0x20AC );
 644     # Equivalent to: $UTF8{Euro} = "\x{20ac}";
 645  
 646  Inspecting C<$UTF8{Euro}> shows that it contains 3 bytes:
 647  "\xe2\x82\xac". However, it contains only 1 character, number 0x20AC.
 648  The round trip can be completed with C<unpack>:
 649  
 650     $Unicode{Euro} = unpack( 'U', $UTF8{Euro} );
 651  
 652  Unpacking using the C<U> template code also works on UTF-8 encoded byte
 653  strings.
 654  
 655  Usually you'll want to pack or unpack UTF-8 strings:
 656  
 657     # pack and unpack the Hebrew alphabet
 658     my $alefbet = pack( 'U*', 0x05d0..0x05ea );
 659     my @hebrew = unpack( 'U*', $utf );
 660  
 661  Please note: in the general case, you're better off using
 662  Encode::decode_utf8 to decode a UTF-8 encoded byte string to a Perl
 663  Unicode string, and Encode::encode_utf8 to encode a Perl Unicode string
 664  to UTF-8 bytes. These functions provide means of handling invalid byte
 665  sequences and generally have a friendlier interface.
 666  
 667  =head2 Another Portable Binary Encoding
 668  
 669  The pack code C<w> has been added to support a portable binary data
 670  encoding scheme that goes way beyond simple integers. (Details can
 671  be found at L<http://Casbah.org/>, the Scarab project.)  A BER (Binary Encoded
 672  Representation) compressed unsigned integer stores base 128
 673  digits, most significant digit first, with as few digits as possible.
 674  Bit eight (the high bit) is set on each byte except the last. There
 675  is no size limit to BER encoding, but Perl won't go to extremes.
 676  
 677     my $berbuf = pack( 'w*', 1, 128, 128+1, 128*128+127 );
 678  
 679  A hex dump of C<$berbuf>, with spaces inserted at the right places,
 680  shows 01 8100 8101 81807F. Since the last byte is always less than
 681  128, C<unpack> knows where to stop.
 682  
 683  
 684  =head1 Template Grouping
 685  
 686  Prior to Perl 5.8, repetitions of templates had to be made by
 687  C<x>-multiplication of template strings. Now there is a better way as
 688  we may use the pack codes C<(> and C<)> combined with a repeat count.
 689  The C<unpack> template from the Stack Frame example can simply
 690  be written like this:
 691  
 692     unpack( 'v2 (vXXCC)5 v5', $frame )
 693  
 694  Let's explore this feature a little more. We'll begin with the equivalent of
 695  
 696     join( '', map( substr( $_, 0, 1 ), @str ) )
 697  
 698  which returns a string consisting of the first character from each string.
 699  Using pack, we can write
 700  
 701     pack( '(A)'.@str, @str )
 702  
 703  or, because a repeat count C<*> means "repeat as often as required",
 704  simply
 705  
 706     pack( '(A)*', @str )
 707  
 708  (Note that the template C<A*> would only have packed C<$str[0]> in full
 709  length.)
 710  
 711  To pack dates stored as triplets ( day, month, year ) in an array C<@dates>
 712  into a sequence of byte, byte, short integer we can write
 713  
 714     $pd = pack( '(CCS)*', map( @$_, @dates ) );
 715  
 716  To swap pairs of characters in a string (with even length) one could use
 717  several techniques. First, let's use C<x> and C<X> to skip forward and back:
 718  
 719     $s = pack( '(A)*', unpack( '(xAXXAx)*', $s ) );
 720  
 721  We can also use C<@> to jump to an offset, with 0 being the position where
 722  we were when the last C<(> was encountered:
 723  
 724     $s = pack( '(A)*', unpack( '(@1A @0A @2)*', $s ) );
 725  
 726  Finally, there is also an entirely different approach by unpacking big
 727  endian shorts and packing them in the reverse byte order:
 728  
 729     $s = pack( '(v)*', unpack( '(n)*', $s );
 730  
 731  
 732  =head1 Lengths and Widths
 733  
 734  =head2 String Lengths
 735  
 736  In the previous section we've seen a network message that was constructed
 737  by prefixing the binary message length to the actual message. You'll find
 738  that packing a length followed by so many bytes of data is a 
 739  frequently used recipe since appending a null byte won't work
 740  if a null byte may be part of the data. Here is an example where both
 741  techniques are used: after two null terminated strings with source and
 742  destination address, a Short Message (to a mobile phone) is sent after
 743  a length byte:
 744  
 745     my $msg = pack( 'Z*Z*CA*', $src, $dst, length( $sm ), $sm );
 746  
 747  Unpacking this message can be done with the same template:
 748  
 749     ( $src, $dst, $len, $sm ) = unpack( 'Z*Z*CA*', $msg );
 750  
 751  There's a subtle trap lurking in the offing: Adding another field after
 752  the Short Message (in variable C<$sm>) is all right when packing, but this
 753  cannot be unpacked naively:
 754  
 755     # pack a message
 756     my $msg = pack( 'Z*Z*CA*C', $src, $dst, length( $sm ), $sm, $prio );
 757  
 758     # unpack fails - $prio remains undefined!
 759     ( $src, $dst, $len, $sm, $prio ) = unpack( 'Z*Z*CA*C', $msg );
 760  
 761  The pack code C<A*> gobbles up all remaining bytes, and C<$prio> remains
 762  undefined! Before we let disappointment dampen the morale: Perl's got
 763  the trump card to make this trick too, just a little further up the sleeve.
 764  Watch this:
 765  
 766     # pack a message: ASCIIZ, ASCIIZ, length/string, byte
 767     my $msg = pack( 'Z* Z* C/A* C', $src, $dst, $sm, $prio );
 768  
 769     # unpack
 770     ( $src, $dst, $sm, $prio ) = unpack( 'Z* Z* C/A* C', $msg );
 771  
 772  Combining two pack codes with a slash (C</>) associates them with a single
 773  value from the argument list. In C<pack>, the length of the argument is
 774  taken and packed according to the first code while the argument itself
 775  is added after being converted with the template code after the slash.
 776  This saves us the trouble of inserting the C<length> call, but it is 
 777  in C<unpack> where we really score: The value of the length byte marks the
 778  end of the string to be taken from the buffer. Since this combination
 779  doesn't make sense except when the second pack code isn't C<a*>, C<A*>
 780  or C<Z*>, Perl won't let you.
 781  
 782  The pack code preceding C</> may be anything that's fit to represent a
 783  number: All the numeric binary pack codes, and even text codes such as
 784  C<A4> or C<Z*>:
 785  
 786     # pack/unpack a string preceded by its length in ASCII
 787     my $buf = pack( 'A4/A*', "Humpty-Dumpty" );
 788     # unpack $buf: '13  Humpty-Dumpty'
 789     my $txt = unpack( 'A4/A*', $buf );
 790  
 791  C</> is not implemented in Perls before 5.6, so if your code is required to
 792  work on older Perls you'll need to C<unpack( 'Z* Z* C')> to get the length,
 793  then use it to make a new unpack string. For example
 794  
 795     # pack a message: ASCIIZ, ASCIIZ, length, string, byte (5.005 compatible)
 796     my $msg = pack( 'Z* Z* C A* C', $src, $dst, length $sm, $sm, $prio );
 797  
 798     # unpack
 799     ( undef, undef, $len) = unpack( 'Z* Z* C', $msg );
 800     ($src, $dst, $sm, $prio) = unpack ( "Z* Z* x A$len C", $msg );
 801  
 802  But that second C<unpack> is rushing ahead. It isn't using a simple literal
 803  string for the template. So maybe we should introduce...
 804  
 805  =head2 Dynamic Templates
 806  
 807  So far, we've seen literals used as templates. If the list of pack
 808  items doesn't have fixed length, an expression constructing the
 809  template is required (whenever, for some reason, C<()*> cannot be used).
 810  Here's an example: To store named string values in a way that can be
 811  conveniently parsed by a C program, we create a sequence of names and
 812  null terminated ASCII strings, with C<=> between the name and the value,
 813  followed by an additional delimiting null byte. Here's how:
 814  
 815     my $env = pack( '(A*A*Z*)' . keys( %Env ) . 'C',
 816                     map( { ( $_, '=', $Env{$_} ) } keys( %Env ) ), 0 );
 817  
 818  Let's examine the cogs of this byte mill, one by one. There's the C<map>
 819  call, creating the items we intend to stuff into the C<$env> buffer:
 820  to each key (in C<$_>) it adds the C<=> separator and the hash entry value.
 821  Each triplet is packed with the template code sequence C<A*A*Z*> that
 822  is repeated according to the number of keys. (Yes, that's what the C<keys>
 823  function returns in scalar context.) To get the very last null byte,
 824  we add a C<0> at the end of the C<pack> list, to be packed with C<C>.
 825  (Attentive readers may have noticed that we could have omitted the 0.)
 826  
 827  For the reverse operation, we'll have to determine the number of items
 828  in the buffer before we can let C<unpack> rip it apart:
 829  
 830     my $n = $env =~ tr/\0// - 1;
 831     my %env = map( split( /=/, $_ ), unpack( "(Z*)$n", $env ) );
 832  
 833  The C<tr> counts the null bytes. The C<unpack> call returns a list of
 834  name-value pairs each of which is taken apart in the C<map> block. 
 835  
 836  
 837  =head2 Counting Repetitions
 838  
 839  Rather than storing a sentinel at the end of a data item (or a list of items),
 840  we could precede the data with a count. Again, we pack keys and values of
 841  a hash, preceding each with an unsigned short length count, and up front
 842  we store the number of pairs:
 843  
 844     my $env = pack( 'S(S/A* S/A*)*', scalar keys( %Env ), %Env );
 845  
 846  This simplifies the reverse operation as the number of repetitions can be
 847  unpacked with the C</> code:
 848  
 849     my %env = unpack( 'S/(S/A* S/A*)', $env );
 850  
 851  Note that this is one of the rare cases where you cannot use the same
 852  template for C<pack> and C<unpack> because C<pack> can't determine
 853  a repeat count for a C<()>-group.
 854  
 855  
 856  =head1 Packing and Unpacking C Structures
 857  
 858  In previous sections we have seen how to pack numbers and character
 859  strings. If it were not for a couple of snags we could conclude this
 860  section right away with the terse remark that C structures don't
 861  contain anything else, and therefore you already know all there is to it.
 862  Sorry, no: read on, please.
 863  
 864  If you have to deal with a lot of C structures, and don't want to
 865  hack all your template strings manually, you'll probably want to have
 866  a look at the CPAN module C<Convert::Binary::C>. Not only can it parse
 867  your C source directly, but it also has built-in support for all the
 868  odds and ends described further on in this section.
 869  
 870  =head2 The Alignment Pit
 871  
 872  In the consideration of speed against memory requirements the balance
 873  has been tilted in favor of faster execution. This has influenced the
 874  way C compilers allocate memory for structures: On architectures
 875  where a 16-bit or 32-bit operand can be moved faster between places in
 876  memory, or to or from a CPU register, if it is aligned at an even or 
 877  multiple-of-four or even at a multiple-of eight address, a C compiler
 878  will give you this speed benefit by stuffing extra bytes into structures.
 879  If you don't cross the C shoreline this is not likely to cause you any
 880  grief (although you should care when you design large data structures,
 881  or you want your code to be portable between architectures (you do want
 882  that, don't you?)).
 883  
 884  To see how this affects C<pack> and C<unpack>, we'll compare these two
 885  C structures:
 886  
 887     typedef struct {
 888       char     c1;
 889       short    s;
 890       char     c2;
 891       long     l;
 892     } gappy_t;
 893  
 894     typedef struct {
 895       long     l;
 896       short    s;
 897       char     c1;
 898       char     c2;
 899     } dense_t;
 900  
 901  Typically, a C compiler allocates 12 bytes to a C<gappy_t> variable, but
 902  requires only 8 bytes for a C<dense_t>. After investigating this further,
 903  we can draw memory maps, showing where the extra 4 bytes are hidden:
 904  
 905     0           +4          +8          +12
 906     +--+--+--+--+--+--+--+--+--+--+--+--+
 907     |c1|xx|  s  |c2|xx|xx|xx|     l     |    xx = fill byte
 908     +--+--+--+--+--+--+--+--+--+--+--+--+
 909     gappy_t
 910  
 911     0           +4          +8
 912     +--+--+--+--+--+--+--+--+
 913     |     l     |  h  |c1|c2|
 914     +--+--+--+--+--+--+--+--+
 915     dense_t
 916  
 917  And that's where the first quirk strikes: C<pack> and C<unpack>
 918  templates have to be stuffed with C<x> codes to get those extra fill bytes.
 919  
 920  The natural question: "Why can't Perl compensate for the gaps?" warrants
 921  an answer. One good reason is that C compilers might provide (non-ANSI)
 922  extensions permitting all sorts of fancy control over the way structures
 923  are aligned, even at the level of an individual structure field. And, if
 924  this were not enough, there is an insidious thing called C<union> where
 925  the amount of fill bytes cannot be derived from the alignment of the next
 926  item alone.
 927  
 928  OK, so let's bite the bullet. Here's one way to get the alignment right
 929  by inserting template codes C<x>, which don't take a corresponding item 
 930  from the list:
 931  
 932    my $gappy = pack( 'cxs cxxx l!', $c1, $s, $c2, $l );
 933  
 934  Note the C<!> after C<l>: We want to make sure that we pack a long
 935  integer as it is compiled by our C compiler. And even now, it will only
 936  work for the platforms where the compiler aligns things as above.
 937  And somebody somewhere has a platform where it doesn't.
 938  [Probably a Cray, where C<short>s, C<int>s and C<long>s are all 8 bytes. :-)]
 939  
 940  Counting bytes and watching alignments in lengthy structures is bound to 
 941  be a drag. Isn't there a way we can create the template with a simple
 942  program? Here's a C program that does the trick:
 943  
 944     #include <stdio.h>
 945     #include <stddef.h>
 946  
 947     typedef struct {
 948       char     fc1;
 949       short    fs;
 950       char     fc2;
 951       long     fl;
 952     } gappy_t;
 953  
 954     #define Pt(struct,field,tchar) \
 955       printf( "@%d%s ", offsetof(struct,field), # tchar );
 956  
 957     int main() {
 958       Pt( gappy_t, fc1, c  );
 959       Pt( gappy_t, fs,  s! );
 960       Pt( gappy_t, fc2, c  );
 961       Pt( gappy_t, fl,  l! );
 962       printf( "\n" );
 963     }
 964  
 965  The output line can be used as a template in a C<pack> or C<unpack> call:
 966  
 967    my $gappy = pack( '@0c @2s! @4c @8l!', $c1, $s, $c2, $l );
 968  
 969  Gee, yet another template code - as if we hadn't plenty. But 
 970  C<@> saves our day by enabling us to specify the offset from the beginning
 971  of the pack buffer to the next item: This is just the value
 972  the C<offsetof> macro (defined in C<E<lt>stddef.hE<gt>>) returns when
 973  given a C<struct> type and one of its field names ("member-designator" in 
 974  C standardese).
 975  
 976  Neither using offsets nor adding C<x>'s to bridge the gaps is satisfactory.
 977  (Just imagine what happens if the structure changes.) What we really need
 978  is a way of saying "skip as many bytes as required to the next multiple of N".
 979  In fluent Templatese, you say this with C<x!N> where N is replaced by the
 980  appropriate value. Here's the next version of our struct packaging:
 981  
 982    my $gappy = pack( 'c x!2 s c x!4 l!', $c1, $s, $c2, $l );
 983  
 984  That's certainly better, but we still have to know how long all the
 985  integers are, and portability is far away. Rather than C<2>,
 986  for instance, we want to say "however long a short is". But this can be
 987  done by enclosing the appropriate pack code in brackets: C<[s]>. So, here's
 988  the very best we can do:
 989  
 990    my $gappy = pack( 'c x![s] s c x![l!] l!', $c1, $s, $c2, $l );
 991  
 992  
 993  =head2 Dealing with Endian-ness
 994  
 995  Now, imagine that we want to pack the data for a machine with a
 996  different byte-order. First, we'll have to figure out how big the data
 997  types on the target machine really are. Let's assume that the longs are
 998  32 bits wide and the shorts are 16 bits wide. You can then rewrite the
 999  template as:
1000  
1001    my $gappy = pack( 'c x![s] s c x![l] l', $c1, $s, $c2, $l );
1002  
1003  If the target machine is little-endian, we could write:
1004  
1005    my $gappy = pack( 'c x![s] s< c x![l] l<', $c1, $s, $c2, $l );
1006  
1007  This forces the short and the long members to be little-endian, and is
1008  just fine if you don't have too many struct members. But we could also
1009  use the byte-order modifier on a group and write the following:
1010  
1011    my $gappy = pack( '( c x![s] s c x![l] l )<', $c1, $s, $c2, $l );
1012  
1013  This is not as short as before, but it makes it more obvious that we
1014  intend to have little-endian byte-order for a whole group, not only
1015  for individual template codes. It can also be more readable and easier
1016  to maintain.
1017  
1018  
1019  =head2 Alignment, Take 2
1020  
1021  I'm afraid that we're not quite through with the alignment catch yet. The
1022  hydra raises another ugly head when you pack arrays of structures:
1023  
1024     typedef struct {
1025       short    count;
1026       char     glyph;
1027     } cell_t;
1028  
1029     typedef cell_t buffer_t[BUFLEN];
1030  
1031  Where's the catch? Padding is neither required before the first field C<count>,
1032  nor between this and the next field C<glyph>, so why can't we simply pack
1033  like this:
1034  
1035     # something goes wrong here:
1036     pack( 's!a' x @buffer,
1037           map{ ( $_->{count}, $_->{glyph} ) } @buffer );
1038  
1039  This packs C<3*@buffer> bytes, but it turns out that the size of 
1040  C<buffer_t> is four times C<BUFLEN>! The moral of the story is that
1041  the required alignment of a structure or array is propagated to the
1042  next higher level where we have to consider padding I<at the end>
1043  of each component as well. Thus the correct template is:
1044  
1045     pack( 's!ax' x @buffer,
1046           map{ ( $_->{count}, $_->{glyph} ) } @buffer );
1047  
1048  =head2 Alignment, Take 3
1049  
1050  And even if you take all the above into account, ANSI still lets this:
1051  
1052     typedef struct {
1053       char     foo[2];
1054     } foo_t;
1055  
1056  vary in size. The alignment constraint of the structure can be greater than
1057  any of its elements. [And if you think that this doesn't affect anything
1058  common, dismember the next cellphone that you see. Many have ARM cores, and
1059  the ARM structure rules make C<sizeof (foo_t)> == 4]
1060  
1061  =head2 Pointers for How to Use Them
1062  
1063  The title of this section indicates the second problem you may run into
1064  sooner or later when you pack C structures. If the function you intend
1065  to call expects a, say, C<void *> value, you I<cannot> simply take
1066  a reference to a Perl variable. (Although that value certainly is a
1067  memory address, it's not the address where the variable's contents are
1068  stored.)
1069  
1070  Template code C<P> promises to pack a "pointer to a fixed length string".
1071  Isn't this what we want? Let's try:
1072  
1073      # allocate some storage and pack a pointer to it
1074      my $memory = "\x00" x $size;
1075      my $memptr = pack( 'P', $memory );
1076  
1077  But wait: doesn't C<pack> just return a sequence of bytes? How can we pass this
1078  string of bytes to some C code expecting a pointer which is, after all,
1079  nothing but a number? The answer is simple: We have to obtain the numeric
1080  address from the bytes returned by C<pack>.
1081  
1082      my $ptr = unpack( 'L!', $memptr );
1083  
1084  Obviously this assumes that it is possible to typecast a pointer
1085  to an unsigned long and vice versa, which frequently works but should not
1086  be taken as a universal law. - Now that we have this pointer the next question
1087  is: How can we put it to good use? We need a call to some C function
1088  where a pointer is expected. The read(2) system call comes to mind:
1089  
1090      ssize_t read(int fd, void *buf, size_t count);
1091  
1092  After reading L<perlfunc> explaining how to use C<syscall> we can write
1093  this Perl function copying a file to standard output:
1094  
1095      require 'syscall.ph';
1096      sub cat($){
1097          my $path = shift();
1098          my $size = -s $path;
1099          my $memory = "\x00" x $size;  # allocate some memory
1100          my $ptr = unpack( 'L', pack( 'P', $memory ) );
1101          open( F, $path ) || die( "$path: cannot open ($!)\n" );
1102          my $fd = fileno(F);
1103          my $res = syscall( &SYS_read, fileno(F), $ptr, $size );
1104          print $memory;
1105          close( F );
1106      }
1107  
1108  This is neither a specimen of simplicity nor a paragon of portability but
1109  it illustrates the point: We are able to sneak behind the scenes and
1110  access Perl's otherwise well-guarded memory! (Important note: Perl's
1111  C<syscall> does I<not> require you to construct pointers in this roundabout
1112  way. You simply pass a string variable, and Perl forwards the address.) 
1113  
1114  How does C<unpack> with C<P> work? Imagine some pointer in the buffer
1115  about to be unpacked: If it isn't the null pointer (which will smartly
1116  produce the C<undef> value) we have a start address - but then what?
1117  Perl has no way of knowing how long this "fixed length string" is, so
1118  it's up to you to specify the actual size as an explicit length after C<P>.
1119  
1120     my $mem = "abcdefghijklmn";
1121     print unpack( 'P5', pack( 'P', $mem ) ); # prints "abcde"
1122  
1123  As a consequence, C<pack> ignores any number or C<*> after C<P>.
1124  
1125  
1126  Now that we have seen C<P> at work, we might as well give C<p> a whirl.
1127  Why do we need a second template code for packing pointers at all? The 
1128  answer lies behind the simple fact that an C<unpack> with C<p> promises
1129  a null-terminated string starting at the address taken from the buffer,
1130  and that implies a length for the data item to be returned:
1131  
1132     my $buf = pack( 'p', "abc\x00efhijklmn" );
1133     print unpack( 'p', $buf );    # prints "abc"
1134  
1135  
1136  
1137  Albeit this is apt to be confusing: As a consequence of the length being
1138  implied by the string's length, a number after pack code C<p> is a repeat
1139  count, not a length as after C<P>. 
1140  
1141  
1142  Using C<pack(..., $x)> with C<P> or C<p> to get the address where C<$x> is
1143  actually stored must be used with circumspection. Perl's internal machinery
1144  considers the relation between a variable and that address as its very own 
1145  private matter and doesn't really care that we have obtained a copy. Therefore:
1146  
1147  =over 4
1148  
1149  =item * 
1150  
1151  Do not use C<pack> with C<p> or C<P> to obtain the address of variable
1152  that's bound to go out of scope (and thereby freeing its memory) before you
1153  are done with using the memory at that address.
1154  
1155  =item * 
1156  
1157  Be very careful with Perl operations that change the value of the
1158  variable. Appending something to the variable, for instance, might require
1159  reallocation of its storage, leaving you with a pointer into no-man's land.
1160  
1161  =item * 
1162  
1163  Don't think that you can get the address of a Perl variable
1164  when it is stored as an integer or double number! C<pack('P', $x)> will
1165  force the variable's internal representation to string, just as if you
1166  had written something like C<$x .= ''>.
1167  
1168  =back
1169  
1170  It's safe, however, to P- or p-pack a string literal, because Perl simply
1171  allocates an anonymous variable.
1172  
1173  
1174  
1175  =head1 Pack Recipes
1176  
1177  Here are a collection of (possibly) useful canned recipes for C<pack>
1178  and C<unpack>:
1179  
1180      # Convert IP address for socket functions
1181      pack( "C4", split /\./, "123.4.5.6" ); 
1182  
1183      # Count the bits in a chunk of memory (e.g. a select vector)
1184      unpack( '%32b*', $mask );
1185  
1186      # Determine the endianness of your system
1187      $is_little_endian = unpack( 'c', pack( 's', 1 ) );
1188      $is_big_endian = unpack( 'xc', pack( 's', 1 ) );
1189  
1190      # Determine the number of bits in a native integer
1191      $bits = unpack( '%32I!', ~0 );
1192  
1193      # Prepare argument for the nanosleep system call
1194      my $timespec = pack( 'L!L!', $secs, $nanosecs );
1195  
1196  For a simple memory dump we unpack some bytes into just as 
1197  many pairs of hex digits, and use C<map> to handle the traditional
1198  spacing - 16 bytes to a line:
1199  
1200      my $i;
1201      print map( ++$i % 16 ? "$_ " : "$_\n",
1202                 unpack( 'H2' x length( $mem ), $mem ) ),
1203            length( $mem ) % 16 ? "\n" : '';
1204  
1205  
1206  =head1 Funnies Section
1207  
1208      # Pulling digits out of nowhere...
1209      print unpack( 'C', pack( 'x' ) ),
1210            unpack( '%B*', pack( 'A' ) ),
1211            unpack( 'H', pack( 'A' ) ),
1212            unpack( 'A', unpack( 'C', pack( 'A' ) ) ), "\n";
1213  
1214      # One for the road ;-)
1215      my $advice = pack( 'all u can in a van' );
1216  
1217  
1218  =head1 Authors
1219  
1220  Simon Cozens and Wolfgang Laun.
1221