1
package Encode::Encoding;
3
# Base class for classes which implement encodings
6
our $VERSION = do { my @r = ( q$Revision: 2.5 $ =~ /\d+/g ); sprintf "%d." . "%02d" x $#r, @r };
14
my $canonical = shift;
15
$obj = bless { Name => $canonical }, $obj unless ref $obj;
17
# warn "$canonical => $obj\n";
18
Encode::define_encoding( $obj, $canonical, @_ );
21
sub name { return shift->{'Name'} }
24
require Encode::MIME::Name;
25
return Encode::MIME::Name::get_mime_name(shift->name);
28
# sub renew { return $_[0] }
32
my $clone = bless {%$self} => ref($self);
33
$clone->{renewed}++; # so the caller can see it
34
DEBUG and warn $clone->{renewed};
38
sub renewed { return $_[0]->{renewed} || 0 }
40
*new_sequence = \&renew;
45
eval { require PerlIO::encoding };
49
# (Temporary|legacy) methods
51
sub toUnicode { shift->decode(@_) }
52
sub fromUnicode { shift->encode(@_) }
55
# Needs to be overloaded or just croak
61
my $class = ref($obj) ? ref($obj) : $obj;
62
Carp::croak( $class . "->encode() not defined!" );
68
my $class = ref($obj) ? ref($obj) : $obj;
69
Carp::croak( $class . "->encode() not defined!" );
79
Encode::Encoding - Encode Implementation Base Class
83
package Encode::MyEncoding;
84
use base qw(Encode::Encoding);
86
__PACKAGE__->Define(qw(myCanonical myAlias));
90
As mentioned in L<Encode>, encodings are (in the current
91
implementation at least) defined as objects. The mapping of encoding
92
name to object is via the C<%Encode::Encoding> hash. Though you can
93
directly manipulate this hash, it is strongly encouraged to use this
94
base class module and add encode() and decode() methods.
96
=head2 Methods you should implement
98
You are strongly encouraged to implement methods below, at least
99
either encode() or decode().
103
=item -E<gt>encode($string [,$check])
105
MUST return the octet sequence representing I<$string>.
111
If I<$check> is true, it SHOULD modify I<$string> in place to remove
112
the converted part (i.e. the whole string unless there is an error).
113
If perlio_ok() is true, SHOULD becomes MUST.
117
If an error occurs, it SHOULD return the octet sequence for the
118
fragment of string that has been converted and modify $string in-place
119
to remove the converted part leaving it starting with the problem
120
fragment. If perlio_ok() is true, SHOULD becomes MUST.
124
If I<$check> is is false then C<encode> MUST make a "best effort" to
125
convert the string - for example, by using a replacement character.
129
=item -E<gt>decode($octets [,$check])
131
MUST return the string that I<$octets> represents.
137
If I<$check> is true, it SHOULD modify I<$octets> in place to remove
138
the converted part (i.e. the whole sequence unless there is an
139
error). If perlio_ok() is true, SHOULD becomes MUST.
143
If an error occurs, it SHOULD return the fragment of string that has
144
been converted and modify $octets in-place to remove the converted
145
part leaving it starting with the problem fragment. If perlio_ok() is
146
true, SHOULD becomes MUST.
150
If I<$check> is false then C<decode> should make a "best effort" to
151
convert the string - for example by using Unicode's "\x{FFFD}" as a
152
replacement character.
158
If you want your encoding to work with L<encoding> pragma, you should
159
also implement the method below.
163
=item -E<gt>cat_decode($destination, $octets, $offset, $terminator [,$check])
165
MUST decode I<$octets> with I<$offset> and concatenate it to I<$destination>.
166
Decoding will terminate when $terminator (a string) appears in output.
167
I<$offset> will be modified to the last $octets position at end of decode.
168
Returns true if $terminator appears output, else returns false.
172
=head2 Other methods defined in Encode::Encodings
174
You do not have to override methods shown below unless you have to.
182
sub name { return shift->{'Name'} }
184
MUST return the string representing the canonical name of the encoding.
186
=item -E<gt>mime_name
191
require Encode::MIME::Name;
192
return Encode::MIME::Name::get_mime_name(shift->name);
195
MUST return the string representing the IANA charset name of the encoding.
203
my $clone = bless { %$self } => ref($self);
208
This method reconstructs the encoding object if necessary. If you need
209
to store the state during encoding, this is where you clone your object.
211
PerlIO ALWAYS calls this method to make sure it has its own private
218
sub renewed { $_[0]->{renewed} || 0 }
220
Tells whether the object is renewed (and how many times). Some
221
modules emit C<Use of uninitialized value in null operation> warning
222
unless the value is numeric so return 0 for false.
224
=item -E<gt>perlio_ok()
229
eval{ require PerlIO::encoding };
233
If your encoding does not support PerlIO for some reasons, just;
237
=item -E<gt>needs_lines()
241
sub needs_lines { 0 };
243
If your encoding can work with PerlIO but needs line buffering, you
244
MUST define this method so it returns true. 7bit ISO-2022 encodings
245
are one example that needs this. When this method is missing, false
250
=head2 Example: Encode::ROT13
252
package Encode::ROT13;
254
use base qw(Encode::Encoding);
256
__PACKAGE__->Define('rot13');
259
my ($obj, $str, $chk) = @_;
260
$str =~ tr/A-Za-z/N-ZA-Mn-za-m/;
261
$_[1] = '' if $chk; # this is what in-place edit means
265
# Jr pna or ynml yvxr guvf;
270
=head1 Why the heck Encode API is different?
272
It should be noted that the I<$check> behaviour is different from the
273
outer public API. The logic is that the "unchecked" case is useful
274
when the encoding is part of a stream which may be reporting errors
275
(e.g. STDERR). In such cases, it is desirable to get everything
276
through somehow without causing additional errors which obscure the
277
original one. Also, the encoding is best placed to know what the
278
correct replacement character is, so if that is the desired behaviour
279
then letting low level code do it is the most efficient.
281
By contrast, if I<$check> is true, the scheme above allows the
282
encoding to do as much as it can and tell the layer above how much
283
that was. What is lacking at present is a mechanism to report what
284
went wrong. The most likely interface will be an additional method
285
call to the object, or perhaps (to avoid forcing per-stream objects
286
on otherwise stateless encodings) an additional parameter.
288
It is also highly desirable that encoding classes inherit from
289
C<Encode::Encoding> as a base class. This allows that class to define
290
additional behaviour for all encoding objects.
292
package Encode::MyEncoding;
293
use base qw(Encode::Encoding);
295
__PACKAGE__->Define(qw(myCanonical myAlias));
297
to create an object with C<< bless {Name => ...}, $class >>, and call
298
define_encoding. They inherit their C<name> method from
301
=head2 Compiled Encodings
303
For the sake of speed and efficiency, most of the encodings are now
304
supported via a I<compiled form>: XS modules generated from UCM
305
files. Encode provides the enc2xs tool to achieve that. Please see
306
L<enc2xs> for more details.
310
L<perlmod>, L<enc2xs>
318
The fixup routine gets passed the remaining fragment of string being
319
processed. It modifies it in place to remove bytes/characters it can
320
understand and returns a string used to represent them. For example:
323
my $ch = substr($_[0],0,1,'');
324
return sprintf("\x{%02X}",ord($ch);
327
This scheme is close to how the underlying C code for Encode works,
328
but gives the fixup routine very little context.
332
The fixup routine gets passed the original string, an index into
333
it of the problem area, and the output string so far. It appends
334
what it wants to the output string and returns a new index into the
335
original string. For example:
338
# my ($s,$i,$d) = @_;
339
my $ch = substr($_[0],$_[1],1);
340
$_[2] .= sprintf("\x{%02X}",ord($ch);
344
This scheme gives maximal control to the fixup routine but is more
345
complicated to code, and may require that the internals of Encode be tweaked to
346
keep the original string intact.
350
Hybrids of the above.
352
Multiple return values rather than in-place modifications.
354
Index into the string could be C<pos($str)> allowing C<s/\G...//>.
added
removed
lib/Encode/Encoding.pm
