[BACK]Return to Overview.html CVS log [TXT] [DIR] Up to [Public] / charlint
File:  [Public] / charlint / Overview.html
Revision 1.37: download - view: text, annotated - select for diffs
Mon Jul 24 07:24:24 2000 UTC (23 years, 10 months ago) by duerst
Branches: MAIN
CVS tags: HEAD
(duerst) Changed through Jigsaw.

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"
                      "http://www.w3.org/TR/REC-html40/loose.dtd">
<html>
<head>
  <meta http-equiv="Content-Type" content="text/html">
  <meta http-equiv="Content-Style-Type" content="text/css">
  <!--BASE href="http://www.w3.org/Consortium/Translation/"-->
  <!--LINK rel="stylesheet" href="../i18n.css"-->
  <style type="text/css">  <!--
H1.title {text-align: center }
P.toolbar { text-align: center }
DIV.deliverable { margin-left: 2em;
margin-right: 2em }
P.note { margin-left: 10%;
margin-right: 10%;
color: green }
TH { text-align: left }
TH, TD { padding: 2px }
.external { font-style: italic }
-->

  </style>
  <title>Charlint - A Character Normalization Tool</title>
  <link rel="stylesheet" type="text/css" href="../../StyleSheets/base.css">
</head>

<body bgcolor="#FFFFFF" text="#000000">
<p><a href="/"><img border="0" src="/Icons/WWW/w3c_home" alt="W3C" width="72"
height="48"></a> <a href="/International"><img src="/Icons/WWW/i18n-alt"
alt="International" width="72" height="48" border="0"></a></p>

<h1>Charlint - A Character Normalization Tool</h1>

<p><a href="#Perl">Perl source</a> | <a href="#Recommended">Recommended Data
Files</a> | <a href="#How">How to use</a> | <a href="#Future">Future Plans</a>
| <a href="#Background">Background </a>| <a href="#Version">Version
History</a></p>

<p><strong>IMPORTANT</strong>: New version 0.34, fixes serious bug (24 July
2000)</p>

<p>Charlint is a character normalization/checking tool written in Perl. Among
else, it implements Normalization Form C of <a
href="http://www.unicode.org/unicode/reports/tr15/">Unicode TR 15</a>.</p>

<h3><a name="Perl">Perl Source</a> and Installation</h3>

<p>Charlint , aka 'Charlie', is written in <a
href="http://www.perl.com/pace/pub/perldocs/latest.html">Perl 5</a>. You can
get the source from <a
href="http://www.w3.org/International/charlint/charlint.pl">http://www.w3.org/International/charlint/charlint.pl</a>.
Charlint is covered by the <a
href="http://www.w3.org/Consortium/Legal/copyright-software.html">W3C software
licence</a>. To install charlint, please make sure you have installed <a
href="http://www.perl.com/pace/pub/perldocs/latest.html">Perl 5</a>, you have
downloaded an appropriate character data file, and you have downloaded the <a
href="http://www.w3.org/International/charlint/charlint.pl">Perl source</a>.
Please send error reports or comments to <a
href="mailto:duerst@w3.org">duerst@w3.org</a>; for anouncements and public
discussion please see the Winter mailing list (www-international@w3.org).</p>

<h3><a name="Recommended">Recommended Character Data Files</a></h3>

<p>Charlint needs information on characters in order to work correctly. To
indicate the file you want to use, please use the -f option. The currently
recommended character data file is available from <a
href="ftp://ftp.unicode.org/Public/3.0-Update/UnicodeData-3.0.0.beta.txt">ftp://ftp.unicode.org/Public/3.0-Update/UnicodeData-3.0.0.beta.txt</a>.
Composition exclusions are currently hard-coded and are based on <a
href="ftp://ftp.unicode.org/Public/3.0-Update/CompositionExclusions-1.beta.txt">ftp://ftp.unicode.org/Public/3.0-Update/CompositionExclusions-1.beta.txt</a>
[version of 04 Aug 1999 for both files]. Additional information on these and
other files can be found at <a
href="http://www.unicode.org/unicode/standard/versions/Unicode3.0-beta.html">http://www.unicode.org/unicode/standard/versions/Unicode3.0-beta.html</a>.
Please note that this data file is a beta version; the beta test will last up
to 15 August 1999. Using charlint is one way to test this data file. Please
send any comments on <em>the data file</em> to <a
href="mailto:errata@unicode.org">errata@unicode.org</a>.</p>

<h3><a name="How">How to use charlint</a></h3>

<p>Charlint is a perl script that works as a simple filter. It uses UTF-8 both
for input and for output. Behaviour can be fine-tuned with various options. A
list of options such as the one below can be optained by using <kbd>charlint
-h</kbd>.</p>
<pre>(options prefixed by # are currently not available)
-b: Remove initial 'Byte Order Mark'
-B: Supress warning about initial 'Byte Order Mark'
-d: Debug: Thoroughly check character data table input
-D: Leave after reading in character data
-e: # remove undefined codepoints
-E: Do not warn about undefined codepoints
-f file: Read data from file
         (please use newest V3.0 beta datafiles)
-C: # Do not normalize
-h: Prints out this short description
-n: Accept &amp;#ddddd; and &amp;#xhhhh; on input
        (beware of &lt;![CDATA[, &lt;SCRIPT>, &lt;STYLE>)
-N: Produce &amp;#xhh; on output
-o: Print out 'unprintable' bytes as octal
-p: # Remove stuff in private zone
-P: Supress checking private zone
-u: # Fix UTF-8 (convert or remove)
-U: Supress checking correctness of UTF-8
-v: Print version</pre>

<h3><a name="Version">Version History</a></h3>
<pre># 1999/06/23: 0.30, preparation for W3C member test, without Hangul  MJD
# 1999/06/25: 0.31, fixed reordering bug, going public               MJD
# 1999/07/01: 0.32, adapted surrogates/exclusions to 3.0.0.beta      MJD<br>
# 1999/08/16: 0.33, updated for second version of 3.0.0.beta         MJD
# 2000/07/24: 0.34, $chClass = $CombClass{ch}; should read $chClass = $CombClass{$ch};
#                   implemented -C                                   MJD</pre>

<h3><a name="Background">Background</a></h3>
<ul>
  <li><a href="http://www.w3.org/TR/WD-charmod">Character Model for the World
    Wide Web</a> (W3C Working Draft)</li>
  <li><a href="http://www.unicode.org/unicode/reports/tr15/">Unicode Technical
    Report #15</a> (Version 14 approved subject to various changes)</li>
  <li><a href="http://www.w3.org/Status">W3C Open Source Releases</a></li>
</ul>

<h3><a name="Future">Future Plans</a></h3>

<p>We have just released the first version of charlint. There are many things
we plan to add in the future:</p>
<ul>
  <li>Hangul syllable normalization</li>
  <li>Removal of undefined codepoints and codepoints in the private zone</li>
  <li>Removal/fix of incorrect UTF-8</li>
  <li>Compatibility character detection or removal.</li>
  <li>Detection or removal of characters not suitable for markup.</li>
</ul>

<p>Your help (bug reports, patches, ideas, test cases) is welcome.</p>
<hr>

<address>
  <a href="mailto:duerst@w3.org">Martin Dürst</a> <br>
  <a href="/Help/Webmaster.html">Webmaster</a> <br>
  last revised $Date: 2000/07/24 07:24:24 $ by $Author: duerst $ 
</address>

<p class="policyfooter"><small><a
href="/Consortium/Legal/ipr-notice.html#Copyright">Copyright</a>  ©  1997 <a
href="http://www.w3.org">W3C</a> (<a href="http://www.lcs.mit.edu">MIT</a>, <a
href="http://www.inria.fr/">INRIA</a>, <a
href="http://www.keio.ac.jp/">Keio</a> ), All Rights Reserved. W3C <a
href="/Consortium/Legal/ipr-notice.html#Legal Disclaimer">liability,</a> <a
href="/Consortium/Legal/ipr-notice.html#W3C Trademarks">trademark</a>, <a
href="/Consortium/Legal/copyright-documents.html">document use </a>and <a
href="/Consortium/Legal/copyright-software.html">software licensing </a>rules
apply. Your interactions with this site are in accordance with our <a
href="/Consortium/Legal/privacy-statement.html#Public">public</a> and <a
href="/Consortium/Legal/privacy-statement.html#Members">Member</a> privacy
statements.</small></p>
</body>
</html>
Webmaster