charlint/Overview.html - view

File: [Public] / charlint / Overview.html
Revision 1.23: download - view: text, annotated - select for diffs
Thu Jul 1 09:06:41 1999 UTC (24 years, 11 months ago) by duerst
Branches: MAIN
CVS tags: HEAD

(duerst) Changed through Jigsaw.

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd"> <HTML> <HEAD> <META http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"> <META http-equiv="Content-Style-Type" content="text/css">   <STYLE type="text/css">  </STYLE> <TITLE>Charlint - A Character Normalization Tool</TITLE> <LINK rel="stylesheet" type="text/css" href="../../StyleSheets/base.css"> </HEAD> <BODY bgcolor="#FFFFFF" text="#000000"> <P> <A HREF="/"><IMG BORDER="0" ALT="W3C" WIDTH="72" HEIGHT="48" SRC="w3c_home"></A> <H1> Charlint - A Character Normalization Tool </H1> <P> <A href="#Perl">Perl source</A> | <A HREF="#Recommended">Recommended Data Files</A> | <A HREF="#How">How to use</A> | <A HREF="#Future">Future Plans</A> | Background | <A HREF="#Version">Version History</A> <H3> <A NAME="Perl">Perl Source</A> and Installation </H3> <P> Charlint is writen in <A HREF="http://www.perl.com/pace/pub/perldocs/latest.html">Perl 5</A>. You can get the source from <A HREF="http://www.w3.org/International/charlint/charlint.pl">http://www.w3.org/International/charlint/charlint.pl</A>. Charlint is covered by the <A HREF="http://www.w3.org/Consortium/Legal/copyright-software.html">W3C software licence</A>. To install charlint, please make sure you have installed <A HREF="http://www.perl.com/pace/pub/perldocs/latest.html">Perl 5</A>, you have downloaded an appropriate character data file, and you have downloaded the <A HREF="http://www.w3.org/International/charlint/charlint.pl">Perl source</A>. Please send error reports or comments to <A HREF="mailto:duerst@w3.org">duerst@w3.org</A>; for anouncements and public discussion please see the Winter mailing list (www-international@w3.org). <H3> <A NAME="Recommended">Recommended Character Data Files</A> </H3> <P> Charlint needs information on characters in order to work correctly. To indicate the file you want to use, please use the -f option. The currently recommended character data file is available from <A HREF="ftp://ftp.unicode.org/Public/3.0-Update/UnicodeData-3.0.0.beta.txt">ftp://ftp.unicode.org/Public/3.0-Update/UnicodeData-3.0.0.beta.txt</A>. Composition exclusions are currently hard-coded and are based on <A HREF="ftp://ftp.unicode.org/Public/3.0-Update/CompositionExclusions-1.beta.txt">ftp://ftp.unicode.org/Public/3.0-Update/CompositionExclusions-1.beta.txt</A>. Additional information on these and other files can be found at <A HREF="http://www.unicode.org/unicode/standard/versions/Unicode3.0-beta.html">http://www.unicode.org/unicode/standard/versions/Unicode3.0-beta.html</A>. Please note that this data file is a beta version; the beta test will last up to 15 August 1999. Using charlint is one way to test this data file. Please send any comments on <EM>the data file</EM> to <A HREF="mailto:errata@unicode.org">errata@unicode.org</A>. <H3> <A NAME="How">How to use charlint</A> </H3> <P> Charlint is a perl script that works as a simple filter. It uses UTF-8 both for input and for output. Behaviour can be fine-tuned with various options. A list of options such as the one below can be optained by using <KBD>charlint -h</KBD>. <PRE>(options prefixed by # are currently not available) -b: Remove initial 'Byte Order Mark' -B: Supress warning about initial 'Byte Order Mark' -d: Debug: Thoroughly check character data table input -D: Leave after reading in character data -e: # remove undefined codepoints -E: Do not warn about undefined codepoints -f file: Read data from file (please use newest V3.0 beta datafiles) -C: # Do not normalize -h: Prints out this short description -n: Accept &#ddddd; and &#xhhhh; on input (beware of <![CDATA[, <SCRIPT>, <STYLE>) -N: Produce &#xhh; on output -o: Print out 'unprintable' bytes as octal -p: # Remove stuff in private zone -P: Supress checking private zone -u: # Fix UTF-8 (convert or remove) -U: Supress checking correctness of UTF-8 -v: Print version </PRE> <H3> <A NAME="Version">Version History</A> </H3> <PRE># 1999/06/23: 0.30, preparation for W3C member test, without Hangul MJD # 1999/06/25: 0.31, fixed reordering bug, going public MJD # 1999/07/01: 0.32, adapted surrogates/exclusions to 3.0.0.beta MJD </PRE> <H3> <A NAME="Future">Future Plans</A> </H3> <P> We have just released the first version of charlint. There are many things we plan to add in the future: <UL> <LI> Hangul syllable normalization <LI> Removal of undefined codepoints and codepoints in the private zone <LI> Removal/fix of incorrect UTF-8 <LI> Incorporate knowledge of V3.0 decompositions to automatically detect future precomposites. <LI> Compatibility character detection or removal. <LI> Detection or removal of characters not suitable for markup. </UL> <P> <HR> <ADDRESS> <A HREF="mailto:duerst@w3.org">Martin Dürst</A> <BR> <A HREF="../Help/Webmaster.html">Webmaster</A> <BR> last revised $Date: 1999/07/01 09:06:41 $ by $Author: duerst $ </ADDRESS> <P class=policyfooter> <SMALL><A href="/Consortium/Legal/ipr-notice.html#Copyright">Copyright</A>  ©  1997 <A href="http://www.w3.org">W3C</A> (<A href="http://www.lcs.mit.edu">MIT</A>, <A href="http://www.inria.fr/">INRIA</A>, <A href="http://www.keio.ac.jp/">Keio</A> ), All Rights Reserved. W3C <A href="/Consortium/Legal/ipr-notice.html#Legal Disclaimer">liability,</A> <A href="/Consortium/Legal/ipr-notice.html#W3C Trademarks">trademark</A>, <A href="/Consortium/Legal/copyright-documents.html">document use </A>and <A href="/Consortium/Legal/copyright-software.html">software licensing </A>rules apply. Your interactions with this site are in accordance with our <A href="/Consortium/Legal/privacy-statement.html#Public">public</A> and <A href="/Consortium/Legal/privacy-statement.html#Members">Member</A> privacy statements.</SMALL> </BODY></HTML>