Multibyte support

From Swishewiki

You do not have permission to edit this page, for the following reasons:

• The action you have requested is limited to users in the group: Users.
• You must confirm your e-mail address before editing pages. Please set and validate your e-mail address through your user preferences.

You can view and copy the source of this page:

Multibyte support is a sorely missing feature in SWISH-E. It is understood that adding multibyte support (UTF-8 is preferred) to Swish-e is a non-trivial task. Perhaps somehow the code in SWISH-E can slowly be massaged to support UTF-8. This page will document ideas for a rational approach on how to add UTF-8 support to SWISH-E. Conceptually, perhaps the lowest levels can be rewritten first, progressively working up through the system removing non-UTF-8 assumptions. Perhaps a step towards that goal is some sort of documentation explaning the internal architecture of Swish-e -- which files do what, and roughly how. * Q: Is there any such documentation at this point? * A: Nothing formal. Only some commenting in the source code. To get an idea of what programming in a UTF-8-aware mode entails, have a look at this short [[Unicode C Example]]. ==Multibyte Links and Reference Material== '''"[[man utf8]]"''' output from Fedora Core 1. * "It can be hoped that in the foreseeable future, UTF-8 will replace ASCII and ISO 8859 at all levels" in POSIX. --Markus Kuhn, 2001 '''"[[man charsets]]"''' output from Fedora Core 1. * "Linux is an international operating system." [http://www.cl.cam.ac.uk/~mgk25/ucs/utf-8-history.txt UTF-8 History] by "Rob 'Commander' Pike", 2003 [http://www-106.ibm.com/developerworks/linux/library/l-linuni.html Linux Unicode programming], by Thomas W. Burger, 2001 [http://www.cl.cam.ac.uk/~mgk25/unicode.html UTF-8 and Unicode FAQ for Unix/Linux], Marcus Kuhn, 1999-2005. Website for the book [http://www.unicode.org/versions/Unicode4.0.0/ The Unicode Standard, Version 4.0] By the Unicode Consortium, Et. al., 2003 The document [http://www.gnu.org/software/libc/manual/html_node/Extended-Char-Intro.html The Gnu C Libraries "Introduction to Extended Characters"] may prove useful for this discussion. Perhaps it will also be useful to examine case studies of other software packages that have made the step from single-byte to UTF-8 aware. Ones that come to mind as having successfully made the transition include '''[http://mysql.com mysql]''' and '''[http://perl.org perl]'''.

Return to Multibyte support.