World Library  
Flag as Inappropriate
Email this Article

Standard Compression Scheme for Unicode

Article Id: WHEBN0000653715
Reproduction Date:

Title: Standard Compression Scheme for Unicode  
Author: World Heritage Encyclopedia
Language: English
Subject: Comparison of Unicode encodings, Unicode, Byte order mark, UTF-1, UTF-16
Collection: Unicode Transformation Formats
Publisher: World Heritage Encyclopedia
Publication
Date:
 

Standard Compression Scheme for Unicode

The Standard Compression Scheme for Unicode (SCSU)[1] is a Unicode Technical Standard for reducing the number of bytes needed to represent Unicode text, especially if that text uses mostly characters from one or a small number of per-language character blocks. It does so by dynamically mapping values in the range 128–255 to offsets within particular blocks of 128 characters. The initial conditions of the encoder mean that existing strings in ASCII and ISO-8859-1 that do not contain C0 control codes other than NULL TAB CR and LF can be treated as SCSU strings. Since most alphabets do reside in blocks of contiguous Unicode codepoints, texts that use small alphabets and either ASCII punctuation or punctuation that fits within the window for the main alphabet can be encoded at one byte per character (plus setup overhead, which for common languages is often only 1 byte), most other punctuation can be encoded at 2 bytes per symbol through non-locking shifts. SCSU can also switch to UTF-16 internally to handle non-alphabetic languages.

Symbian OS, an operating system for mobile phones and other mobile devices, uses SCSU to serialise strings.

Reuters, the organization that floated the first draft of SCSU, is believed to use SCSU internally.

SQL Server 2008 R2 uses SCSU to compress Unicode values stored in nchar(n) and nvarchar(n) columns, achieving space savings between 15% and 50%, depending on the language of the data.[2]

Comparison with general-purpose plain text compression schemes

In comparison with general-purpose compressors, it is not necessarily advantageous to use SCSU. Few applications need to compress so much Unicode text that it is worth using a special-purpose compression scheme which does not have widespread support. Also, while it can be used as a text encoding, it can be difficult to handle internally.

Treated purely as a compression algorithm, SCSU is inferior to most commonly used general-purpose algorithms for texts of over a few kilobytes.

SCSU does have the advantage that it can usefully compress texts that are only a few characters long, whereas most full-scale compressors need hundreds of bytes of data to break even against their own overhead. In Symbian OS, SCSU is used even for Clipboard operations, e.g. Cut, Copy & Paste of small strings of text.

See also

References

  1. ^ "UTS #6: Compression Scheme for Unicode". 2005-05-06. Retrieved 2008-06-13. 
  2. ^ "Unicode Compression Implementation (SQL Server 2008 R2 Books Online)". Retrieved 2008-08-18. 
This article was sourced from Creative Commons Attribution-ShareAlike License; additional terms may apply. World Heritage Encyclopedia content is assembled from numerous content providers, Open Access Publishing, and in compliance with The Fair Access to Science and Technology Research Act (FASTR), Wikimedia Foundation, Inc., Public Library of Science, The Encyclopedia of Life, Open Book Publishers (OBP), PubMed, U.S. National Library of Medicine, National Center for Biotechnology Information, U.S. National Library of Medicine, National Institutes of Health (NIH), U.S. Department of Health & Human Services, and USA.gov, which sources content from all federal, state, local, tribal, and territorial government publication portals (.gov, .mil, .edu). Funding for USA.gov and content contributors is made possible from the U.S. Congress, E-Government Act of 2002.
 
Crowd sourced content that is contributed to World Heritage Encyclopedia is peer reviewed and edited by our editorial staff to ensure quality scholarly research articles.
 
By using this site, you agree to the Terms of Use and Privacy Policy. World Heritage Encyclopedia™ is a registered trademark of the World Public Library Association, a non-profit organization.
 


Copyright © World Library Foundation. All rights reserved. eBooks from Project Gutenberg are sponsored by the World Library Foundation,
a 501c(4) Member's Support Non-Profit Organization, and is NOT affiliated with any governmental agency or department.