GNU IDN Library - Libidn

Introduction

GNU Libidn is a fully documented implementation of the Stringprep, Punycode and IDNA 2003 specifications. Libidn's purpose is to encode and decode internationalized domain names. The native C, C# and Java libraries are available under the GNU Lesser General Public License version 2.1 or later.

Please be aware that GNU libidn2 is the successor of GNU libidn. It comes with IDNA 2008 and TR46 implementation and also provides a compatibility layer for GNU libidn.

The library contains a generic Stringprep implementation. Profiles for Nameprep, iSCSI, SASL, XMPP and Kerberos V5 are included. Punycode and ASCII Compatible Encoding (ACE) via IDNA are supported. A mechanism to define Top-Level Domain (TLD) specific validation tables, and to compare strings against those tables, is included. Default tables for some TLDs are also included.

The Stringprep API consists of two main functions, one for converting data from the system's native representation into UTF-8, and one function to perform the Stringprep processing. Adding a new Stringprep profile for your application within the API is straightforward. The Punycode API consists of one encoding function and one decoding function. The IDNA API consists of the ToASCII and ToUnicode functions, as well as an high-level interface for converting entire domain names to and from the ACE encoded form. The TLD API consists of one set of functions to extract the TLD name from a domain string, one set of functions to locate the proper TLD table to use based on the TLD name, and core functions to validate a string against a TLD table, and some utility wrappers to perform all the steps in one call.

The library is used by, e.g., GNU SASL and Shishi to process user names and passwords. Libidn can be built into GNU Libc to enable a new system-wide getaddrinfo flag for IDN processing.

Libidn is developed for the GNU/Linux system, but runs on over 20 Unix platforms (including Solaris, IRIX, AIX, and Tru64) and Windows. The library is written in C and (parts of) the API is also accessible from C++, Emacs Lisp, Python and Java. A native Java and C# port is included.

Also included is a command line tool, several self tests, code examples, and more, all licensed under the GNU General Public License version 3.0 or later.

Table of Contents

News

Note that new releases are only mentioned here if they introduce a major feature or is significant in some other way. Read the help-libidn mailing list if you seek more frequent announcements.

Information on what is new in the library itself is found in the NEWS file (live version).

Try it

A web interface to libidn is available online. Try libidn before you buy it.

A simple IDN web server is also available.

Documentation

Refer to the Libidn Manual web page for links to the manual in all formats; however, quick links to the most popular formats:

You may also be interested in a preliminary document with Nameprep and IDNA test vectors.

See also the various standard texts:

Downloading

Libidn can be found on https://ftp.gnu.org/gnu/libidn/ [via HTTPS] and ftp://ftp.gnu.org/gnu/libidn/ [via FTP]. It can also be found on one of our FTP mirrors; please use a mirror if possible.

All official releases are signed with an OpenPGP key with fingerprint 0xB565716F or with OpenPGP key 0x08302DB6A2670428.

Support

A mailing list where users of Libidn may help each other exists, and you can reach it by sending e-mail to [email protected]. Archives of the mailing list discussions, and an interface to manage subscriptions, is available through the World Wide Web at https://lists.gnu.org/mailman/listinfo/help-libidn.

If you are interested in paid support for Libidn, or sponsor the development, please contact me. If you provide paid services for Libidn, and would like to be mentioned here, also contact me.

If you find Libidn useful, please consider making a donation. No amount is too small!

Development

There is a Savannah Libidn project page. You can check out the sources by using git as follows: 

$ git clone git://git.savannah.gnu.org/libidn.git

The online git interface is available.

For every release, we publish cyclomatic code complexity charts for the package. There is also self-test code coverage charts available. Finally, clang-analyzer output is also available.

Bugs

Report all problems to [email protected], but please read the manual on how to report bugs first.

Related implementations

The following is a list of links to other free IDN, or otherwise related, implementations. The list is not conclusive, suggestions appreciated.

Projects using GNU Libidn include:

Projects using libidn2 include:

Let us know about more projects that use GNU Libidn!

How to use it?

Read data from user, convert it to UTF-8 and then pass it to stringprep(). Example code below (it is included in the distribution as example.c). To simplify compiling, use libtool and pkg-config. More information and more examples are included in the manual.

See also the other example*.c files in the source distribution on how to use other features of the library (punycode, IDNA). 

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

/*
 * Compiling using libtool and pkg-config is recommended:
 *
 * $ libtool cc -o example example.c `pkg-config --cflags --libs libidn`
 * $ ./example
 * Input string encoded as `ISO-8859-1': ª
 * Before locale2utf8 (length 2): aa 0a
 * Before stringprep (length 3): c2 aa 0a
 * After stringprep (length 2): 61 0a
 * $
 *
 */

int main(int argc, char *argv[])
{
  char buf[BUFSIZ];
  char *p;
  int rc, i;

  printf("Input string encoded as `%s': ",
	 stringprep_locale_charset ());
  fflush(stdout);
  fgets(buf, BUFSIZ, stdin);

  printf("Before locale2utf8 (length %d): ", strlen(buf));
  for (i=0; i < strlen(buf); i++)
    printf("%02x ", buf[i] & 0xFF);
  printf("\n");

  p = stringprep_locale_to_utf8 (buf);
  if (p)
    {
      strcpy(buf, p);
      free(p);
    }
  else
    printf("Could not convert string to UTF-8, continuing anyway...\n");

  printf("Before stringprep (length %d): ", strlen(buf));
  for (i=0; i < strlen(buf); i++)
    printf("%02x ", buf[i] & 0xFF);
  printf("\n");

  rc = stringprep(buf, BUFSIZ, 0, stringprep_nameprep);
  if (rc != STRINGPREP_OK)
    printf("Stringprep failed with rc %d...\n", rc);
  else
    {
      printf("After stringprep (length %d): ", strlen(buf));
      for (i=0; i < strlen(buf); i++)
        printf("%02x ", buf[i] & 0xFF);
      printf("\n");
    }

  return 0;
}

Libidn2

Libidn2 is an implementation of the IDNA2008 + TR46 specifications (RFC 5890, RFC 5891, RFC 5892, RFC 5893, TR 46). Libidn2 is a standalone library, without any dependency on Libidn. Libidn2 is believed to be a complete IDNA2008 / TR46 implementation, but has yet to be as extensively used as the original Libidn library.

Libidn2 uses GNU libunistring for Unicode processing and GNU libiconv for character set conversion.

Libidn2 can be downloaded from https://ftp.gnu.org/gnu/libidn/ [via HTTPS] and ftp://ftp.gnu.org/gnu/libidn/ [via FTP]. It can also be found on one of our FTP mirrors; please use a mirror if possible.

For documentation see the Libidn2 Manual web page, but quick links to the most popular formats:

For development, see the Libidn2 GitLab project page.

For Quality Assurance, we publish code coverage report and clang static analyzer output.

Initial development of Libidn2 has been sponsored by DENIC.

Valid XHTML 1.0 Strict