In mobile telephony GSM 03.38 or 3GPP 23.038 is a character encoding used in GSM networks for SMS (Short Message Service), CB (Cell Broadcast) and USSD (Unstructured Supplementary Service Data). The 3GPP TS 23.038 standard (originally GSM recommendation 03.38) defines GSM 7-bit default alphabet which is mandatory for GSM handsets and network elements, but the character set is suitable only for English and a number of Western-European languages. Languages such as Chinese, Korean or Japanese must be transferred using the 16-bit UCS-2 character encoding. A limited number of languages, like Portuguese, Spanish, Turkish and a number of languages used in India written with a Brahmic scripts may use 7-bit encoding with national language shift table defined in 3GPP 23.038. For binary messages, 8-bit encoding is used.
GSM 7-bit default alphabet and extension table of 3GPP TS 23.038 / GSM 03.38
The standard encoding for GSM messages is the 7-bit default alphabet as defined in the 23.038 recommendation.
Seven-bit characters must be encoded into octets following one of three packing modes:
CBS: using this encoding, it is possible to send up to 93 characters (packed in up to 82 octets) in one SMS message in a Cell Broadcast Service. SMS: using this encoding, it is possible to send up to 160 characters (packed in up to 140 octets) in one SMS message in the GSM network. USSD: using this encoding, it is possible to send up to 182 characters (packed in up to 160 octets) in one SMS message of Unstructured Supplementary Service Data.
It is important (especially when a message is to be segmented using concatenated SMS mechanism) that characters from the Basic Character Set table take one septet, characters from the Basic Character Set Extension table take two septets.
Note that the second part of the table is only accessible if the GSM device supports the 7-bit extension mechanism, using the ESC character prefix. Otherwise, the ESC code itself is interpreted as a space, and the following character will be treated as if there was no leading ESC code.
Most of the high part of the table is not used in the default character set, but the GSM standard defines some language code indicators that allows the system to identify national variants of this part, to support more characters than those displayed in the above table.
In a standard GSM text message, all characters are encoded using 7-bit code units, packed together to fill all bits of octets. So, for example, the 140-octet envelope of an SMS, with no other language indicator but only the standard class prefix, can transport up to (140*8)/7=160, that is 160 GSM 7-bit characters (but note that the ESC code counts for one of them, if characters in the high part of the table are used).
Longer messages may be sent, but will require a continuation prefix and a sequence number on subsequent SMS messages (these prefix bytes and sequence number are counted within the maximum length of the 140-octet payload of the envelope format).
When there are 1 to 6 spare bits in the last octet of a message, these bits are set to zero (these bits do not count as a character but only as a filler). When there are 7 spare bits in the last octet of a message, these bits are set to the 7-bit code of the CR control (also used as a padding filler) instead of being set to zero (where they would be confused with the 7-bit code of an '@' character).
This 7-bit encoding allows the transport of texts consisting of printable characters from Basic Latin (Unicode block) (with the exception of the grave accent/backtick), as well as some characters of the ISO Latin 1 character set. It also allows the encoding of texts written in the Greek script, but only capitals; for such use in Greek, the Latin capital letters that look like the Greek letters are reused with the same code, so that the above character set is complete only for modern monotonic Greek restricted to capital letters. A complete support for the Greek alphabet (including small letters) requires a national version of the shifted 7-bit table (using the ESC code for each national character encoded in this shifted table), or an unspecified proprietary 8-bit encoding, or the use of the UCS-2 encoding (see below).
Note that the special code marked SS2 in the table above has also been assigned (and encoded as 0x1B,0x1B) to allow using another alternate 7-bit shift table. But this mechanism has never been used and the UCS-2 encoding has been preferred.
Note that the character 0x09 (Ç, capital C with cedilla) should instead be replaced by ç (small c with cedilla) in modern implementation, as recommended by Unicode, since the uppercase version is of little use.