SYSTEM WARNING: Creating default object from empty value

Mantis - Squeak
Viewing Issue Advanced Details
6524 Multilingual major always 06-05-07 05:07 11-20-08 20:27
closed 3.10  
none 3.10  
0006524: TTFonts can't load unicode character maps
TTFonts will currently choose between either the Mac or the Win encoded table of a TTFont, making it impossible to correctly load and display a font with unicode characters. Attached changes fix this.
related to 0006520closed  recursive fonts cause crash when running tests 
child of 0006570assigned tim A Mother for font and font test problems 
 TTFLoadFixes.cs [^] (4,500 bytes) 06-05-07 05:07
 TTCFontFixes-ONLY-FOR-3.10-POST-7105.cs [^] (3,690 bytes) 06-13-07 15:48
 TTCFontFixes-ONLY-FOR-3.9.1.cs [^] (2,169 bytes) 06-13-07 15:50

06-05-07 05:08   
"Change Set: TTFLoadFixes
Date: 4 June 2007
Author: Andreas Raab

- integrate TTF fixes which would prevent TTFonts to load the correct (unicode) character map and instead use an ascii (windows/mac) character map instead"
06-05-07 05:08   
"Change Set: TTCFontFixes
Date: 4 June 2007
Author: Andreas Raab

- integrate TTCFont fixes which would prevent fonts with non-ascii (unicode) code points to render correctly"
06-12-07 06:44   
I filed in the two changesets into the image
Squeak3.10alpha.7105 while using the VM3.10.4.

When testing

(((80 asCharacter asString) , 603 asCharacter asString) asStringMorph font:
((TextStyle named: 'LucidaSansUnicode') fontOfSize: 36)) openInWorld

only the 'P' (character 80) was shown. The LATIN SMALL LETTER OPEN E (epsilon) was shown as a question mark in a different font. So Unicode characters are not shown in this environment. The font used contains this symbol.

In which image(s) have these fixes tested so far with success?
06-12-07 14:11   
Test: Load the changesets into Squeak3.9-final-7067.image using the pre-UTF8 VM.

Result: After dropping a TTF font onto the image soon afterwards a emergency evaluator comes up.
06-13-07 00:13   
I see. You guys are running systems w/o the color caching fixes that I posted. The TTCFontFixes-NCC (NCC=No Color Caching) includes a variant that will work in 3.9 and 3.10 (I think - I tested it against an alpha of 3.10 so I can't say for sure if later versions include the color cache or not etc).
06-13-07 14:29   
Take an unmodified Squeak3.9-final-7067.image

Run it under Windows XP (with the VM which came with the 3.9 release)

1) TTCFontFixes-NCC.1.cs
2) TTFLoadFixes.cs
3) TTCFontFixes.cs
in this order

After installation (i.e. without actually importing a TTF file) an emergency walkback comes up.

- What am I missing here?
- One of the classes which is changed is TTCFont which is a specialisation of TTFont. What is the intention of this specialisation. For what does the 'C' stand for? The code seems to have still duplicate parts and some methods probably need to be refactored to spot the bugs more easily.
06-13-07 14:31   

Take an unmodified Squeak Squeak3.10alpha.7105.image


The image freezes.
06-13-07 14:36   
It is probably a good strategy for fixing to split this issue into two

1) Get right
description _ TTFontDescription addFromTTFile: 'AfontName.ttf'.
Test: The glyhsTable has to have more than 255 entries for a Unicode font

2) Create a style based on the instance of TTFontDescription consisting of TTCFont (or TTFont?) instances

TTCFont newTextStyleFromTT: description
Test: The glyphsTable in the specific font instances still has to have more than 255 entries.
06-13-07 15:55   
Dude, what you are missing is that there are images WITH and images WITHOUT the color cache fixes and YOU CANNOT LOAD ONE VERSION INTO THE OTHER (I thought that was completely obvious given that we are talking about the lowest level of text rendering). So loading a NCC version into an image that does have the color cache fixes (like 3.10-7105) will explode; as will loading a version that requires color caching into an image that does not have it (like 3.9).


Since this seems to be so unclear, I have renamed these change sets into one that is calles "ONLY-FOR-3.10-POST-7105" and "ONLY-FOR-3.9".
06-13-07 21:30   
The renaming of the changesets was useful in identifiying what to install.

RETESTING the change sets

RESULT: failure, see below.


Part A)
Take an unmodified Squeak3.10alpha-7105 image on WindowsXP (Pre UTF8-VM)

File in 'TTFLoadFixes.cs'
File in 'TTCFontFixes-ONLY-FOR-3.10-POST-7105.cs'

Drop file 'L_10646.ttf' (Lucida Sans Unicode) onto the image.


(((603 asCharacter asString) , 604 asCharacter asString) asStringMorph font:
((TextStyle named: 'LucidaSansUnicode') fontOfSize: 36)) openInWorld

NO display of Unicode characters

Part B)
Copy TrueType file 'L_10646.ttf' from %sysroot%\fonts to the directory where the image is.

Inspect the following expression
   TTFontDescription addFromTTFile: 'l_10646.ttf'

Inspect instance variable glyphs : size of the Array is 1776
Inspect instance variable glyphTable : size of the Array is 256.

The changeset TTFLoadFixes.cs does not work for this particular font (and some others).

Which font did you use for testing?
Which test code did you execute?
What is the version number of Squeak 3.9.1? 7067?
06-15-07 06:32   
I tested with Arial using Russian in 3.10-7105 try:

(((1046 asCharacter asString) , 1047 asCharacter asString) asStringMorph font:
((TextStyle named: 'Arial') fontOfSize: 36)) openInWorld

CONCLUSION: Works just as advertised.
06-16-07 16:31   
RETEST 3.10-7105 Russian Characters in Arial and 'Lucida Sans Unicode'

1) Take an unmodified Squeak3.10alpha.7105.image on Windows XP Home edition

and VM3.10.4
2) File in TTFLoadFixes.cs
3) File in TTCFontFixes-ONLY-FOR-3.10-POST-7105.cs

4) Drop the file 'arial.ttf' onto the image
5) Evaluate
(((1046 asCharacter asString) , 1047 asCharacter asString) asStringMorph font:
((TextStyle named: 'Arial') fontOfSize: 72)) openInWorld

RESULT: Success, Russian Unicode characters are displayed.

6) Drop the file 'L_10646.ttf' onto the image
7) Evaluate
(((1046 asCharacter asString) , 1047 asCharacter asString) asStringMorph font:
((TextStyle named: 'LucidaSansUnicode') fontOfSize: 72)) openInWorld

RESULT: Failure, Russian characters are not displayed though the glyphs are in the

font (Verified with MSWindows CharacterMap Tool)

Fix 'TTFLoadFixes.cs' is incomplete.
06-16-07 16:34   
TEST LucidaSansUnicode on 3.10-7081 (without any fix included in this report)

1) Take an _unmodified_ (no changes posted here filed in)
    Squeak3.10alpha.7081 image


2) Evaluate

TTCFontReader encodingTag: Latin1Environment leadingChar. TTCFontSet
newTextStyleFromTTFile: 'C:\WINDOWS\Fonts\L_10646.TTF'.

(((1046 asCharacter asString) , 1047 asCharacter asString) asStringMorph font:
((TextStyle named: 'MultiLucidaSansUnicode') fontOfSize: 72)) openInWorld

RESULT: Success, The Russian Unicode characters are displayed on a unmodified

3.10-7081 image without the fixes in this bug report.

Between 9.10-7081 and 9.10-7105 the Unicode font loading mechanism was modified so that loading of Unicode fonts no longer works. The fix 'TTFLoadFixes.cs' reconstructs this PARTIALLY.

It seems that MultiTTCFont is used in 7081 and it is no longer is connected to the loading mechanism in 7105. To verify.

Update of 'TTFLoadFixes.cs' needed, so that the font LucidaSansUnicode (and other fonts) fully work in 3.10-7105. The functionality of 3.10-7105 and later should not be less than in 3.10-7081.
07-17-07 10:16   
I have a question about Registry as I send to list.
I copy here the relevant part

> Because I see the TTCFont class changes , the new def don't have Registry
> and still three methods use Registry.
> TTCFont class familyName: n pointSize: s emphasis: code
> TTCFont class register: anObject at: symbolName
> TTCFont class unregister: symbolName

I add Registry for not having Undeclared (and for crash to the image when loading the .mcz).
Wish answer or TTCFont could be phased out of release.
11-20-08 20:26   
Harvested in update 7120 and released with Squeak 3.10.