Mantis Bugtracker
  

Viewing Issue Simple Details Jump to Notes ] View Advanced ] Issue History ] Print ]
ID Category Severity Reproducibility Date Submitted Last Update
0006524 [Squeak] Multilingual major always 06-05-07 05:07 11-20-08 20:27
Reporter andreas View Status public  
Assigned To
Priority normal Resolution fixed  
Status closed   Product Version 3.10
Summary 0006524: TTFonts can't load unicode character maps
Description TTFonts will currently choose between either the Mac or the Win encoded table of a TTFont, making it impossible to correctly load and display a font with unicode characters. Attached changes fix this.
Additional Information
Attached Files  TTFLoadFixes.cs [^] (4,500 bytes) 06-05-07 05:07
 TTCFontFixes-ONLY-FOR-3.10-POST-7105.cs [^] (3,690 bytes) 06-13-07 15:48
 TTCFontFixes-ONLY-FOR-3.9.1.cs [^] (2,169 bytes) 06-13-07 15:50

- Relationships

SYSTEM WARNING: Creating default object from empty value

related to 0006520closed  recursive fonts cause crash when running tests 
child of 0006570assigned tim A Mother for font and font test problems 

- Notes
(0010767 - 230 - 264 - 264 - 264 - 264 - 264)
andreas
06-05-07 05:08

"Change Set: TTFLoadFixes
Date: 4 June 2007
Author: Andreas Raab

- integrate TTF fixes which would prevent TTFonts to load the correct (unicode) character map and instead use an ascii (windows/mac) character map instead"
 
(0010768 - 184 - 218 - 218 - 218 - 218 - 218)
andreas
06-05-07 05:08

"Change Set: TTCFontFixes
Date: 4 June 2007
Author: Andreas Raab

- integrate TTCFont fixes which would prevent fonts with non-ascii (unicode) code points to render correctly"
 
(0010801 - 550 - 610 - 610 - 610 - 610 - 610)
hirzel
06-12-07 06:44

I filed in the two changesets into the image
Squeak3.10alpha.7105 while using the VM3.10.4.

When testing

(((80 asCharacter asString) , 603 asCharacter asString) asStringMorph font:
((TextStyle named: 'LucidaSansUnicode') fontOfSize: 36)) openInWorld

only the 'P' (character 80) was shown. The LATIN SMALL LETTER OPEN E (epsilon) was shown as a question mark in a different font. So Unicode characters are not shown in this environment. The font used contains this symbol.

In which image(s) have these fixes tested so far with success?
 
(0010802 - 180 - 192 - 192 - 192 - 192 - 192)
hirzel
06-12-07 14:11

Test: Load the changesets into Squeak3.9-final-7067.image using the pre-UTF8 VM.

Result: After dropping a TTF font onto the image soon afterwards a emergency evaluator comes up.
 
(0010806 - 299 - 299 - 299 - 299 - 299 - 299)
andreas
06-13-07 00:13

I see. You guys are running systems w/o the color caching fixes that I posted. The TTCFontFixes-NCC (NCC=No Color Caching) includes a variant that will work in 3.9 and 3.10 (I think - I tested it against an alpha of 3.10 so I can't say for sure if later versions include the color cache or not etc).
 
(0010807 - 656 - 752 - 752 - 752 - 752 - 752)
hirzel
06-13-07 14:29

TESTCASE
Take an unmodified Squeak3.9-final-7067.image

Run it under Windows XP (with the VM which came with the 3.9 release)

FileIn
1) TTCFontFixes-NCC.1.cs
2) TTFLoadFixes.cs
3) TTCFontFixes.cs
in this order

RESULT
After installation (i.e. without actually importing a TTF file) an emergency walkback comes up.

QUESTION
- What am I missing here?
- One of the classes which is changed is TTCFont which is a specialisation of TTFont. What is the intention of this specialisation. For what does the 'C' stand for? The code seems to have still duplicate parts and some methods probably need to be refactored to spot the bugs more easily.
 
(0010808 - 127 - 175 - 175 - 175 - 175 - 175)
hirzel
06-13-07 14:31

TESTCASE

Take an unmodified Squeak Squeak3.10alpha.7105.image

FileIn
TTCFontFixes-NCC.1.cs

RESULT
The image freezes.
 
(0010809 - 473 - 527 - 527 - 527 - 527 - 527)
hirzel
06-13-07 14:36

It is probably a good strategy for fixing to split this issue into two

1) Get right
description _ TTFontDescription addFromTTFile: 'AfontName.ttf'.
Test: The glyhsTable has to have more than 255 entries for a Unicode font

2) Create a style based on the instance of TTFontDescription consisting of TTCFont (or TTFont?) instances

TTCFont newTextStyleFromTT: description
Test: The glyphsTable in the specific font instances still has to have more than 255 entries.
 
(0010810 - 668 - 712 - 712 - 712 - 712 - 712)
andreas
06-13-07 15:55

Dude, what you are missing is that there are images WITH and images WITHOUT the color cache fixes and YOU CANNOT LOAD ONE VERSION INTO THE OTHER (I thought that was completely obvious given that we are talking about the lowest level of text rendering). So loading a NCC version into an image that does have the color cache fixes (like 3.10-7105) will explode; as will loading a version that requires color caching into an image that does not have it (like 3.9).

Most importantly, LOADING BOTH VERSIONS WILL ALWAYS EXPLODE!!!

Since this seems to be so unclear, I have renamed these change sets into one that is calles "ONLY-FOR-3.10-POST-7105" and "ONLY-FOR-3.9".
 
(0010811 - 1172 - 1439 - 1439 - 1439 - 1439 - 1439)
hirzel
06-13-07 21:30

The renaming of the changesets was useful in identifiying what to install.

RETESTING the change sets

RESULT: failure, see below.


TESTCASE

Part A)
Take an unmodified Squeak3.10alpha-7105 image on WindowsXP (Pre UTF8-VM)

File in 'TTFLoadFixes.cs'
File in 'TTCFontFixes-ONLY-FOR-3.10-POST-7105.cs'

Drop file 'L_10646.ttf' (Lucida Sans Unicode) onto the image.

Execute

(((603 asCharacter asString) , 604 asCharacter asString) asStringMorph font:
((TextStyle named: 'LucidaSansUnicode') fontOfSize: 36)) openInWorld

RESULT
NO display of Unicode characters


Part B)
Copy TrueType file 'L_10646.ttf' from %sysroot%\fonts to the directory where the image is.

Inspect the following expression
   TTFontDescription addFromTTFile: 'l_10646.ttf'

Inspect instance variable glyphs : size of the Array is 1776
Inspect instance variable glyphTable : size of the Array is 256.
SHOULD BE MORE THAN THIS.

CONCLUSION
The changeset TTFLoadFixes.cs does not work for this particular font (and some others).

QUESTIONS
Which font did you use for testing?
Which test code did you execute?
What is the version number of Squeak 3.9.1? 7067?
 
(0010814 - 232 - 262 - 262 - 262 - 262 - 262)
andreas
06-15-07 06:32

I tested with Arial using Russian in 3.10-7105 try:

(((1046 asCharacter asString) , 1047 asCharacter asString) asStringMorph font:
((TextStyle named: 'Arial') fontOfSize: 36)) openInWorld

CONCLUSION: Works just as advertised.
 
(0010816 - 934 - 1114 - 1114 - 1114 - 1114 - 1114)
hirzel
06-16-07 16:31

RETEST 3.10-7105 Russian Characters in Arial and 'Lucida Sans Unicode'

Preparation
1) Take an unmodified Squeak3.10alpha.7105.image on Windows XP Home edition

and VM3.10.4
2) File in TTFLoadFixes.cs
3) File in TTCFontFixes-ONLY-FOR-3.10-POST-7105.cs


Test
4) Drop the file 'arial.ttf' onto the image
5) Evaluate
(((1046 asCharacter asString) , 1047 asCharacter asString) asStringMorph font:
((TextStyle named: 'Arial') fontOfSize: 72)) openInWorld


RESULT: Success, Russian Unicode characters are displayed.

6) Drop the file 'L_10646.ttf' onto the image
7) Evaluate
(((1046 asCharacter asString) , 1047 asCharacter asString) asStringMorph font:
((TextStyle named: 'LucidaSansUnicode') fontOfSize: 72)) openInWorld


RESULT: Failure, Russian characters are not displayed though the glyphs are in the

font (Verified with MSWindows CharacterMap Tool)

CONCLUSION
Fix 'TTFLoadFixes.cs' is incomplete.
 
(0010817 - 1200 - 1404 - 1404 - 1404 - 1404 - 1404)
hirzel
06-16-07 16:34

TEST LucidaSansUnicode on 3.10-7081 (without any fix included in this report)

Preparation
1) Take an _unmodified_ (no changes posted here filed in)
    Squeak3.10alpha.7081 image


Test

2) Evaluate

TTCFontReader encodingTag: Latin1Environment leadingChar. TTCFontSet
newTextStyleFromTTFile: 'C:\WINDOWS\Fonts\L_10646.TTF'.
"LucidaSansUnicode"

(((1046 asCharacter asString) , 1047 asCharacter asString) asStringMorph font:
((TextStyle named: 'MultiLucidaSansUnicode') fontOfSize: 72)) openInWorld

RESULT: Success, The Russian Unicode characters are displayed on a unmodified

3.10-7081 image without the fixes in this bug report.

CONCLUSION
Between 9.10-7081 and 9.10-7105 the Unicode font loading mechanism was modified so that loading of Unicode fonts no longer works. The fix 'TTFLoadFixes.cs' reconstructs this PARTIALLY.

REMARK
It seems that MultiTTCFont is used in 7081 and it is no longer is connected to the loading mechanism in 7105. To verify.

FURTHER NEED.
Update of 'TTFLoadFixes.cs' needed, so that the font LucidaSansUnicode (and other fonts) fully work in 3.10-7105. The functionality of 3.10-7105 and later should not be less than in 3.10-7081.
 
(0010886 - 509 - 593 - 593 - 593 - 593 - 593)
edgardec
07-17-07 10:16

I have a question about Registry as I send to list.
I copy here the relevant part

> Because I see the TTCFont class changes , the new def don't have Registry
> and still three methods use Registry.
>
> TTCFont class familyName: n pointSize: s emphasis: code
> TTCFont class register: anObject at: symbolName
> TTCFont class unregister: symbolName

I add Registry for not having Undeclared (and for crash to the image when loading the .mcz).
Wish answer or TTCFont could be phased out of release.
 
(0012788 - 55 - 55 - 55 - 55 - 55 - 55)
KenCausey
11-20-08 20:26

Harvested in update 7120 and released with Squeak 3.10.
 

- Issue History
Date Modified Username Field Change
06-05-07 05:07 andreas New Issue
06-05-07 05:07 andreas File Added: TTFLoadFixes.cs
06-05-07 05:07 andreas File Added: TTCFontFixes.cs
06-05-07 05:08 andreas Note Added: 0010767
06-05-07 05:08 andreas Note Added: 0010768
06-12-07 06:44 hirzel Note Added: 0010801
06-12-07 14:11 hirzel Note Added: 0010802
06-13-07 00:11 andreas File Added: TTCFontFixes-NCC.1.cs
06-13-07 00:13 andreas Note Added: 0010806
06-13-07 14:29 hirzel Note Added: 0010807
06-13-07 14:31 hirzel Note Added: 0010808
06-13-07 14:36 hirzel Note Added: 0010809
06-13-07 15:38 andreas File Deleted: TTCFontFixes.cs
06-13-07 15:48 andreas File Added: TTCFontFixes-ONLY-FOR-3.10-POST-7105.cs
06-13-07 15:48 andreas File Deleted: TTCFontFixes-NCC.1.cs
06-13-07 15:50 andreas File Added: TTCFontFixes-ONLY-FOR-3.9.1.cs
06-13-07 15:55 andreas Note Added: 0010810
06-13-07 21:30 hirzel Note Added: 0010811
06-15-07 06:32 andreas Note Added: 0010814
06-16-07 16:31 hirzel Note Added: 0010816
06-16-07 16:34 hirzel Note Added: 0010817
06-27-07 03:25 pdhpdh Issue Monitored: pdhpdh
06-27-07 09:44 pdhpdh Issue End Monitor: pdhpdh
07-17-07 10:16 edgardec Note Added: 0010886
07-19-07 23:51 wiz Relationship added related to 0006520
09-12-07 00:22 wiz Relationship added child of 0006570
11-20-08 20:26 KenCausey Status new => closed
11-20-08 20:26 KenCausey Note Added: 0012788
11-20-08 20:27 KenCausey Resolution open => fixed
11-20-08 20:27 KenCausey Fixed in Version  => 3.10


Mantis 1.0.8[^]
Copyright © 2000 - 2007 Mantis Group
123 total queries executed.
67 unique queries executed.
Powered by Mantis Bugtracker