0007071: Key values from Input Method are not converted correctly on new Unicode Windows VM

(0012214 - 221 - 221 - 221 - 221 - 221 - 221)
umejava
05-29-08 01:04

Yoshiki informed me that there has already been UTF32InputInterpreter (and its peers) in OLPC eToys image. His patch is more complete. I think his patch should be applied to 3.10 image also. (See: 0482UnicodeInput1-yo.cs)

(0012227 - 148 - 148 - 148 - 148 - 148 - 148)
umejava
05-30-08 05:54

More recent versions from etoys-dev-3.0.image (1713UnicodeInput1-yo.1.cs, 1714UnicodeInput2-yo.1.cs). I confirmed they are loadable in 3.10.1 image.

(0012263 - 661 - 739 - 739 - 739 - 739 - 739)
wiz
06-05-08 23:15

Hi umejava,

Thanks for the bug report and tracking the olpc repairs down.

There are a lot of font stuff in 3.10.n that need updating.
To get things into the image and insure they have some correctness we will need tests.

I am encouraging all who offer fixes to supplies tests that cover their changes. The simple criteria is that the tests should fail before the patch and pass afterwords. The tests serve to explain why the changes are needed and that the changes will do the job.

If you can help with this, it would be appreciated.

Partial progress counts and some tests are better than none.

I note that the changes are going to be large.

(0012507 - 2493 - 4452 - 4652 - 4652 - 4652 - 4652)
nicolas cellier
08-21-08 16:04
edited on: 08-21-08 16:22

The same behavior appears on unix recent VM with unicode support (VM 3.10.1 for example).

Bert proposed OLPC updated patches at http://lists.squeakfoundation.org/pipermail/beginners/2008-July/004656.html [^]

But I noticed a few differences between OLPC versions and 3.10 version for some ParagraphEditor methods in 1713UnicodeInput1-yo.1.cs :

ParagraphEditor>>#readKeyboard

OLPC: self zapSelectionWithCompositionWith: typeAhead contents.
3.10: self zapSelectionWith:
    (Text string: typeAhead contents emphasis: emphasisHere).
THIS ONE IS OK, emphasisHere IS HANDLED CORRECTLY IN #zapSelectionWithCompositionWith:

TextMorphEditor>>#zapSelectionWith:

OLPC: lacks support for multilevel undo
3.10:
      "===Support for multilevel undo start ==="
      rText _ (paragraph text copyFrom: start to: (stop - 1)).
      rInterval _ start to: (stop - 1).
      isInTypeRun _ self isInTypeRun.
      "===Support for multilevel undo end ==="
etc...
      "===Support for multilevel undo start ==="
      (Preferences multipleTextUndo and: [isInTypeRun not])ifTrue:
         [ self addEditCommand:
               (EditCommand
                  textMorph: morph
                  replacedText: rText
                  replacedTextInterval: rInterval
                  newText: aText
                  newTextInterval: super selectionInterval)].
      "===Support for multilevel undo end ==="

1935unicodeFallback-bf.cs also has some uncompatibilities with 3.10 image:

ParagraphEditor>>#dispatchOnCharacter:with:

OLPC: lacks support for Preferences autoIndent on return
   char asciiValue = 13 ifTrue: [
      ^ sensor controlKeyPressed
         ifTrue: [self crWithIndent: typeAheadStream]
         ifFalse: [self normalCharacter: typeAheadStream]].
3.10:
   char asciiValue = 13 ifTrue: [
      ^Preferences autoIndent
         ifTrue: [
            sensor controlKeyPressed
               ifTrue: [self normalCharacter: typeAheadStream]
               ifFalse: [self crWithIndent: typeAheadStream]]
         ifFalse: [
            sensor controlKeyPressed
               ifTrue: [self crWithIndent: typeAheadStream]
               ifFalse: [self normalCharacter: typeAheadStream]]].

Maybe it would be better to have an updated patch for 3.10 images.
Least TODO list before integrating in an official release:
- test if the patch works with an old VM (unix and windows),
- test if the patch works with new VM (unix and windows).

(0012508 - 242 - 292 - 292 - 292 - 292 - 292)
matthewf
08-21-08 18:54

"fix begin"
Installer bug: 7071 fix: '1713UnicodeInput1-yo.1.cs'.
Installer bug: 7071 fix: '1714UnicodeInput2-yo.1.cs'.
Installer bug: 7071 fix: '1925unixLatin1Input-yo.cs'.
Installer bug: 7071 fix: '1935unicodeFallback-bf.cs'.
"fix end"

(0012509 - 360 - 390 - 390 - 390 - 390 - 390)
nicolas cellier
08-21-08 19:03

Hi Matthew, also give a try to UnixVM-UnicodeInput-310ImageCompatible-M7071-nice.1.cs and add it to the list of patches if you want to preserve the two features that are in 3.10 image but do not seem to be in OLPC image:

- Preferences autoIndent
- Preferences multipleTextUndo

Unless these are unecessary? (I did not check what these features are worth)

(0012966 - 270 - 320 - 320 - 320 - 320 - 320)
Keith_Hodges
02-15-09 05:17

"fix begin"
Installer mantis bug: 7071 fix: '1713UnicodeInput1-yo.1.cs'.
Installer mantis bug: 7071 fix: '1714UnicodeInput2-yo.1.cs'.
Installer mantis bug: 7071 fix: '1925unixLatin1Input-yo.cs'.
Installer mantis bug: 7071 fix: '1935unicodeFallback-bf.cs'.
"fix end"

(0013378 - 167 - 185 - 641 - 641 - 641 - 641)
nicolas cellier
11-09-09 18:00

Fixed in
http://source.squeak.org/trunk/Multilingual-nice.59.mcz [^]
http://source.squeak.org/trunk/ST80-nice.63.mcz [^]
http://source.squeak.org/trunk/Morphic-nice.223.mcz [^]

Issue History
Date Modified	Username	Field	Change
05-28-08 09:58	umejava	New Issue
05-28-08 09:58	umejava	File Added: WinUnicodeInputInterpreter-mu.4.cs
05-29-08 00:58	umejava	File Added: 0482UnicodeInput1-yo.cs
05-29-08 01:04	umejava	Note Added: 0012214
05-30-08 05:49	umejava	File Added: 1713UnicodeInput1-yo.1.cs
05-30-08 05:49	umejava	File Added: 1714UnicodeInput2-yo.1.cs
05-30-08 05:54	umejava	Note Added: 0012227
06-05-08 23:01	wiz	Relationship added	child of 0006570
06-05-08 23:15	wiz	Note Added: 0012263
08-21-08 16:04	nicolas cellier	Note Added: 0012507
08-21-08 16:05	nicolas cellier	File Added: 1925unixLatin1Input-yo.cs
08-21-08 16:05	nicolas cellier	File Added: 1935unicodeFallback-bf.cs
08-21-08 16:12	nicolas cellier	Note Edited: 0012507
08-21-08 16:22	nicolas cellier	Note Edited: 0012507
08-21-08 16:39	nicolas cellier	File Added: UnixVM-UnicodeInput-310ImageCompatible-M7071-nice.1.cs
08-21-08 18:52	matthewf	Priority	normal => high
08-21-08 18:54	matthewf	Note Added: 0012508
08-21-08 19:03	nicolas cellier	Note Added: 0012509
01-10-09 02:13	Keith_Hodges	Status	new => pending
01-10-09 02:59	Keith_Hodges	Status	pending => testing
01-10-09 03:40	Keith_Hodges	Status	testing => resolved
01-10-09 03:40	Keith_Hodges	Fixed in Version	=> 3.11
01-10-09 03:40	Keith_Hodges	Resolution	open => fixed
01-10-09 03:40	Keith_Hodges	Assigned To	=> Keith_Hodges
01-10-09 03:41	Keith_Hodges	Status	resolved => testing
02-15-09 05:17	Keith_Hodges	Note Added: 0012966
10-03-09 19:33	Keith_Hodges	Status	testing => assigned
10-03-09 19:33	Keith_Hodges	Assigned To	Keith_Hodges => andreas
11-09-09 18:00	nicolas cellier	Status	assigned => resolved
11-09-09 18:00	nicolas cellier	Fixed in Version	3.11 => trunk
11-09-09 18:00	nicolas cellier	Note Added: 0013378
04-18-10 21:59	andreas	Status	resolved => closed

Notes
(0012214 - 221 - 221 - 221 - 221 - 221 - 221) umejava 05-29-08 01:04	Yoshiki informed me that there has already been UTF32InputInterpreter (and its peers) in OLPC eToys image. His patch is more complete. I think his patch should be applied to 3.10 image also. (See: 0482UnicodeInput1-yo.cs)

(0012227 - 148 - 148 - 148 - 148 - 148 - 148) umejava 05-30-08 05:54	More recent versions from etoys-dev-3.0.image (1713UnicodeInput1-yo.1.cs, 1714UnicodeInput2-yo.1.cs). I confirmed they are loadable in 3.10.1 image.

(0012263 - 661 - 739 - 739 - 739 - 739 - 739) wiz 06-05-08 23:15	Hi umejava, Thanks for the bug report and tracking the olpc repairs down. There are a lot of font stuff in 3.10.n that need updating. To get things into the image and insure they have some correctness we will need tests. I am encouraging all who offer fixes to supplies tests that cover their changes. The simple criteria is that the tests should fail before the patch and pass afterwords. The tests serve to explain why the changes are needed and that the changes will do the job. If you can help with this, it would be appreciated. Partial progress counts and some tests are better than none. I note that the changes are going to be large.

(0012507 - 2493 - 4452 - 4652 - 4652 - 4652 - 4652) nicolas cellier 08-21-08 16:04 edited on: 08-21-08 16:22	The same behavior appears on unix recent VM with unicode support (VM 3.10.1 for example). Bert proposed OLPC updated patches at http://lists.squeakfoundation.org/pipermail/beginners/2008-July/004656.html [^] But I noticed a few differences between OLPC versions and 3.10 version for some ParagraphEditor methods in 1713UnicodeInput1-yo.1.cs : ParagraphEditor>>#readKeyboard OLPC: self zapSelectionWithCompositionWith: typeAhead contents. 3.10: self zapSelectionWith: (Text string: typeAhead contents emphasis: emphasisHere). THIS ONE IS OK, emphasisHere IS HANDLED CORRECTLY IN #zapSelectionWithCompositionWith: TextMorphEditor>>#zapSelectionWith: OLPC: lacks support for multilevel undo 3.10: "===Support for multilevel undo start ===" rText _ (paragraph text copyFrom: start to: (stop - 1)). rInterval _ start to: (stop - 1). isInTypeRun _ self isInTypeRun. "===Support for multilevel undo end ===" etc... "===Support for multilevel undo start ===" (Preferences multipleTextUndo and: [isInTypeRun not])ifTrue: [ self addEditCommand: (EditCommand textMorph: morph replacedText: rText replacedTextInterval: rInterval newText: aText newTextInterval: super selectionInterval)]. "===Support for multilevel undo end ===" 1935unicodeFallback-bf.cs also has some uncompatibilities with 3.10 image: ParagraphEditor>>#dispatchOnCharacter:with: OLPC: lacks support for Preferences autoIndent on return char asciiValue = 13 ifTrue: [ ^ sensor controlKeyPressed ifTrue: [self crWithIndent: typeAheadStream] ifFalse: [self normalCharacter: typeAheadStream]]. 3.10: char asciiValue = 13 ifTrue: [ ^Preferences autoIndent ifTrue: [ sensor controlKeyPressed ifTrue: [self normalCharacter: typeAheadStream] ifFalse: [self crWithIndent: typeAheadStream]] ifFalse: [ sensor controlKeyPressed ifTrue: [self crWithIndent: typeAheadStream] ifFalse: [self normalCharacter: typeAheadStream]]]. Maybe it would be better to have an updated patch for 3.10 images. Least TODO list before integrating in an official release: - test if the patch works with an old VM (unix and windows), - test if the patch works with new VM (unix and windows).

(0012508 - 242 - 292 - 292 - 292 - 292 - 292) matthewf 08-21-08 18:54	"fix begin" Installer bug: 7071 fix: '1713UnicodeInput1-yo.1.cs'. Installer bug: 7071 fix: '1714UnicodeInput2-yo.1.cs'. Installer bug: 7071 fix: '1925unixLatin1Input-yo.cs'. Installer bug: 7071 fix: '1935unicodeFallback-bf.cs'. "fix end"

(0012509 - 360 - 390 - 390 - 390 - 390 - 390) nicolas cellier 08-21-08 19:03	Hi Matthew, also give a try to UnixVM-UnicodeInput-310ImageCompatible-M7071-nice.1.cs and add it to the list of patches if you want to preserve the two features that are in 3.10 image but do not seem to be in OLPC image: - Preferences autoIndent - Preferences multipleTextUndo Unless these are unecessary? (I did not check what these features are worth)

(0012966 - 270 - 320 - 320 - 320 - 320 - 320) Keith_Hodges 02-15-09 05:17	"fix begin" Installer mantis bug: 7071 fix: '1713UnicodeInput1-yo.1.cs'. Installer mantis bug: 7071 fix: '1714UnicodeInput2-yo.1.cs'. Installer mantis bug: 7071 fix: '1925unixLatin1Input-yo.cs'. Installer mantis bug: 7071 fix: '1935unicodeFallback-bf.cs'. "fix end"

(0013378 - 167 - 185 - 641 - 641 - 641 - 641) nicolas cellier 11-09-09 18:00	Fixed in http://source.squeak.org/trunk/Multilingual-nice.59.mcz [^] http://source.squeak.org/trunk/ST80-nice.63.mcz [^] http://source.squeak.org/trunk/Morphic-nice.223.mcz [^]