Mantis Bugtracker
  

Viewing Issue Simple Details Jump to Notes ] View Advanced ] Issue History ] Print ]
ID Category Severity Reproducibility Date Submitted Last Update
0007476 [Squeak] VM major sometimes 03-10-10 19:33 04-18-10 21:57
Reporter eem View Status public  
Assigned To lewis
Priority normal Resolution fixed  
Status closed   Product Version trunk
Summary 0007476: clipboardSize on Win32 can return -1 (4294967295) on failure
Description On Fri, 19 Feb 2010 16:25:20 -0500, Jon Hylands <jon@huv.com> wrote:

>I'm getting a weird problem, and its been so many years since I saw
>this, I'm at a loss as to what to do...
>
>On my robot Brainbot, I have a 2.83 GHz Core 2 Quad with 2 GB RAM and
>a 32 GB solid state HD. It is running Windows XP Pro.
>
>I'm getting a Low Space Notifier whenever I try and copy text to the
>clipboard, or when I try and paste. I can do other development things,
>writing a saving code, and opening browsers and windows with no
>problems.
>
>Task Manager says Squeak is taking up 37,980 K, and the machine has
>1.2 GB of free physical memory.
>
>I'm running the following VM:
>
>Squeak 3.10.6 (release) from Aug 30 2007
>
>And the following image:
>
>Squeak 3.10.2
>latest update: 0007179

Okay, so this is still bothering me, and I dug a little deeper, and
came across something that blew me away...

http://www.huv.com/Clipboard-OutOfMemory.png [^]

Basically, when I do 'copy', the primitive returns a ByteString that
is 4 billion bytes long -- 2^32 - 1 to be exact. Clearly that is not
really allocated, because even when I have this inspector open, Squeak
is still only taking about 37 MB of memory.

However, the method String >> #withSqueakLineEndings is trying to
create a new string that is 4 billion bytes long, and that is where I
run into the low space warning.

Primitive 141 appears to be causing the issue.

I'm not much of a VM person, so hopefully someone who is can look at
that primitive and figure out what case is causing it.

One more thing to note - I'm running Windows in a Remote Desktop, from
a Linux machine. To clarify, my development machine is a desktop,
running Ubuntu linux. The robot is a mini-itx running Windows XP Pro.
I use remote desktop from linux to access the Windows GUI on the
robot.

Later,
Jon
============
The primitive looks like this in my quite possibly obsolete VM sources:

primitiveClipboardText
    "When called with a single string argument, post the string to
    the clipboard. When called with zero arguments, return a
    string containing the current clipboard contents."
    | s sz |
    argumentCount = 1
        ifTrue: [s := self stackTop.
            (self isBytes: s) ifFalse: [^ self primitiveFail].
            self successful
                ifTrue: [sz := self stSizeOf: s.
                    self clipboardWrite: sz From: s + BaseHeaderSize At: 0.
                    self pop: 1]]
        ifFalse: [sz := self clipboardSize.
            (self sufficientSpaceToAllocate: sz) ifFalse:[^self primitiveFail].
            s := self instantiateClass: (self splObj: ClassString) indexableSize: sz.
            self clipboardRead: sz Into: s + BaseHeaderSize At: 0.
            self pop: 1 thenPush: s]

I'm willing to bet that clipboardSize is returning -1.

It probably needs to read something like

primitiveClipboardText
    "When called with a single string argument, post the string to
    the clipboard. When called with zero arguments, return a
    string containing the current clipboard contents."
    | s sz |
    argumentCount = 1
        ifTrue: [s := self stackTop.
            (self isBytes: s) ifFalse: [^ self primitiveFail].
            self successful
                ifTrue: [sz := self stSizeOf: s.
                    self clipboardWrite: sz From: s + BaseHeaderSize At: 0.
                    self pop: 1]]
        ifFalse: [sz := self clipboardSize.
>> (sz >= 0 and: [self sufficientSpaceToAllocate: sz]) ifFalse:[^self primitiveFail].
            s := self instantiateClass: (self splObj: ClassString) indexableSize: sz.
            self clipboardRead: sz Into: s + BaseHeaderSize At: 0.
            self pop: 1 thenPush: s]

I'm not much of a VM person, so hopefully someone who is can look at
that primitive and figure out what case is causing it.

One more thing to note - I'm running Windows in a Remote Desktop, from
a Linux machine. To clarify, my development machine is a desktop,
running Ubuntu linux. The robot is a mini-itx running Windows XP Pro.
I use remote desktop from linux to access the Windows GUI on the
robot.

The Windows VM's clipboardText function is a tad unsafe:

int clipboardSize(void) {
  HANDLE h;
  WCHAR *src;
  unsigned char *tmp;
  int i, count, bytesNeeded;

  /* Do we have text in the clipboard? */
  if(!IsClipboardFormatAvailable(CF_UNICODETEXT))
    return 0;

  if(!OpenClipboard(stWindow))
    return 0;

  /* Get it in unicode format. */
  h = GetClipboardData(CF_UNICODETEXT);
  src = GlobalLock(h);

  /* How many bytes do we need to store those unicode chars in UTF8 format? */
  bytesNeeded = WideCharToMultiByte(CP_UTF8, 0, src, -1,
                    NULL, 0, NULL, NULL );
  tmp = malloc(bytesNeeded+1);

  /* Convert Unicode text to UTF8. */
  WideCharToMultiByte(CP_UTF8, 0, src, -1, tmp, bytesNeeded , NULL, NULL);

  /* Count CrLfs for which we remove the extra Lf */
  count = bytesNeeded; /* ex. terminating zero */
  for(i=0; i<count; i++) {
    if((tmp[i] == 13) && (tmp[i+1] == 10)) bytesNeeded--;
  }
  bytesNeeded--; /* discount terminating zero */
  free(tmp); /* no longer needed */

  GlobalUnlock(h);
  CloseClipboard();

  return bytesNeeded;
}

If bytesNeeded is zero then the function will return (int)-1.

From the MS manual page:

Return Value

Returns the number of bytes written to the buffer pointed to by lpMultiByteStr if successful. If the function succeeds and cbMultiByte is 0, the return value is the required size, in bytes, for the buffer indicated by lpMultiByteStr.

The function returns 0 if it does not succeed. To get extended error information, the application can call GetLastError, which can return one of the following error codes:

ERROR_INSUFFICIENT_BUFFER. A supplied buffer size was not large enough, or it was incorrectly set to NULL.
ERROR_INVALID_FLAGS. The values supplied for flags were not valid.
ERROR_INVALID_PARAMETER. Any of the parameter values was invalid.
ERROR_NO_UNICODE_TRANSLATION. Invalid Unicode was found in a string.

So something like

  /* How many bytes do we need to store those unicode chars in UTF8 format? */
     bytesNeeded = WideCharToMultiByte(CP_UTF8, 0, src, -1,
                    NULL, 0, NULL, NULL );
     if (bytesNeeded <= 0)
          return 0;

would be wise.

Eliot
Additional Information A better fix is the following which remembers to unlock and close the clipboard.

int clipboardSize(void) {
  HANDLE h;
  WCHAR *src;
  int i, count, bytesNeeded;

  /* Do we have text in the clipboard? */
  if(!IsClipboardFormatAvailable(CF_UNICODETEXT))
    return 0;

  if(!OpenClipboard(stWindow))
    return 0;

  /* Get it in unicode format. */
  h = GetClipboardData(CF_UNICODETEXT);
  src = GlobalLock(h);

  /* How many bytes do we need to store those unicode chars in UTF8 format? */
  bytesNeeded = WideCharToMultiByte(CP_UTF8, 0, src, -1,
                    NULL, 0, NULL, NULL );
  if (bytesNeeded > 0) {
    unsigned char *tmp = malloc(bytesNeeded+1);

    /* Convert Unicode text to UTF8. */
    WideCharToMultiByte(CP_UTF8, 0, src, -1, tmp, bytesNeeded , NULL, NULL);

    /* Count CrLfs for which we remove the extra Lf */
    count = bytesNeeded; /* ex. terminating zero */
    for(i=0; i<count; i++) {
      if((tmp[i] == 13) && (tmp[i+1] == 10)) bytesNeeded--;
    }
    bytesNeeded--; /* discount terminating zero */
    free(tmp); /* no longer needed */
  }

  GlobalUnlock(h);
  CloseClipboard();

  return bytesNeeded;
}
Attached Files  sqWin32Window.c [^] (94,543 bytes) 03-10-10 19:33

- Relationships

- Notes
(0013546 - 13 - 13 - 13 - 13 - 13 - 13)
andreas
03-10-10 22:02

Patch in SVN.
 

- Issue History
Date Modified Username Field Change
03-10-10 19:33 eem New Issue
03-10-10 19:33 eem Status new => assigned
03-10-10 19:33 eem Assigned To  => lewis
03-10-10 19:33 eem File Added: sqWin32Window.c
03-10-10 22:02 andreas Status assigned => resolved
03-10-10 22:02 andreas Resolution open => fixed
03-10-10 22:02 andreas Note Added: 0013546
04-18-10 21:57 andreas Status resolved => closed


Mantis 1.0.8[^]
Copyright © 2000 - 2007 Mantis Group
41 total queries executed.
32 unique queries executed.
Powered by Mantis Bugtracker