Mantis - Squeak
Viewing Issue Advanced Details
1660 Collections minor always 08-12-05 16:42 09-09-08 14:45
KenCausey  
 
normal  
closed  
fixed  
none    
none 3.10  
0001660: [ENH] SpeedTalkingStreams
schwa@cc.gatech.edu:

"from preamble:

"Change Set: FastTalkingStreams
Date: 24 August 2003
Author: Joshua Gargus

This changeset greatly speeds up writing word data onto a
RWBinaryOrTextStream. It also provides default implementations for
#byteSize and #bytesPerElement for variable-sized objects.

I noticed the performance problem when trying to write B3DRotationArrays
onto a ReferenceStream on top of a RWBinaryOrTextStream.

Thanks to Boris Gaertner and Yoshiki Ohshima, who provided the ideas for
this changeset.

Sorry about the name, I couldn't come up with a one as witty as
ZippierStreams.

Test with the following code. The first part is the part that
experiences the speedup, and the second verifies correctness. Note: you
should fileIn the method #bytesPerElement first, because FloatArray
won't understand it otherwise. I got a 30x speed improvement on this
benchmark.

floats _ (1 to: 100000) collect: [:ignore | (101 atRandom / 61 atRandom)
asFloat].
floats _ FloatArray withAll: floats.
binStream := RWBinaryOrTextStream on: ''.
binStream binary.
refStream := ReferenceStream on: binStream.
[refStream nextPut: floats] timeToRun.
refStream close.

contents := binStream contents.
binStream := RWBinaryOrTextStream with: contents.
binStream position: contents size. binStream reset.
refStream := ReferenceStream on: binStream.
[refStream next = floats] assert.
refStream close.
"!"
 SpeedTalkingStreams.cs.gz [^] (1,619 bytes) 08-12-05 16:42

Notes
(0002278)
KenCausey   
08-12-05 16:47   
"Boris Gaertner" <Boris.Gaertner@gmx.net:

">> from preamble:
> I got a 30x speed improvement on this
> benchmark.
This is impressive!

My change set proposal1.cs was not impressive at all,
not only gave it wrong results, it also slowed down a
slow algorithm. Sorry about that!

Yoshiki Ohshima found a really admireable solution."
(0002279)
KenCausey   
08-12-05 16:50   
"Joshua 'Schwa' Gargus" <schwa@cc.gatech.edu>:

"Then reply with some harvesting tags, and let's get this in the image!
http://minnow.cc.gatech.edu/squeak/3103 [^]

(although what we really need is for it to be tested on both big-endian
and little-endian machines. I only have x86)

> My change set proposal1.cs was not impressive at all,
> not only gave it wrong results, it also slowed down a
> slow algorithm. Sorry about that!

Ah, but I found that proposal2.cs (included in my changeset) made
a significant improvement. Thanks!

>
> Yoshiki Ohshima found a really admireable solution.

Yes, he did. Thanks Yoshiki!"
(0002281)
KenCausey   
08-12-05 17:12   
Yoshiki Ohshima <Yoshiki.Ohshima@acm.org>:

"You're always welcome, and don't give me too much credit. The write
of hackBits: mechanism is the guy."
(0002282)
KenCausey   
08-12-05 17:16   
"Joshua 'Schwa' Gargus" <schwa@cc.gatech.edu>:

"Ted Kaehler is the only set of initials I see on the code, but that's
from 2000, so others might have had something to do with it before
Squeak 3.0 (I'm sure it was around before that)."
(0002283)
KenCausey   
08-12-05 17:20   
"Boris Gaertner" <Boris.Gaertner@gmx.net>:

"This is impressive!

I did the test on a 80486 with Squeak 3.6 beta.
As Joshua pointed out, it would be nice to have a test on
a big-endian machine."
(0002284)
KenCausey   
08-12-05 17:23   
Karl Ramberg <karl.ramberg@chello.se>:

"Got these numbers from the [refStream nextPut: floats] timeToRun.
9362 before filing in the change set.
3038 after.
Mac os 8.6. PPC604."
(0002285)
KenCausey   
08-12-05 17:26   
"Joshua 'Schwa' Gargus" <schwa@cc.gatech.edu>:

"Really? And the float array was the same size? You must have created
a new stream before writing the floats again, otherwise it would have
just appended a reference instead of writing the words, and would have
been much faster. (scratches head)

The 9 seconds timing is pretty much in line with mine; it takes 1.2
seconds on my 2GHz P4. Despite this, I stepped through part of an
invocation of the old code, and didn't see anything that would be
sped up greatly by being on a big-endian machine. Therefore, my guess
is that it is the new code not giving as much of a speed-up, rather
than the old code being faster. But I can't imagine why this would
be. Could you send results of a MessageTally?

Thanks for testing,"
(0002286)
KenCausey   
08-12-05 17:29   
Karl Ramberg <karl.ramberg@chello.se>:

"It's most likely some mistake I made...
How should I run this benchmark ?
Select each line and doIt/ printIt ?
Run it and keep the Workspace temp 'floats' for use again after filing
in ?

floats _ (1 to: 100000) collect: [:ignore | (101 atRandom / 61 atRandom)
asFloat].
floats _ FloatArray withAll: floats.
binStream := RWBinaryOrTextStream on: ''.
binStream binary.
refStream := ReferenceStream on: binStream.
[refStream nextPut: floats] timeToRun.
refStream close.

contents := binStream contents.
binStream := RWBinaryOrTextStream with: contents.
binStream position: contents size. binStream reset.
refStream := ReferenceStream on: binStream.
[refStream next = floats] assert.
refStream close."
(0002287)
KenCausey   
08-12-05 17:32   
"Joshua 'Schwa' Gargus" <schwa@cc.gatech.edu>:

"> It's most likely some mistake I made...
> How should I run this benchmark ?
> Select each line and doIt/ printIt ?

That's how I did it.

> Run it and keep the Workspace temp 'floats' for use again after filing
> in ?
>

I did, but it shouldn't matter. Just to be clear:

"do this once"
floats := (1 to: 100000) collect: [:ignore | (101 atRandom / 61 atRandom) asFloat].
floats := FloatArray withAll: floats.

"select all of this and printIt, before and after filing in"
binStream := RWBinaryOrTextStream on: ''.
binStream binary.
refStream := ReferenceStream on: binStream.
time := [refStream nextPut: floats] timeToRun.
refStream close.
time

Thanks"
(0002288)
KenCausey   
08-12-05 17:36   
Karl Ramberg <karl.ramberg@chello.se>:

"Before: 11889
After: 102 !!!!!

Thats pretty :-)"
(0002289)
KenCausey   
08-12-05 17:38   
"Joshua 'Schwa' Gargus" <schwa@cc.gatech.edu>:

"That's more like it!

Thanks Karl"
(0002290)
KenCausey   
08-12-05 17:43   
denker@iam.unibe.ch:

"Checking SpeedTalkingStreams.cs (#5395) for method conflicts with
changesets after 5395ImportsAccessorFix-yo ...
...checking WriteStream>>nextPutAll:
CONFLICT found for WriteStream class>>nextPutAll: with newer changeset
5477CollectedTextFixes-BG
...checking WriteStream>>next:putAll:startingAt:
CONFLICT found for WriteStream class>>next:putAll:startingAt: with newer
changesets 5477CollectedTextFixes-BG 5996-006-incChanges
...checking WriteStream>>growTo:
CONFLICT found for WriteStream class>>growTo: with newer changeset
5477CollectedTextFixes-BG
3 conflicts found."
(0002291)
KenCausey   
08-12-05 17:46   
I loaded this into 3.8-6665-basic without errors but did not test further.
(0004665)
MarcusDenker   
04-02-06 11:17   
changeset needs to be adapted to 3.9a.
(0010539)
edgardec   
04-13-07 21:01   
This now is 7084CollectionEnh.cs and was in updates for 3.10
Thanks all !
(0012602)
KenCausey   
09-09-08 14:45   
Harvested in update 7084 and released with 3.10.