Author Message

i was having a discussion w/some friends and ....

we're curious about how fast (obviously chip speed will make a difference, but
we're not trying to be too scientific, just trying to get a general notion) the
current flavors of apl (as opposed to my old stsc dos versions) can calculate
the final distribution, {2,3,...,11,12}, of say 3,000,000 rolls of a pair of
dice.

if a few people have a few minutes to write an idiom, time it, and post the
machine used and the results of the test it would be appreciated.

if 3,000,000 blows up your machine because of outerproduct type memory
limitations, then feel free to use a lower number of iterations, but obviously
don't forgot to note this.

Fri, 25 Jun 2004 06:55:46 GMT
Using J on my P3-500 laptop:

6!:2 'd=: <: #/.~ (i.11), (?3e6\$6) + ?3e6\$6'
5.67563

That is, it takes 5.67563 seconds.

d
83269 166134 250296 333175 416709 499021 417548 333717 249358 167224 83549

These are the counts of getting sums of 2, 3, ..., 12.
(Actually sums of 0, 1, to 10, because the dice faces
are numbered from 0 to 5, but it's equivalent.)

+/d   NB. a sanity check
3000000

The phrase is explained as follows:

?3e6\$6 generates 3 million throws of a die, done twice for
the two dice, and then the face spots are added together.
(As I said, the faces are numbered from 0 to 5.)  The key
conjunction (dyadic operator) x u/.y works as follows:
items of x specify keys for corresponding items of y and
the verb u is applied to each collection of y having identical
keys.  In this case, we use the sums as their own keys,
u~ y is y u y.  The verb # gives the cardinality of each set
of sums.   Finally, we preface the sums by the ordered universe
i.11, so that the final counts are in that order, and decrement
(<:) each count by one to account for the preface.

Quote:
----- Original Message -----

Sent: Sunday, January 06, 2002 16:39 PM

> i was having a discussion w/some friends and ....

> we're curious about how fast (obviously chip speed will make a difference,
but
> we're not trying to be too scientific, just trying to get a general
notion) the
> current flavors of apl (as opposed to my old stsc dos versions) can
calculate
> the final distribution, {2,3,...,11,12}, of say 3,000,000 rolls of a pair
of
> dice.

> if a few people have a few minutes to write an idiom, time it, and post
the
> machine used and the results of the test it would be appreciated.

> if 3,000,000 blows up your machine because of outerproduct type memory
> limitations, then feel free to use a lower number of iterations, but
obviously
> don't forgot to note this.

Fri, 25 Jun 2004 10:07:33 GMT
On my 1.7 ghz machine, APL2 does this:

1001 72534 630276 557742
A is 3000000 2 reshape ?6000000 reshape 6
1001 84762 642504 557742
+/(1+ iota 11)jot .=+/A
82843 166000 250382 333361 417755 500099 415952 332574 250055 167453 83526
1001 86945 644687 557742

12228 milliseconds are spent generating the random data.
2183 milliseconds are spent doing the sums

David Liebtag
APL Products and Services

Fri, 25 Jun 2004 12:27:39 GMT

Quote:

> we're curious about how fast (obviously chip speed will make a difference, but
> we're not trying to be too scientific, just trying to get a general notion) the
> current flavors of apl (as opposed to my old stsc dos versions) can calculate
> the final distribution, {2,3,...,11,12}, of say 3,000,000 rolls of a pair of
> dice.

On my Sony Vaio 800Mhz with 512M Ram
J:  4.64087 sec
K:  1.319 sec
The J code has already been published here.
The K code I used was
t:{s:&11; s[+/(2,x) _draw 6]+:1; :s}
\t t[3000000]

Fri, 25 Jun 2004 13:20:53 GMT

Quote:

> we're curious about how fast (obviously chip speed will make a difference, but
> we're not trying to be too scientific, just trying to get a general notion) the
> current flavors of apl (as opposed to my old stsc dos versions) can calculate
> the final distribution, {2,3,...,11,12}, of say 3,000,000 rolls of a pair of
> dice.

8.7 sec on a 450Mhz P-II running Dyalog APL/W  9.0 -- 5.7 sec to generate
the rolls, 3.0 sec to count them.

Fri, 25 Jun 2004 13:52:12 GMT

Quote:
> i was having a discussion w/some friends and ....

> we're curious about how fast (obviously chip speed will make a difference,
but
> we're not trying to be too scientific, just trying to get a general
notion) the
> current flavors of apl (as opposed to my old stsc dos versions) can
calculate
> the final distribution, {2,3,...,11,12}, of say 3,000,000 rolls of a pair
of
> dice.

4400 ms on an AMD Athlon 650 MHz and Dyalog 9:

+/{each}({enclose}(?3000000{rho}6)+(?3000000{rho}6))={each}{iota}12

+/0 83250 166488 250553 332718 417663 499870 417208 332230 250354 166704
82962
3000000

/ Tomas

Fri, 25 Jun 2004 18:39:25 GMT
thanks all.

it had been a few years since i did a series of similar experiments (things like
roll dice, run N ols regressions, raw looping,  sorts, etc etc - simple stuff -
not by any means comprehensive) between apl, apl2, & k (i hadn't tested j) and
was curious if/how things have changed - relatively.

for the record, my old apl's struggle mightily with a 6000000 element vector and
certainly with the inner product..

on a dell 333mhz pII w/256k, i get 1.291 seconds(+/- .002) using K with 2
different, but fundamentally similar solutions (thanks to mkr, sa & dbb for
understanding why they're fundamentally similar)....

foo2:{t:&11; t[+/(2,x) _draw 6]+:1; :t}

i am curious about the results on some other tests, but i'll spare the group -:)

Quote:
> i was having a discussion w/some friends and ....

> we're curious about how fast (obviously chip speed will make a difference, but
> we're not trying to be too scientific, just trying to get a general notion)
the
> current flavors of apl (as opposed to my old stsc dos versions) can calculate
> the final distribution, {2,3,...,11,12}, of say 3,000,000 rolls of a pair of
> dice.

> if a few people have a few minutes to write an idiom, time it, and post the
> machine used and the results of the test it would be appreciated.

> if 3,000,000 blows up your machine because of outerproduct type memory
> limitations, then feel free to use a lower number of iterations, but obviously
> don't forgot to note this.

Fri, 25 Jun 2004 19:59:30 GMT

Quote:
> i was having a discussion w/some friends and ....

> we're curious about how fast (obviously chip speed will make a difference,
but
> we're not trying to be too scientific, just trying to get a general
notion) the
> current flavors of apl (as opposed to my old stsc dos versions) can
calculate
> the final distribution, {2,3,...,11,12}, of say 3,000,000 rolls of a pair
of
> dice.

2250 ms on an AMD K7 Athlon 1000 MHz and APL+WIN 3.6:

+/{each}({enclose}(?3000000{rho}6)+(?3000000{rho}6))={each}{iota}12
+/0 83597 166211 250030 333921 416488 500167 416200 333081 250625 166547
83133
3000000

Fred

Fri, 25 Jun 2004 21:17:39 GMT

Quote:
> thanks all.
> for the record, my old apl's struggle mightily with a 6000000 element vector
and
> certainly with the inner product..

errh, outer product....

Fri, 25 Jun 2004 21:47:36 GMT
oops, my apologies for the large jpg-file; I did not notice the size of it

Bernard Houben

Mon, 28 Jun 2004 02:49:28 GMT

Quote:

> Most recent computers are apl-proof. For several years
> I use the attached standard-procedures (I wonder if the jpg-file will
> survive)
>  to compute frequency distributions out of large sets of data. No trouble.

> I calculated your dice-test on a pentium 800 with dyalog 8.2
> and it took 8.1 second.
> This however includes the computing of the uniques.

> Would anyone know a faster standard general-purpose procedure
> calculate frequency distributions ?

> Bernard Houben
> http://www.apl.olap.club.tip.nl

> > i was having a discussion w/some friends and ....

> > we're curious about how fast (obviously chip speed will make a difference,
> but
> > we're not trying to be too scientific, just trying to get a general
> notion) the
> > current flavors of apl (as opposed to my old stsc dos versions) can
> calculate
> > the final distribution, {2,3,...,11,12}, of say 3,000,000 rolls of a pair
> of
> > dice.

> > if a few people have a few minutes to write an idiom, time it, and post
> the
> > machine used and the results of the test it would be appreciated.

> > if 3,000,000 blows up your machine because of outerproduct type memory
> > limitations, then feel free to use a lower number of iterations, but
> obviously
> > don't forgot to note this.

>  [Image]

'+/({iota}11){jot}.=+/3000000 2{rho}6000000{rhp}6' loop 10     <repeat
10 times>
83633 166171 249915 333448 416642 500076 416734 332907 250117 166749 83608
82998 166201 249756 333052 415703 499944 416671 334641 250637 166528 83869
82887 166243 250650 333258 417772 499173 416711 333399 249870 166476 83561
83003 167051 249661 333660 418282 498665 416415 333540 249498 166533 83692
83285 166714 249931 334043 416231 498984 417627 333031 250269 166808 83077
83546 166542 250101 333162 416155 500409 417050 333237 250680 166263 82855
83270 166802 249960 333942 416461 499622 416952 333190 249972 166576 83253
83379 166966 249519 333479 416497 499715 416300 333423 250837 166427 83458
83389 165735 249539 333650 417261 499670 417574 333513 249344 166727 83598
83154 166256 249792 333102 416123 499802 417882 334366 249562 166799 83162
average= 5.54  in cpu second
'fq a'loop 10     <repeat frequency_distribution 10 times with argument
a = (?3000000{rho}6) + (?3000000{rho}6) >
average= 5.881 in cpu second
[0]   r {is]fq a d;u;f
[2]   u{is]1,(1{drop]d) {not_equal}{neg}1{drop}d  BOOLEAN UNIQUNESS pattern
[3]   r{is]u/d                                                           NUB
(unique elements)
[4]   f{is]u 1{cut}{rho} d                                          COUNT their
frequencies using SAX's {cut} operator
[5]   r{is}({catenate}r),{catenate}f                          make Frequency
Histogram (see example below)

fq 3 3 4 5 6 7 7 7 2 2 2 2 0 1 1     <result is a matrix>
0 1
1 2
2 4
3 2
4 1
5 1
6 1
7 3

On 480 Mhz pentium equivalent machine under SAX (Sharp APL for Unix).

.../koji

Usual discalmer in here...I speak for myself.

Mon, 28 Jun 2004 04:16:44 GMT

Quote:

> Most recent computers are apl-proof. For several years
> I use the attached standard-procedures (I wonder if the jpg-file will
> survive)
>  to compute frequency distributions out of large sets of data. No trouble.

> I calculated your dice-test on a pentium 800 with dyalog 8.2
> and it took 8.1 second.
> This however includes the computing of the uniques.

> Would anyone know a faster standard general-purpose procedure
> calculate frequency distributions ?

> Bernard Houben
> http://www.apl.olap.club.tip.nl

> > i was having a discussion w/some friends and ....

> > we're curious about how fast (obviously chip speed will make a difference,
> but
> > we're not trying to be too scientific, just trying to get a general
> notion) the
> > current flavors of apl (as opposed to my old stsc dos versions) can
> calculate
> > the final distribution, {2,3,...,11,12}, of say 3,000,000 rolls of a pair
> of
> > dice.

> > if a few people have a few minutes to write an idiom, time it, and post
> the
> > machine used and the results of the test it would be appreciated.

> > if 3,000,000 blows up your machine because of outerproduct type memory
> > limitations, then feel free to use a lower number of iterations, but
> obviously
> > don't forgot to note this.

>  [Image]

If your apl has fast <nub> primirtive, you should avoid using <sorting array>
in your frequency distribution function.  For example

fq2
[0] r {is} fq2 data;nub
[1] nub {is} {nub}data                   get unique entries
[2] nub {is] nub[{gradeup}nub]     sort smaller vecotor
[3] r {is} nub {link} +-/ nub ={rank}0 1 data

This function runs 4 times faster under SAX.

.../koji

Mon, 28 Jun 2004 23:24:59 GMT
Thank you
The nub primitive in Dyalog/apl however works fast with integers, but
is not suitable for general use: it has great problems with large
non-integer vectors (more than 1000000), at least on my pc.
I do not recognize the functions link and rank. Could you explain
?

Bernard Houben

Quote:

>   If your apl has fast <nub> primirtive, you should avoid using <sorting array>
> in your frequency distribution function.  For example

>         fq2
> [0] r {is} fq2 data;nub
> [1] nub {is} {nub}data                   get unique entries
> [2] nub {is] nub[{gradeup}nub]     sort smaller vecotor
> [3] r {is} nub {link} +-/ nub ={rank}0 1 data

> This function runs 4 times faster under SAX.

> .../koji

Tue, 29 Jun 2004 19:15:52 GMT

Quote:

> Thank you
> I find your nub-solution interesting.
> The nub primitive in Dyalog/apl however works fast with integers, but
> is not suitable for general use: it has great problems with large
> non-integer vectors (more than 1000000), at least on my pc.
> I do not recognize the functions link and rank. Could you explain
> ?

> Bernard Houben

> >   If your apl has fast <nub> primirtive, you should avoid using <sorting array>
> > in your frequency distribution function.  For example

> >         fq2
> > [0] r {is} fq2 data;nub
> > [1] nub {is} {nub}data                   get unique entries
> > [2] nub {is] nub[{gradeup}nub]     sort smaller vecotor
> > [3] r {is} nub {link} +-/ nub ={rank}0 1 data

> > This function runs 4 times faster under SAX.

> > .../koji

{link} in SAX is same as <enclosed array> in APL2, {rank} operator
is not in APL2 or Dyalog Apl. It works like

vector  (verb) {rank} 0 1  matrix

In the above, (verb) is applied to rank 0 of vector element and rank 1 of matrix, so`

nub ={rank}0 1 data         case

it genarates
boolean matix of
1-st element from nub   =    data (this is already rank 1)
2-nd elemnt from nub   =     data
....
then +/ calculates the frequency, though it needs large amount of memory to hold
the intermediate boolean matrix for your data.

.../koji

Tue, 29 Jun 2004 23:18:57 GMT
'link' seems to be an equivalent of {enclose} in dyalog APL.
'rank' seems to be {each}.

Dyalog APL has a "union" and an "intersection" primitive (an "u" and an
upside-down "u"). The Union symbol in monadic format is the "unique"
primitive.

For a while i was confused by the WS full error from "unique" on a 3-million
element vector of decimal values in a 48 MB WS. But i soon realised that the
vector occupies 24 MB of memory (3 million 8-byte floats) and we know that
APL's use temporary copies while executing, so nothing strange really.
Performance time on a 2 million element vector two seconds or so...

But more interesting: Using "unique" on a vector of 2 million one's and
two's was about twice as fast as when the vector had only one's. Having more
different integers in the argument vector seemed to further increase the
performance speed. But after all, who takes uniques from 2 million one's
:-).
/ Tomas

Quote:
> Thank you
> I find your nub-solution interesting.
> The nub primitive in Dyalog/apl however works fast with integers, but
> is not suitable for general use: it has great problems with large
> non-integer vectors (more than 1000000), at least on my pc.
> I do not recognize the functions link and rank. Could you explain
> ?

> Bernard Houben

Quote:

> >   If your apl has fast <nub> primirtive, you should avoid using <sorting
array>
> > in your frequency distribution function.  For example

> >         fq2
> > [0] r {is} fq2 data;nub
> > [1] nub {is} {nub}data                   get unique entries
> > [2] nub {is] nub[{gradeup}nub]     sort smaller vecotor
> > [3] r {is} nub {link} +-/ nub ={rank}0 1 data

> > This function runs 4 times faster under SAX.

> > .../koji

Wed, 30 Jun 2004 00:54:27 GMT

 Page 1 of 2 [ 20 post ] Go to page: [1] [2]

Relevant Pages