inhibiting GC 
Author Message
 inhibiting GC

hi,

I'm interested to know if there are ways to inhibit the GC from taking over.
Suppose you have a fn that you don't want to be interrupted by the GC.  I've
seen, in MP packages mp::without-scheduling, and (in ACL)
excl::without-interrupts, but I don't think these prevent GC.  Of course, this
might not sound like a good idea, but the code I'm trying to block would be
short enough not to cause potential problems with memory.

is one of these the way to go?  Or is there a better way?

dave



Sat, 17 Nov 2001 03:00:00 GMT  
 inhibiting GC

| I'm interested to know if there are ways to inhibit the GC from taking
| over.  Suppose you have a fn that you don't want to be interrupted by the
| GC.

  ("taking over"?  is that differing from "being performed"?)

  there are two basic ways to avoid GC in a particular function: (1) don't
  cons, and (2) GC before calling it.  it is not particularly hard to avoid
  consing, but it takes a special concern when writing the code, and you
  will realize that it involves doing your own local memory management,
  which basically means you either use all pre-allocated structures or you
  have to reimplement the functions that do the allocation.  a non-consing
  CONS could re-use cons cells off of a pre-allocated list, but you would
  have to free them when you are done with them.

  the problem is basically this: when GC occurs because you're out of free
  memory, what would you rather do?  fail, somehow?  that would easily cost
  more than a GC would, especially if your worry is real-time response.

  if you are only looking to avoid global gc's, that's controllable.  see
  the writeup in doc/cl/gc.htm in your Allegro CL installation directory.
  if you are looking for ways to minimize the cost of a scavenge in ACL,
  you can pretty much control that with the size of the newspace and the
  size of the oldspace allocation size.  if you don't ask for global gc
  when oldspace is full (the default, by the way), you don't do a global gc
  automatically (see *GLOBAL-GC-BEHAVIOR* and *TENURED-BYTES-LIMIT*), you
  can do global gc when you feel like it.  if your system is idle for long
  periods of time, you can make global GC happen in the idle time if that
  is more convenient.

  in general, you have pretty good control over the garbage collection, but
  I highly recommend reading the manual.  this is generally tricky stuff.

#:Erik



Sun, 18 Nov 2001 03:00:00 GMT  
 inhibiting GC

Quote:

>   the problem is basically this: when GC occurs because you're out
>   of free memory, what would you rather do?  fail, somehow?

Ask more memory from the system?

I'm not very familiar with other GC's, but I know that Elisp memory
allocator does not GC when there is no more free memory -- in that
case, it allocates more from the heap.  The GC happens after a certain
number of bytes have been allocated.  In Emacs, doing what David asks
would be achieved with:

(let ((gc-cons-threshold most-positive-fixnum))
  ... code ...)

All of this would be, of course, very implementation-dependent.



Sun, 18 Nov 2001 03:00:00 GMT  
 inhibiting GC

| Ask more memory from the system?

  you seem to have taken "no free memory" a bit differently than it was
  intended, but let's assume for the sake of argument that the system
  denies your request.  since this no different from asking more memory
  from a bounded set, the same argument applies: if GC occurs when there is
  no free memory available, what do you do?

| I'm not very familiar with other GC's, but I know that Elisp memory
| allocator does not GC when there is no more free memory

  since he was asking about the Allegro CL scavenging garbage collector, I
  thought it would be more instructive to answer in that context, but there
  are of course other GC implementations around.  Emacs', for instance, is
  horribly, horribly in efficient, and Allegro CL's is very fast and also
  very wasteful.  I'm not sure the reason is solely a question of when GC
  occurs, but after I modified my Emacs to let GC free pages that it had
  emptied of objects, it no longer grew to be quite so big.  go figure...

#:Erik



Sun, 18 Nov 2001 03:00:00 GMT  
 inhibiting GC

Quote:

>| Ask more memory from the system?

>  you seem to have taken "no free memory" a bit differently than it was
>  intended, but let's assume for the sake of argument that the system
>  denies your request.  since this no different from asking more memory
>  from a bounded set, the same argument applies: if GC occurs when there is
>  no free memory available, what do you do?

I would recommend that whatever mechanism is provided by the implementation
be in the form of a "hint".  Many GC's are proactive or use heuristics to
try to determine when a good time for GC is (e.g. waiting until the
application is idle), but the application should be able to indicate "this
isn't a good time for a GC".  If there's an emergency need for a GC, it
should probably ignore the hint.

There could be a stronger version that completely prohibits GC.  If it runs
out of available memory during that time, it should crash the application,
just as if it had performed a GC and it didn't free up any memory.

--

GTE Internetworking, Powered by BBN, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.



Sun, 18 Nov 2001 03:00:00 GMT  
 inhibiting GC

| since he was asking about the Allegro CL scavenging garbage collector, I
| thought it would be more instructive to answer in that context, but there
| are of course other GC implementations around.  Emacs', for instance, is
| horribly, horribly in efficient, and Allegro CL's is very fast and also
| very wasteful.

  cursed be editing without proper substructure navigation.  that last
  sentence was intended to read:

  Emacs', for instance, is horribly, horribly inefficient and also very
  wasteful, and Allegro CL's is very fast.

  sigh.

#:Erik



Sun, 18 Nov 2001 03:00:00 GMT  
 inhibiting GC

Quote:

>   there are two basic ways to avoid GC in a particular function: (1) don't
>   cons, and (2) GC before calling it.

I'm not sure if the original poster was interested just in ACL (where
Erik knows more than me!), but in general this isn't a safe way of
doing it because the GC may well be living in a different thread than
your code, and could be  making  all sorts of complex decisions about
whether know was a good time to GC.  Even non-multithreaded
implementations may have GCs that live off the back of timer
interrupts or something.

So I think in general you need some kind of special WITHOUT-GC type
form which the implementation might provide (and of course for
multithreaded implementations you also need WITH-SCHEDULING-INHIBITED
or something at least as much as GC inhibition).

A good example of the kind of hairiness this can incolve is Genera
which has a GC which can run in its own thread and a large number of
GC, process & memory controlling switches (which fortunately you
almost never need to adjust) up to and including things to lock bits
of virtual memory in core (very useful for doing audio on a 36xx).

--tim



Sun, 18 Nov 2001 03:00:00 GMT  
 inhibiting GC

[About GCs]

Quote:
>   Emacs', for instance, is horribly, horribly inefficient and also very
>   wasteful, and Allegro CL's is very fast.

There's an interesting story about GC and Emacs.  Back before Java,
the standard mythology was that languages with GC spent all their time
in GC which was why real programmers used sbrk instead.  At some
point, an Emacs version I use (Lemacs as was) came with a profiler
which worked at a pretty low level (below elisp, although it was
controlled from elisp), and would tell you a lot of interesting things
about where the system spent its time.

I once turned this thing on and left it turned on for several days in
the Emacs I was using to do all my work, including several largish
elisp packages like VM and gnus and probably psgml &c.  The results
were that it spent about 1-2% of its time in GC, and about 60-80% in
updating the screen.  Like Erik says, Emacs' GC is rudimentary, but it
was so far from the critical path in terms of performance as to be
totally irrelevant.  So much for mythology.

--tim



Mon, 19 Nov 2001 03:00:00 GMT  
 inhibiting GC

| I once turned this thing on and left it turned on for several days in the
| Emacs I was using to do all my work, including several largish elisp
| packages like VM and gnus and probably psgml &c.  The results were that
| it spent about 1-2% of its time in GC, and about 60-80% in updating the
| screen.  Like Erik says, Emacs' GC is rudimentary, but it was so far from
| the critical path in terms of performance as to be totally irrelevant.
| So much for mythology.

  there are several interesting aspects to this: (1) people see that
  something is happening when the screen updates, (2) people see a pause
  when the "Garbage collecting..." message lights up, and (3) they thought
  Emacs got a lot faster when that message was removed (at my request).

  if it moves slowly, humans can deal well with it.  if it moves in jerks
  and spasms, humans get nervous.  the overall impression of "smoothness"
  is very important to user perception of speed; any break in smoothness is
  then taken as a serious flaw.  we see the same thing in many areas:
  people had no problems dealing with 110 baud ttys 20 years ago, and if
  the information trickles in, people can still deal with very slow lines
  (witness the World Wide Wait, and how people happily wait for images of
  text to crawl across the line at the rate of 1 character a second), but
  if you give somebody a 56kbps line and it goes from 56 kbps to 110 bps
  and back at random intervals, you can drive people literally insane, and
  that's precisely what the WWW does to congested links.  still, it's the
  greatest thing since sliced bread.  all we have to do now is make the WWW
  better than sex for the dumb people who click on the advertising, so they
  won't procreate!  ahem, I digress.

#:Erik



Tue, 20 Nov 2001 03:00:00 GMT  
 inhibiting GC

Quote:

> There could be a stronger version that completely prohibits GC.  If it runs
> out of available memory during that time, it should crash the application,
> just as if it had performed a GC and it didn't free up any memory.

Even better would be the ability to dinamically specify the size of
a preallocated virtual memory reserve to use if you run out of normal blocks
when GC is turned off. It would be good if the implementation could ensure
that a {*filter*} page only needed to get a physical frame allocated (or it could
even be already locked in) instead of having to swap in a boring block from disk.


Fri, 23 Nov 2001 03:00:00 GMT  
 inhibiting GC

Quote:

>   very wasteful.  I'm not sure the reason is solely a question of when GC
>   occurs, but after I modified my Emacs to let GC free pages that it had
>   emptied of objects, it no longer grew to be quite so big.  go figure...

Did you report this modification to the Emacs community?  I would like to know
how you did this (esp. if it can speed up XEmacs's GC).

dave



Sat, 01 Dec 2001 03:00:00 GMT  
 inhibiting GC

Quote:

> >   very wasteful.  I'm not sure the reason is solely a question of when GC
> >   occurs, but after I modified my Emacs to let GC free pages that it had
> >   emptied of objects, it no longer grew to be quite so big.  go figure...

> Did you report this modification to the Emacs community?

Given that Erik certainly is part of that community the question is
rather useless..

Quote:
> I would like to know how you did this (esp. if it can speed up
> XEmacs's GC).

XEmacs's GC already does this.

Jan



Sun, 02 Dec 2001 03:00:00 GMT  
 
 [ 12 post ] 

 Relevant Pages 

1. GC, and objects finalization (was: GC,again)

2. GC and Aging, GC and Allocator Surveys available

3. To GC or not to GC?

4. How to inhibit panning of a waveform graph?

5. Inhibit creation of backup (~) files in edwin

6. Inhibiting of destrying X in Win95

7. Win95: inhibit network drive assumption for "//"

8. GS4.01, inhibit startup window, how?

9. Inhibiting duplicate apps in Windows?

10. 'conservative' GC == 'risky' GC

11. real-time & generational GC (free incremental GC, too)

12. 'conservative' GC == 'risky' GC

 

 
Powered by phpBB® Forum Software