Pathology Of Bad Software Architecture - C++ Language (Moderated)

C++Talk.NET
C++ language newsgroups

Archives

FAQ

Memberlist

Usergroups

Profile

Pathology Of Bad Software Architecture
Goto page Previous 1, 2, 3 ... , 15, 16, 17 Next

C++Talk.NET Forum Index -> C++ Language (Moderated)

View previous topic :: View next topic

Author

Message

Hyman Rosen
Guest

Posted: Sat Jul 09, 2005 1:33 am Post subject: Re: Pathology Of Bad Software Architecture

Sergey P. Derevyago wrote:

Quote:

IMHO I see the contradiction.
I believe, the "finalizers" you're working on aren't well defined.

I believe I have made myself sufficiently clear as to
what my finalizers look and behave like. If you claim
that they are not well-defined, please describe the
circumstances under which their behavior is unspecified.

I will continue to call my finalizers "finalizers"
because that's what they are and that's what they do.

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Peter Dimov
Guest

Posted: Sat Jul 09, 2005 1:34 am Post subject: Re: Pathology Of Bad Software Architecture

Hyman Rosen wrote:

Quote:

Peter Dimov wrote:
1. You run the finalizers in a separate thread while the
program threads are still running.

Finalizers will normally be invoked throughout the lifetime
of the program as objects become collectable, so this will
happen routinely.

Since the finalizer holds a copy of the resource, you run into the
obvious problem of releasing the resource while the live object is
still using it. For a resource that is as thread safe as an int,
this invokes undefined behavior.

What live object? Finalizers are only invoked for objects
which have become inaccessible. There can be a problem only
if copies of the resource exist outside its holder and
finalizer. *Don't Do That(TM)*

I don't understand. This subthread is about a mechanism that automatically
runs finalizers on exit and why it's a bad idea. If you don't run finalizers
on exit - Don't Do That - there's nothing to discuss.

So, before we proceed, let's clarify what, exactly, are we talking about
here.

What does your hypothetical collected runtime do when the program calls
exit() or returns from main()?

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Dilip
Guest

Posted: Sat Jul 09, 2005 1:35 am Post subject: Re: Pathology Of Bad Software Architecture

Peter Dimov wrote:

Quote:

Hyman Rosen wrote:
Peter Dimov wrote:
Why does the above mean that a global setting of implicitly running
finalizers on exit is a good idea?

It doesn't. It means that doing so is no more harmful or difficult
than running atexit functions and destructors of singletons and
statics.

There are two cases:

1. You run the finalizers in a separate thread while the program threads are
still running.

Since the finalizer holds a copy of the resource, you run into the obvious
problem of releasing the resource while the live object is still using it.
For a resource that is as thread safe as an int, this invokes undefined
behavior.

Hyman has been repeating a couple of gazillion times that a finalizer
for an object will be called *only* after the object becomes
inaccessible and hence collectable by the GC. So your statement sounds
contradictory to me.

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Hyman Rosen
Guest

Posted: Sat Jul 09, 2005 1:43 am Post subject: Re: Pathology Of Bad Software Architecture

Gerhard Menzl wrote:

Quote:

I'm having a déjà-vu here ...

Me too.

Quote:

Because it is non-deterministic and I cannot rely on the finalizers
releasing the resources in the required order, unless your concept
allows you to express dependencies between finalizers. Judging from your
descriptions, finalizers are pretty isolated pieces of code that don't
know each other. Am I wrong?

Yes, you're wrong. Finalizers are accessible until after they finish
running, so if they contain references to other objects that will
control the order of finalization. This has been discussed earlier.

Let's say we have two kinds of file descriptors, one of which depends
on another being around, and one which doesn't. The code would look
something like this:

struct IndependentFileHolder
{
struct Finalizer : GC_Finalizer {
int fd;
Finalizer(int fd) : fd(fd) { }
void finalize() { close(fd); }
};
IndependentFileHolder(const char *name) {
fd = open(name);
GC_Register_Finalizer(new Finalizer(fd));
}
int fd;
};

struct DependentFileHolder
{
struct Finalizer : GC_Finalizer {
int fd;
IndependentFileHolder *buddy;
Finalizer(int fd, IndependentFileHolder *buddy) : fd(fd), buddy(buddy) { }
void finalize() { close(fd); }
};
DependentFileHolder(const char *name, IndependentFileHolder *buddy) {
fd = open(name);
GC_Register_Finalizer(new Finalizer(fd, buddy));
}
int fd;
};

Notice how the DependentFileHolder finalizer holds a pointer to
its IndependentFileHolder buddy. That means that the independent
one can't be collected first, because until the dependent finalizer
runs, the independent one is reachable.

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Hyman Rosen
Guest

Posted: Sat Jul 09, 2005 1:45 am Post subject: Re: Pathology Of Bad Software Architecture

Peter Dimov wrote:

Quote:

OK, let's accept the desire to remove temp files as the motivating
example to run finalizers on exit.

OK.

Quote:

Under your model, do finalizers always run on exit?

I would leave that as a program preference, but for the sake of
argument, let's just say that the answer is "yes".

Quote:

Do they run before or after the nuke?

This obviously involves the design of how the compiler deals with
multi-threaded code, cancellation, and atexit processing. I can't
deal with that, so let me just respond that it doesn't matter. If
threads are still running when you try to do the final finalizer
run at exit request, then the set of accessible objects just is
larger; some things that might have gotten finalized will not be.

Quote:

And if running them on exit is optional, how could someone write
code that relies on a finalizer being invoked?

The option is given to the program. If the program wants them to
run, it will say so. If the last finalizer pass is run with threads
active, some finalizers may never be invoked, because the objects
will never become inaccessible. C'est la vie.

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Peter Dimov
Guest

Posted: Sat Jul 09, 2005 2:39 pm Post subject: Re: Pathology Of Bad Software Architecture

Hyman Rosen wrote:

Quote:

Peter Dimov wrote:

Under your model, do finalizers always run on exit?

I would leave that as a program preference, but for the sake of
argument, let's just say that the answer is "yes".

Do they run before or after the nuke?

This obviously involves the design of how the compiler deals with
multi-threaded code, cancellation, and atexit processing. I can't
deal with that, so let me just respond that it doesn't matter. If
threads are still running when you try to do the final finalizer
run at exit request, then the set of accessible objects just is
larger; some things that might have gotten finalized will not be.

That's what Sergey Derevyago has been trying to explain, if I understood his
posts correctly; that the "run finalizers on exit" option doesn't buy you
anything because it doesn't guarantee that every live object will have
either its destructor or its finalizer invoked. So you can't put anything
critical in a finalizer. The "run on exit" option may as well not be there
at all.

Let's assume, for the sake of the argument, that your collector is a mean
realtime finalizing machine that always collects an unreachable object
within a couple of microseconds after the last reference is dropped. At
program exit all remaining objects are reachable and the "run on exit" pass
will do nothing.

If this is what you had in mind, then there are no problems with it, and no
need for a compile-time or a runtime option. The collector is allowed to run
whenever it wants to; it's allowed to never stop running, for that matter.

Dedicated "run on exit" schemes have a different goal: to ensure that all
objects with finalizers are cleaned up and no temporary file is left behind.

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Hyman Rosen
Guest

Posted: Sun Jul 10, 2005 9:37 am Post subject: Re: Pathology Of Bad Software Architecture

Peter Dimov wrote:

Quote:

Is that the problem? OK, but I don't see it as an issue. I don't think
it makes much sense to do the last finalization pass while threads are
still running anyway. If it's done afterwards, then the only live objects
are the finalizers and things reachable from them. If you don't create
cycles within your finalizers, then they'll all be called eventually.

Quote:

So you can't put anything critical in a finalizer. The "run on exit"
option may as well not be there at all. Dedicated "run on exit" schemes
have a different goal: to ensure that all objects with finalizers are
cleaned up and no temporary file is left behind.

And I maintain that this is impossible in general. You cannot call
destructors on objects except as defined by normal program behavior
becuase you do not know what invariants you are breaking. Similarly
you cannot release objects while code is still running which may
access them. The alternative that works best is the one that I've
been suggesting. Stop threads, if you can, while keeping the program
in a consistent state (perhaps cooperatively). Then keep only things
reachable from finalizers as accessible. Then start calling and
releasing finalizers until the system reaches a steady state.

It's no wonder that Java produced a bad implementation, if they were
trying to achieve an impossible goal!

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Hyman Rosen
Guest

Posted: Sun Jul 10, 2005 9:38 am Post subject: Re: Pathology Of Bad Software Architecture

Sergey P. Derevyago wrote:

Quote:

2. Clean up global resources in the last (usually main) thread.

Personally, I see no need for any "nearly useful" features like
finalizers on exit...

I think you don't understand that "finalizers on exit" is how
"clean up global resources" can be done!

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Peter Dimov
Guest

Posted: Sun Jul 10, 2005 2:47 pm Post subject: Re: Pathology Of Bad Software Architecture

Hyman Rosen wrote:

Quote:

Is that the problem? OK, but I don't see it as an issue. I don't think
it makes much sense to do the last finalization pass while threads are
still running anyway. If it's done afterwards, then the only live
objects are the finalizers and things reachable from them.

We're almost there. :-)

Obviously, global references would keep other objects alive, but I see from
your further response that you are aware of that.

Quote:

[...] The alternative that works best is the one that I've
been suggesting. Stop threads, if you can, while keeping the program
in a consistent state (perhaps cooperatively). Then keep only things
reachable from finalizers as accessible. Then start calling and
releasing finalizers until the system reaches a steady state.

Who stops the threads? Who zeroes every global reference in order to keep
only finalizers alive? Who starts calling the finalizers? The programmer or
the runtime?

If it's the programmer, you don't need a "run finalizers on exit" option,
and there's nothing to discuss. If it's the runtime, we run into the usual
problems (and the follow-up questions are When and How.)

(Opaque third-party libraries aside for a moment.)

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Hyman Rosen
Guest

Posted: Mon Jul 11, 2005 6:59 am Post subject: Re: Pathology Of Bad Software Architecture

Peter Dimov wrote:

Quote:

Who stops the threads?

The runtime system/library, when some thread calls "exit" to
terminate the program.

Quote:

Who zeroes every global reference in order to keep only finalizers alive?

Nothing needs to be (or even can be, now that I think about it) zeroed.
Finalizers could execute arbitrary code through using function pointers
so global objects have to stay alive. What can go away are all the auto
objects in the threads that have shut down, and all the ones in the
thread that called exit, since exit won't return.

If the collector can determine that some global objects are in fact not
going to be used again (perhaps because no registered finalizer uses them
nor executes code indirectly), then those could be collected.

Quote:

Who starts calling the finalizers? The programmer or the runtime?

The runtime, on the call to "exit".

A finalizer is a way to allow you to take an action when the collector
reclaims an object. When you call "exit", the collector doesn't need to
bother reclaiming anything, because the memory won't be needed. But if
we want, we can tell it to do so anyway, so that we get a final round
of invoking the finalizers of the reclaimed objects. In that final round
we would like to reclaim as much as we can, but, as they say, not more.

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Vladimir Marko
Guest

Posted: Tue Jul 12, 2005 12:45 am Post subject: Re: Pathology Of Bad Software Architecture

Hyman Rosen wrote:

Quote:

Peter Dimov wrote:
Who zeroes every global reference in order to keep only finalizers alive?

Nothing needs to be (or even can be, now that I think about it) zeroed.
Finalizers could execute arbitrary code through using function pointers
so global objects have to stay alive.

You mean that the final collecting precedes the destruction of the
global objects. Let's not forget that the global objects _shall_ be
destroyed.

Quote:

What can go away are all the auto
objects in the threads that have shut down, and all the ones in the
thread that called exit, since exit won't return.

If the collector can determine that some global objects are in fact not
going to be used again (perhaps because no registered finalizer uses them
nor executes code indirectly), then those could be collected.

I think you refer to GC objects referenced by global objects. Then
the collector would have to determine that no registered finalizer and
no global object _destructor_ uses such an object. Such an analysis
would be next to impossible. Thus a GC object referenced by a global
object would never ever get finalized anyway.

Quote:

Who starts calling the finalizers? The programmer or the runtime?

The runtime, on the call to "exit".

A finalizer is a way to allow you to take an action when the collector
reclaims an object. When you call "exit", the collector doesn't need to
bother reclaiming anything, because the memory won't be needed. But if
we want, we can tell it to do so anyway, so that we get a final round
of invoking the finalizers of the reclaimed objects. In that final round
we would like to reclaim as much as we can, but, as they say, not more.

Unless the user explicitly clears all references to GC objects in
global objects the "as much as we can" seems way too little to justify
running finalizers on exit in your design.

May be if we introduced finalization interleaved with global object
destruction... but we can already get into trouble with the destruction
alone because of its unspecified order (AFAIK the reverse order of the
construction which is unspecified for namespace scope variables from
different translation units). So this is not the way to go.

Regards,

Vladimir Marko

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Dave Harris
Guest

Posted: Tue Jul 12, 2005 10:36 pm Post subject: Re: Pathology Of Bad Software Architecture

[email]bmaxa (AT) volomp (DOT) com[/email] (Branimir Maksimovic) wrote (abridged):

Quote:

class File2;

class File1 {
int fh1;
weak_ptr<File2> p2;
};

class File2 {
int fh2;
weak_ptr<File1> p1;
};

class Finaliser1 {
int fh1;
};

class Finaliser2 {
int fh2;
strong_ptr<File1> p1;
};
[...]

But now there's nothing to prevent File2 from being collected before
File1.

That's the point. This ensures that File2 is finalized before File1.

But File1 uses File2. If File2 is finalised while File1 is still in use,
File1 will break.

Quote:

If we keep File1:p2 as strong pointer then gc cannot determine
finalization order and therefore cannot finalize either, because
of finalization cycle.

Yes. That was my original point. Within this scheme we cannot express the
desired dependencies. Your modification doesn't help; it gets it wrong.

We can imagine schemes which would work. They would probably involve
explicitly registering the exceptional finaliser dependencies, rather than
letting the garbage collector deduce them by chasing pointers.

-- Dave Harris, Nottingham, UK.

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Dave Harris
Guest

Posted: Tue Jul 12, 2005 10:40 pm Post subject: Re: Pathology Of Bad Software Architecture

[email]pdimov (AT) gmail (DOT) com[/email] (Peter Dimov) wrote (abridged):

Quote:

Anyway, this "run finalizers on exit" question isn't interesting at
all. Every modern OS will clean up just fine. All resources are memory
on a low enough level. Smile

My humour analyser is failing here. A classic resource is a file with a
buffer. It's not enough for the finaliser to close the file; to be correct
it has to flush the buffer first.

So our examples in which the finaliser object just held an integer file
descriptor weren't very realistic. They'd also need access to the buffer
and probably the number of bytes in it. This is pretty much all of the
state of the File object itself - and your finaliser can't have access to
the File object.

So we need another level of indirection, a buffer object shared by File
and finaliser, which would not be needed if we only had destructors.
Finalisers complicate the design.

-- Dave Harris, Nottingham, UK.

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Dave Harris
Guest

Posted: Tue Jul 12, 2005 10:41 pm Post subject: Re: Pathology Of Bad Software Architecture

[email]hyrosen (AT) mail (DOT) com[/email] (Hyman Rosen) wrote (abridged):

Quote:

Nothing needs to be (or even can be, now that I think about it) zeroed.
Finalizers could execute arbitrary code through using function pointers
so global objects have to stay alive.

I'm not sure. Global objects do get destroyed eventually, and garbage
collected objects conceptually live forever, so logically we can defer
finalising until after global objects have been destroyed.

This means that finalisers should not use global objects, which is a
restriction. However, it also means that global objects can use finalised
ones.

I think in practice a program architecture which relies on finalisers will
be very different to one which doesn't. Library routines like fclose(), or
even close(), will have to document whether they can be invoked by
finalisers.

I am reverting to the opinion that finalisers, even these new ones, are
only useful for debugging. In my view the complexity they bring isn't
worth it. We should add GC without finalisers.

-- Dave Harris, Nottingham, UK.

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Niklas Matthies
Guest

Posted: Wed Jul 13, 2005 10:33 am Post subject: Re: Pathology Of Bad Software Architecture

On 2005-07-12 22:41, Dave Harris wrote:

Quote:

hyrosen (AT) mail (DOT) com (Hyman Rosen) wrote (abridged):
Nothing needs to be (or even can be, now that I think about it)
zeroed. Finalizers could execute arbitrary code through using
function pointers so global objects have to stay alive.

I'm not sure. Global objects do get destroyed eventually, and
garbage collected objects conceptually live forever, so logically we
can defer finalising until after global objects have been destroyed.

This means that finalisers should not use global objects, which is a
restriction.

Then they can't, in general, invoke global resource release functions
which are likely to operate on some global data.

-- Niklas Matthies

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

	C++Talk.NET Forum Index -> C++ Language (Moderated)	All times are GMT Goto page Previous 1, 2, 3 ... , 15, 16, 17 Next
Page 16 of 17