 |
C++Talk.NET
C++ language newsgroups
|
| View previous topic :: View next topic |
| Author |
Message |
Gianni Mariani
Guest
|
 Posted: Thu Nov 09, 2006 11:21 pm Post subject: Trip Report: Ad-Hoc Meeting on Threads in C++ |
 |
|
I just became aware of this article by Eric Niebler
http://www.artima.com/cppsource/threads_meeting.html
I just want to add one thought on the "upgradable_mutex". I remember
implementing one of these over 10 years ago and quickly reverted it
because it's next to useless in practice. I don't know the exact
semantics of the one proposed by Lawrence Crowl, but I found that the
semantics I had would almost always cause a deadlock. The thing is,
unless you give up your read lock and then request a write lock, you're
almost always going to cause a deadlock.
Simple sequence.
TN - Thread N
T1 - Readlock A
T2 - Readlock B
T1 - Upgrade A to Writelock (must wait until T2 releases Readlock)
T2 - Upgrade A to Writelock (must waut until T1 releases Readlock)
oops - neither T1 or T2 will release readlock A!
So, if the semantics of the upgrade is to release the readlock first I
would argue that it is an "unexpected" thing for most programmers that a
conversion causes a prior release of the lock. In fact, simply
requiring the the read lock is released before quiring the write locks
is a less complex, easier to understand interface.
Also, can anyone point me to the CoreX api ? I have yet to see it.
The yet to be released version (early draft is available here:
http://netcabletv.org/public_releases/) contains a number of new
features. (warning - it's big - 100meg - contains some prebuilt binaries)
As some of you may know I have been developing the Austria C++ api which
also has a threading interface. It's a very simple API. There is also
a "thread pool" type API called ActivityList that allows a hybrid
callback/event model which gives you all the benefits of event
processing with a callback style interface.
There is also a thread safe smart pointer. i.e. two threads can
read/write to the same smart pointer (i.e. atomic smart pointer thing)
which is a specialization of the regular austria smart pointer.
I also was under the impression that the terminology for a
"reader/writer" mutex was an "access" mutex. "Access" in the sense that
it controls the type of access.
One of the other things that I don't see handled very well in real-life
multithreaded code is the notion of thread life (joining) and object
life. At least in Austria C++, a thread is an object and the lifetime
of the thread does not exceed the lifetime of the object (i.e. deleting
the object WILL incur a join). There are too many API's I have come
across where it is impossible to synchronize object deletion and thread
join reliably - (win32's wininet is one).
Perhaps the only niggling repetitious error in clients using the Austria
API is that it requires the client developer to place waits (joins) in
the most derived destructor.
The other interesting feature in the Austria API is the "TwinMT" system.
This allows a class designer to define a reliable "coupling" system.
For example, the Austria C++ Timer API uses this. A client can simply
delete itself and the deregistration of the timer happens automatically
(No fuss).
The concept of object coupling is a very powerful one and makes writing
client code much easier to write.
Now, I don't think I really care which API ends up in the standard as
long as it is not intellectutual property encumbered in any way and
provides all the primitives so I can remove all the cross platform hacks
from my Austria C++ code.
The Austria C++ low level API's from the latest rev) are:
a) Atomic (integer and pointer types)
increment, decrement, add, exchange, compare/exchange
b) Mutex, Condition Variable (mutex/condition), Exclusion lock
c) Thread, Thread::wait (join), ThreadID, Thread exit status
d) scheduler control (yield)
The rest of the Austria C++ Threading API is built upon these and include:
Lock (RAII mutex aquire release)
Unlock (like a Lock but does a release and then an aquire on destruct)
Swaplock ( "swaps" locks )
Trylock (Like an RAII lock but does not block)
smart pointer traits to make atomic smart pointers
thread pools + single threaded pools
While I am at it, I should describe a really really interesting use of
Austria thread pools. The austira C++ thread pools are lists of
"activities" with a special "consumer". The run-o-the-mill consumer
type is a "thread pool" where you get to nominate the maximum number
of threads that are used to consume enqueued activities. To make the
API really simple, the activity list is nominated at the time of
construction of an activity so an activity simply needs to be
"enqueued". Trying to enqueue and already enqueued activity is
basically a nop which means that an activity can only be called by one
thread at any time. Replace the regular thread pool consumer with an
activity and now you have an activity list which will execute activities
one at a time. That means that I can create an object with N activities
and be guarenteed that only one will execute which results low cost
inbuilt synchronization. It turns out the paradigm produces a very
powerful paradigm to build MT systems. It's kind of the COM single
threaded apartment model.
Anyhow, I have enjoyed writing MT code using Austria C++. If you stick
to the API, it avoids many of the classic MT pitfalls and it's quite robust.
---
[ comp.std.c++ is moderated. To submit articles, try just posting with ]
[ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ]
[ --- Please see the FAQ before posting. --- ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html ] |
|
| Back to top |
|
 |
Howard Hinnant
Guest
|
| Posted: Thu Nov 09, 2006 11:35 pm Post subject: Re: Trip Report: Ad-Hoc Meeting on Threads in C++ |
|
|
In article
<45530096$0$3036$5a62ac22@per-qv1-newsreader-01.iinet.net.au>,
gi3nospam (AT) mariani (DOT) ws (Gianni Mariani) wrote:
| Quote: | I just became aware of this article by Eric Niebler
http://www.artima.com/cppsource/threads_meeting.html
I just want to add one thought on the "upgradable_mutex". I remember
implementing one of these over 10 years ago and quickly reverted it
because it's next to useless in practice. I don't know the exact
semantics of the one proposed by Lawrence Crowl,
|
Fwiw, you can blame the upgradable_mutex proposal on me:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2006/n2094.html#mutexs
| Quote: | but I found that the
semantics I had would almost always cause a deadlock. The thing is,
unless you give up your read lock and then request a write lock, you're
almost always going to cause a deadlock.
Simple sequence.
TN - Thread N
T1 - Readlock A
T2 - Readlock B
T1 - Upgrade A to Writelock (must wait until T2 releases Readlock)
T2 - Upgrade A to Writelock (must waut until T1 releases Readlock)
oops - neither T1 or T2 will release readlock A!
|
The proposed upgradable_mutex does not have this problem. Indeed, this
problem is the motivating use case for upgradable_mutex. The
upgradable_mutex as proposed has unique ownership semantics with respect
to other upgradable_mutexes. It can only share with other
sharable_mutexes.
| Quote: | So, if the semantics of the upgrade is to release the readlock first I
would argue that it is an "unexpected" thing for most programmers that a
conversion causes a prior release of the lock. In fact, simply
requiring the the read lock is released before quiring the write locks
is a less complex, easier to understand interface.
|
The proposal also allows for that use case.
| Quote: | One of the other things that I don't see handled very well in real-life
multithreaded code is the notion of thread life (joining) and object
life. At least in Austria C++, a thread is an object and the lifetime
of the thread does not exceed the lifetime of the object (i.e. deleting
the object WILL incur a join). There are too many API's I have come
across where it is impossible to synchronize object deletion and thread
join reliably - (win32's wininet is one).
|
In all of the proposals I've seen to date, this is easily accomplished
by creating a thin wrapper, say thread_barrier, and putting the join
operation in ~thread_barrier().
| Quote: | Perhaps the only niggling repetitious error in clients using the Austria
API is that it requires the client developer to place waits (joins) in
the most derived destructor.
|
I've seen lots of use cases for either desire: join on destruct, detach
on destruct. It is nice to easily allow either behavior.
If you have detach on destruct, it is easy and efficient to turn it into
join on destruct with a thin wrapper. If you have join on destruct and
you want to turn it into detach on destruct, you have to put the handle
on the heap. The latter is much more costly and inconvenient.
| Quote: | The Austria C++ low level API's from the latest rev) are:
a) Atomic (integer and pointer types)
increment, decrement, add, exchange, compare/exchange
b) Mutex, Condition Variable (mutex/condition), Exclusion lock
c) Thread, Thread::wait (join), ThreadID, Thread exit status
d) scheduler control (yield)
|
<nod> Sounds good.
| Quote: | The rest of the Austria C++ Threading API is built upon these and include:
Lock (RAII mutex aquire release)
|
Yup, gotta have it.
| Quote: | Unlock (like a Lock but does a release and then an aquire on destruct)
|
This hasn't been proposed, but I've privately toyed with a generic mutex
adaptor class that does this.
| Quote: | Swaplock ( "swaps" locks )
Trylock (Like an RAII lock but does not block)
|
N2094 integrates these with Lock.
| Quote: | smart pointer traits to make atomic smart pointers
thread pools + single threaded pools
While I am at it, I should describe a really really interesting use of
Austria thread pools. The austira C++ thread pools are lists of
"activities" with a special "consumer". The run-o-the-mill consumer
type is a "thread pool" where you get to nominate the maximum number
of threads that are used to consume enqueued activities. To make the
API really simple, the activity list is nominated at the time of
construction of an activity so an activity simply needs to be
"enqueued". Trying to enqueue and already enqueued activity is
basically a nop which means that an activity can only be called by one
thread at any time. Replace the regular thread pool consumer with an
activity and now you have an activity list which will execute activities
one at a time. That means that I can create an object with N activities
and be guarenteed that only one will execute which results low cost
inbuilt synchronization. It turns out the paradigm produces a very
powerful paradigm to build MT systems. It's kind of the COM single
threaded apartment model.
Anyhow, I have enjoyed writing MT code using Austria C++. If you stick
to the API, it avoids many of the classic MT pitfalls and it's quite robust.
|
Thanks for the feedback.
-Howard
---
[ comp.std.c++ is moderated. To submit articles, try just posting with ]
[ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ]
[ --- Please see the FAQ before posting. --- ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html ] |
|
| Back to top |
|
|
Gianni Mariani
Guest
|
| Posted: Fri Nov 10, 2006 10:29 pm Post subject: Re: Trip Report: Ad-Hoc Meeting on Threads in C++ |
|
|
Howard Hinnant wrote:
..
| Quote: | Are you saying that today's single threaded atexit-registered functions
can not access *any* globals?
|
It seems the thread running f() can run after all atexit functions are
called. Your code does not synchronize the exit of the f() method, does it ?
---
[ comp.std.c++ is moderated. To submit articles, try just posting with ]
[ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ]
[ --- Please see the FAQ before posting. --- ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html ] |
|
| Back to top |
|
|
Howard Hinnant
Guest
|
| Posted: Sat Nov 11, 2006 12:35 am Post subject: Re: Trip Report: Ad-Hoc Meeting on Threads in C++ |
|
|
In article <1163152319.119671.250100 (AT) h48g2000cwc (DOT) googlegroups.com>,
"James Kanze" <james.kanze (AT) gmail (DOT) com> wrote:
| Quote: | Howard Hinnant wrote:
I've seen lots of use cases for either desire: join on
destruct, detach on destruct. It is nice to easily allow
either behavior.
Or neither: I don't like anything that essential being delegated
to a destructor (which can be called as a result of an uncaught
exception). In general, I would consider calling the destructor
on a joinable (but unjoined) thread should be illegal, and both
joining and detaching should be explicit behavior on the part of
the user, and not because someone forgot to catch an exception.
|
What behavior would you suggest for ~thread() where someone forgot to
explicitly join or detach? Error not detected? Throw an exception?
Terminate?
| Quote: | The issue concerning joinable depends somewhat on how and if
futures are supported, but I don't think a destructor should
ever perform an action that can wait for a potentially infinite
amount of time.
|
<nod> That is one disadvantage to join on destruct.
| Quote: | Note that it is arguable that threads should never be detached,
since it is necessary to wait until all threads stop in order to
do a clean shutdown.
|
What is the difference between waiting for a potentially infinite amount
of time for an object to destruct, and waiting for a potentially
infinite amount of time to return from main?
| Quote: | This is probably an overly extreme position;
|
<nod>
| Quote: | But I think it safe to say that in many cases where the
application is using detached threads, it really should be
transfering the threads to some sort of reaper object, which on
shutdown will signal each thread that it should terminate, and
then join on each of the threads, possibly with a time out, just
in case.
|
Sounds like a good use case for a thread pool, or thread group.
| Quote: | The problem, of course, is that calling exit (either directly,
or by returning from main) will generally result in undefined
behavior if any other threads are still running.
|
Are you saying we should outlaw the following behavior?
#include <thread>
#include <cstdio>
#include <cstdlib>
bool stop_now = false;
std::mutex mut;
extern "C" void bye()
{
std::exclusive_lock<std::mutex> lk(mut);
std::printf("stopping\n");
stop_now = true;
}
void f()
{
while (true)
{
std::exclusive_lock<std::mutex> lk(mut);
if (stop_now)
return;
std::printf("f running\n");
}
}
int main()
{
(std::thread(f)); // detaches f on destruct
std::atexit(bye);
}
...
f running
f running
f running
stopping
The program above detaches a thread, lets it continue running after main
returns, but arranges for it to cleanly shut down via other
communication sometime during the atexit chain.
Valid program or not?
-Howard
---
[ comp.std.c++ is moderated. To submit articles, try just posting with ]
[ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ]
[ --- Please see the FAQ before posting. --- ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html ] |
|
| Back to top |
|
|
Gianni Mariani
Guest
|
| Posted: Sat Nov 11, 2006 12:36 am Post subject: Re: Trip Report: Ad-Hoc Meeting on Threads in C++ |
|
|
James Kanze wrote:
| Quote: | Howard Hinnant wrote:
I've seen lots of use cases for either desire: join on
destruct, detach on destruct. It is nice to easily allow
either behavior.
..
The problem, of course, is that calling exit (either directly,
or by returning from main) will generally result in undefined
behavior if any other threads are still running.
|
Exactly.
Given the last paragraph, I don't see how you can support your current
position.
The "undefined behaviour" you refer to here results in a very hard to
track down bug on today's implementations. This is exactly one of the
reasons that I do use join on destruct semantics. Yep, sure, if the
application code has a bug and does not cancel/cleanup/terminate then
that's an application BUG. This type of bug is usually very easy to
track down and fix since on inspection it's quite clear what is going on.
Given this, I don't understand your issue with "forgot to catch". It
seems like you're trying to outlaw well defined behaviour. I argue that
if something undesirable happens (like an exception initialed destructor
hanging), it is an application issue. If the application writer wishes
to abort, it's a behaviour that should be explicitly requested not a
mandated default.
I suppose all I'm arguing is "what is the default". Signal(cancel)+join
on destruct seems like a far more desirable alternative.
As for thread cancel mechanisms, I just don't understand how that can
work properly in the vast majority of threaded applications. I agree
that there should be a mandated way to cause a thread wrap up quickly
and "return" which I also don't understand why that mechanism is not
something that has a standard interface that is simply overridden by the
client.
I see the whole pthread_cancel mechanism to be far more complex than it
needs to be, especially for C++ code which already has a well defined
cleanup mechanism.
My experience shows this whole area of managing thread lifetime is where
most programmers get unstuck.
As for other places where programmers get unstuck, it is the whole
asynchronous event mechanism. The "future" concept is a great start but
I think that it addresses a very narrow slice of real world cases. I'll
need to write up a few cases to make the point.
---
[ comp.std.c++ is moderated. To submit articles, try just posting with ]
[ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ]
[ --- Please see the FAQ before posting. --- ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html ] |
|
| Back to top |
|
|
Gianni Mariani
Guest
|
| Posted: Sat Nov 11, 2006 1:31 am Post subject: Re: Trip Report: Ad-Hoc Meeting on Threads in C++ |
|
|
Howard Hinnant wrote:
| Quote: | In article <1163152319.119671.250100 (AT) h48g2000cwc (DOT) googlegroups.com>,
"James Kanze" <james.kanze (AT) gmail (DOT) com> wrote:
..
The program above detaches a thread, lets it continue running after main
returns, but arranges for it to cleanly shut down via other
communication sometime during the atexit chain.
Valid program or not?
|
No. bye() must join with the thread.
The thing is that once a program starts destroying globals, or the
system starts unloading DLL's/.so's everything past returning from main
is undefined.
---
[ comp.std.c++ is moderated. To submit articles, try just posting with ]
[ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ]
[ --- Please see the FAQ before posting. --- ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html ] |
|
| Back to top |
|
|
Howard Hinnant
Guest
|
| Posted: Sat Nov 11, 2006 2:51 am Post subject: Re: Trip Report: Ad-Hoc Meeting on Threads in C++ |
|
|
In article
<4554d356$0$3015$5a62ac22@per-qv1-newsreader-01.iinet.net.au>,
gi3nospam (AT) mariani (DOT) ws (Gianni Mariani) wrote:
| Quote: | Howard Hinnant wrote:
In article <1163152319.119671.250100 (AT) h48g2000cwc (DOT) googlegroups.com>,
"James Kanze" <james.kanze (AT) gmail (DOT) com> wrote:
.
The program above detaches a thread, lets it continue running after main
returns, but arranges for it to cleanly shut down via other
communication sometime during the atexit chain.
Valid program or not?
No. bye() must join with the thread.
The thing is that once a program starts destroying globals, or the
system starts unloading DLL's/.so's everything past returning from main
is undefined.
|
Are you saying that today's single threaded atexit-registered functions
can not access *any* globals?
That's not the way I read C++03, 3.6.3p3:
| Quote: | -3- If a function is registered with atexit (see <cstdlib>,
lib.support.start.term) then following the call to exit, any objects with
static storage duration initialized prior to the registration of that
function shall not be destroyed until the registered function is called from
the termination process and has completed. For an object with static storage
duration constructed after a function is registered with atexit, then
following the call to exit, the registered function is not called until the
execution of the object's destructor has completed. If atexit is called
during the construction of an object, the complete object to which it belongs
shall be destroyed before the registered function is called.
|
-Howard
---
[ comp.std.c++ is moderated. To submit articles, try just posting with ]
[ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ]
[ --- Please see the FAQ before posting. --- ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html ] |
|
| Back to top |
|
|
Howard Hinnant
Guest
|
| Posted: Sat Nov 11, 2006 5:54 am Post subject: Re: Trip Report: Ad-Hoc Meeting on Threads in C++ |
|
|
In article
<4554fc75$0$3015$5a62ac22@per-qv1-newsreader-01.iinet.net.au>,
Gianni Mariani <gi3nospam (AT) mariani (DOT) ws> wrote:
| Quote: | Howard Hinnant wrote:
.
Are you saying that today's single threaded atexit-registered functions
can not access *any* globals?
It seems the thread running f() can run after all atexit functions are
called. Your code does not synchronize the exit of the f() method, does it ?
|
That is correct. f() could return after all atexit functions are done
running and whatever else the main thread has to do to shut down the
process (which would include killing f()). All my example program
showed was that f() and the main() thread might continue to communicate
via a global while running the atexit chain (and there could well be a
bug still there in my example, just the concept of multithreading at
atexit time is what I'm exploring).
-Howard
---
[ comp.std.c++ is moderated. To submit articles, try just posting with ]
[ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ]
[ --- Please see the FAQ before posting. --- ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html ] |
|
| Back to top |
|
|
Gianni Mariani
Guest
|
| Posted: Sat Nov 11, 2006 7:29 am Post subject: Re: Trip Report: Ad-Hoc Meeting on Threads in C++ |
|
|
Howard Hinnant wrote:
| Quote: | In article
4554fc75$0$3015$5a62ac22@per-qv1-newsreader-01.iinet.net.au>,
Gianni Mariani <gi3nospam (AT) mariani (DOT) ws> wrote:
Howard Hinnant wrote:
.
Are you saying that today's single threaded atexit-registered functions
can not access *any* globals?
It seems the thread running f() can run after all atexit functions are
called. Your code does not synchronize the exit of the f() method, does it ?
That is correct. f() could return after all atexit functions are done
running and whatever else the main thread has to do to shut down the
process (which would include killing f()). All my example program
showed was that f() and the main() thread might continue to communicate
via a global while running the atexit chain (and there could well be a
bug still there in my example, just the concept of multithreading at
atexit time is what I'm exploring).
|
Ah, then the call to std::printf is at risk of exhibiting undefined
behaviour. It sounds like you don't believe me, but having experienced
it for real in many situations, I now firmly believe that the only way
to eliminate these kinds of race conditions is to provide a well defined
shut-down procedure that joins every thread that was created before
returning from main.
I also have a policy to never invoke a new thread before main() (through
global object initializers). To that end, I have a "main() initializer"
system that is used to "initialize" any objects that require threads or
in general initialization after main and shut them down before main()
exits in the Austria C++ library.
The C++ standard should strongly encourage development of simple well
defined use cases for threads and make programmers go out of their way
to create complex and potentially nonsensical/unreliable ones. The code
posted earlier is never going to work reliably in real life without
imposition of unjustifiable constraints in the standard.
I have seen so many very unnecessarily complex thread management
strategies and there is a chance here to "get it right" and result in
more reliable code from the start.
---
[ comp.std.c++ is moderated. To submit articles, try just posting with ]
[ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ]
[ --- Please see the FAQ before posting. --- ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html ] |
|
| Back to top |
|
|
James Kanze
Guest
|
| Posted: Sun Nov 12, 2006 4:22 pm Post subject: Re: Trip Report: Ad-Hoc Meeting on Threads in C++ |
|
|
Howard Hinnant wrote:
| Quote: | In article <1163152319.119671.250100 (AT) h48g2000cwc (DOT) googlegroups.com>,
"James Kanze" <james.kanze (AT) gmail (DOT) com> wrote:
Howard Hinnant wrote:
I've seen lots of use cases for either desire: join on
destruct, detach on destruct. It is nice to easily allow
either behavior.
Or neither: I don't like anything that essential being delegated
to a destructor (which can be called as a result of an uncaught
exception). In general, I would consider calling the destructor
on a joinable (but unjoined) thread should be illegal, and both
joining and detaching should be explicit behavior on the part of
the user, and not because someone forgot to catch an exception.
What behavior would you suggest for ~thread() where someone forgot to
explicitly join or detach? Error not detected? Throw an exception?
Terminate?
|
Fatal error. The true C/C++ tradition would be "undefined
behavior". But I think we both agree that that's not a
particularly good tradition, if we can avoid it. As for
throwing an exception... I expect that most of the time the
problem occurs, it will be because we are unwinding the stack as
a result of an exception, so throwing an exception becomes just
another case of terminate.
Note that it's not really a killer critera, since I can always
write my own wrapper which does whatever is necessary (except
that I don't know how to detach without calling the destructor
of the thread); that's what I do with boost::threads currently.
On the other hand, the current situation will lead to a plethora
of subtly incompatible wrapper classes. Not an ideal situation,
but since we're not talking about a class which typically
appears at the interface level between sub-systems, probably not
a serious problem in practice.
| Quote: | The issue concerning joinable depends somewhat on how and if
futures are supported, but I don't think a destructor should
ever perform an action that can wait for a potentially infinite
amount of time.
nod> That is one disadvantage to join on destruct.
|
A very big one, IMHO. While I don't expect thread unwinding due
to an exception to be blazingly fast, I don't normally expect it
to sit there for hours in one destructor, either.
| Quote: | Note that it is arguable that threads should never be detached,
since it is necessary to wait until all threads stop in order to
do a clean shutdown.
What is the difference between waiting for a potentially
infinite amount of time for an object to destruct, and waiting
for a potentially infinite amount of time to return from main?
|
Not enough:-). There is a difference: presumably, if I catch an
exception, I'm prepared to recover and continue, where as in
main, I"m probably trying to shut down. But it is a problem. A
very big one: we know that in practice, there is no reliable way
to shut down a non-collaborating thread.
| Quote: | This is probably an overly extreme position;
nod
But I think it safe to say that in many cases where the
application is using detached threads, it really should be
transfering the threads to some sort of reaper object, which on
shutdown will signal each thread that it should terminate, and
then join on each of the threads, possibly with a time out, just
in case.
Sounds like a good use case for a thread pool, or thread group.
|
Possibly. I've generally used thread groups, when I've used
them, at a lower level of granularity. Most of the thread
classes I've used (and designed) have also contained a static
registry with *all* threads, and a static function shutdown,
which signaled all running threads, and waited for them to
finish. I'm not at all sure that my solutions, however, would
be appropriate in the standard; they almost always involved some
very application specific aspects. (And they've all counted on
collaborating threads, which explicitly checked from time to
time whether shutdown had been requested or not.)
| Quote: | The problem, of course, is that calling exit (either directly,
or by returning from main) will generally result in undefined
behavior if any other threads are still running.
Are you saying we should outlaw the following behavior?
#include <thread
#include <cstdio
#include <cstdlib
bool stop_now = false;
std::mutex mut;
extern "C" void bye()
{
std::exclusive_lock<std::mutex> lk(mut);
std::printf("stopping\n");
stop_now = true;
}
void f()
{
while (true)
{
std::exclusive_lock<std::mutex> lk(mut);
if (stop_now)
return;
std::printf("f running\n");
}
}
int main()
{
(std::thread(f)); // detaches f on destruct
std::atexit(bye);
}
..
f running
f running
f running
stopping
The program above detaches a thread, lets it continue running
after main returns, but arranges for it to cleanly shut down
via other communication sometime during the atexit chain.
|
Definitly. If the thread uses any global resources, this is
irremediably broken.
Waiting in bye() until all of the threads have actually finished
(which can be determined by other means than join) would make it
almost acceptable, but there can still be problems involving
global resources freed in an earlier function invoked by atexit.
Putting the signalling and wait code at the end of main makes it
OK.
| Quote: | Valid program or not?
|
I wouldn't like to have to formulate the guarantees necessary
for it to be valid.
Note that making it undefined behavior doesn't necessarily mean
that it can't work anywhere. In this case, it may simply mean
that the limitations necessary for it to work will be negotiated
between the compiler implementor and his customers. In
practice, I don't think you can give any reasonably usable
guarantees under Posix, but perhaps other systems (and some
Posix compliant systems give more guarantees than just Posix as
well).
--
James Kanze (Gabi Software) email: james.kanze (AT) gmail (DOT) com
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
---
[ comp.std.c++ is moderated. To submit articles, try just posting with ]
[ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ]
[ --- Please see the FAQ before posting. --- ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html ] |
|
| Back to top |
|
|
James Kanze
Guest
|
| Posted: Sun Nov 12, 2006 4:23 pm Post subject: Re: Trip Report: Ad-Hoc Meeting on Threads in C++ |
|
|
Gianni Mariani wrote:
| Quote: | James Kanze wrote:
Howard Hinnant wrote:
I've seen lots of use cases for either desire: join on
destruct, detach on destruct. It is nice to easily allow
either behavior.
The problem, of course, is that calling exit (either directly,
or by returning from main) will generally result in undefined
behavior if any other threads are still running.
Exactly.
Given the last paragraph, I don't see how you can support your current
position.
|
What in my current position is in disagreement with this?
(Except, maybe, that I haven't outright said that we shouldn't
support detached threads.)
| Quote: | The "undefined behaviour" you refer to here results in a very
hard to track down bug on today's implementations. This is
exactly one of the reasons that I do use join on destruct
semantics. Yep, sure, if the application code has a bug and
does not cancel/cleanup/terminate then that's an application
BUG. This type of bug is usually very easy to track down and
fix since on inspection it's quite clear what is going on.
|
Crashing the program makes it even easier to track down and fix.
Can you give an application scenario where join on destruct is
useful in correct code?
| Quote: | Given this, I don't understand your issue with "forgot to catch". It
seems like you're trying to outlaw well defined behaviour. I argue that
if something undesirable happens (like an exception initialed destructor
hanging), it is an application issue. If the application writer wishes
to abort, it's a behaviour that should be explicitly requested not a
mandated default.
I suppose all I'm arguing is "what is the default". Signal(cancel)+join
on destruct seems like a far more desirable alternative.
|
It's certainly preferrable to simply detaching. If there are
reasonable scenarios as to where it is useful in a correct
application, I'll consider it. Until then, it's a fatal error,
along the same lines as not catching an exception. (After all,
it indirectly means that you cannot catch the exception.)
| Quote: | As for thread cancel mechanisms, I just don't understand how that can
work properly in the vast majority of threaded applications.
|
It depends on what you mean by "thread cancel mechanisms". The
Posix mechanism which goes by that name is advisory, and not
forcing. But any given application does need some means for all
running threads to be notified of a requested shutdown.
| Quote: | I agree
that there should be a mandated way to cause a thread wrap up quickly
and "return" which I also don't understand why that mechanism is not
something that has a standard interface that is simply overridden by the
client.
I see the whole pthread_cancel mechanism to be far more complex than it
needs to be, especially for C++ code which already has a well defined
cleanup mechanism.
|
The pthread_cancel mechanism has the advantage of being
implemented in kernel code, and thus being able to interrupt
blocking system requests.
--
James Kanze (Gabi Software) email: james.kanze (AT) gmail (DOT) com
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
---
[ comp.std.c++ is moderated. To submit articles, try just posting with ]
[ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ]
[ --- Please see the FAQ before posting. --- ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html ] |
|
| Back to top |
|
|
James Kanze
Guest
|
| Posted: Mon Nov 13, 2006 4:52 pm Post subject: Re: Trip Report: Ad-Hoc Meeting on Threads in C++ |
|
|
Howard Hinnant wrote:
| Quote: | In article
4554d356$0$3015$5a62ac22@per-qv1-newsreader-01.iinet.net.au>,
gi3nospam (AT) mariani (DOT) ws (Gianni Mariani) wrote:
Howard Hinnant wrote:
In article <1163152319.119671.250100 (AT) h48g2000cwc (DOT) googlegroups.com>,
"James Kanze" <james.kanze (AT) gmail (DOT) com> wrote:
The program above detaches a thread, lets it continue running after main
returns, but arranges for it to cleanly shut down via other
communication sometime during the atexit chain.
Valid program or not?
No. bye() must join with the thread.
The thing is that once a program starts destroying globals, or the
system starts unloading DLL's/.so's everything past returning from main
is undefined.
Are you saying that today's single threaded atexit-registered functions
can not access *any* globals?
|
I don't think that that's the point. Suppose you have several
threads running. You execute bye(). No problem. You continue
execution in the main thread, executing further destructors.
For example, the destructor for the mutex. What happens to the
thread waiting on the mutex?
The order of functions registered with atexit is defined. The
order things happen in a thread relative to them isn't.
Presumably, the main thread could actually finish executing all
of the destructors, and start cleaning up global memory before
one of the waiting threads woke up.
| Quote: | That's not the way I read C++03, 3.6.3p3:
|
Agreed. But you'll also agree that this only talks of single
threaded behavior.
--
James Kanze (Gabi Software) email: james.kanze (AT) gmail (DOT) com
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
---
[ comp.std.c++ is moderated. To submit articles, try just posting with ]
[ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ]
[ --- Please see the FAQ before posting. --- ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html ] |
|
| Back to top |
|
|
Gianni Mariani
Guest
|
| Posted: Mon Nov 13, 2006 10:42 pm Post subject: Re: Trip Report: Ad-Hoc Meeting on Threads in C++ |
|
|
James Kanze wrote:
| Quote: | Gianni Mariani wrote:
James Kanze wrote:
Howard Hinnant wrote:
I've seen lots of use cases for either desire: join on
destruct, detach on destruct. It is nice to easily allow
either behavior.
The problem, of course, is that calling exit (either directly,
or by returning from main) will generally result in undefined
behavior if any other threads are still running.
Exactly.
Given the last paragraph, I don't see how you can support your current
position.
What in my current position is in disagreement with this?
(Except, maybe, that I haven't outright said that we shouldn't
support detached threads.)
|
Fine, I must have misinterpreted your post. My bad. So then we agree
that detached threads are a bad thing and the standard must strongly
discourage the use of detached threads.
| Quote: |
The "undefined behaviour" you refer to here results in a very
hard to track down bug on today's implementations. This is
exactly one of the reasons that I do use join on destruct
semantics. Yep, sure, if the application code has a bug and
does not cancel/cleanup/terminate then that's an application
BUG. This type of bug is usually very easy to track down and
fix since on inspection it's quite clear what is going on.
Crashing the program makes it even easier to track down and fix.
Can you give an application scenario where join on destruct is
useful in correct code?
|
We'll just have to disagree on this. I think that attaching a debugger
to a process and being able to observe the nature of the code makes it
easier to track down.
| Quote: |
Given this, I don't understand your issue with "forgot to catch". It
seems like you're trying to outlaw well defined behaviour. I argue that
if something undesirable happens (like an exception initialed destructor
hanging), it is an application issue. If the application writer wishes
to abort, it's a behaviour that should be explicitly requested not a
mandated default.
I suppose all I'm arguing is "what is the default". Signal(cancel)+join
on destruct seems like a far more desirable alternative.
It's certainly preferrable to simply detaching. If there are
reasonable scenarios as to where it is useful in a correct
application, I'll consider it. Until then, it's a fatal error,
along the same lines as not catching an exception. (After all,
it indirectly means that you cannot catch the exception.)
|
OK, I hadn't thought through the whole exception thing since this whole
idea of exception propagation is kind of new to me.
My preferred way of handling exceptions in a thread is to have them
handled by the application otherwise just do the regular "uncaught
exception thing" (crash and burn). I find this far more preferable
since at the point of the exception being thrown I usually have a core
dump or a hung process I can go and debug easily. Once the stack is
unwound, it becomes much harder to diagnose.
I certainly don't want to see the current "uncaught exception" in a
thread behaviour be modified, I would like to see an uncaught exception
at the stack frame that caused it. I would be able to live with a child
thread hanging and awaiting for it's reaper to come along but that would
almost certainly mean deadlock in an otherwise "well formed" application.
It appears that for exception propagation to work, you must "store and
forward" the exception because the stack must be unwound (releasing
possibly held mutexes) otherwise there will be deadlocks.
I really really do prefer the uncaught exception behaviour to the store
and forward behaviour.
| Quote: |
As for thread cancel mechanisms, I just don't understand how that can
work properly in the vast majority of threaded applications.
It depends on what you mean by "thread cancel mechanisms". The
Posix mechanism which goes by that name is advisory, and not
forcing. But any given application does need some means for all
running threads to be notified of a requested shutdown.
|
Yes, it does. I have used application mechanisms in the past, I'm not
opposed to having a "proper" standard mechanism.
| Quote: |
I agree
that there should be a mandated way to cause a thread wrap up quickly
and "return" which I also don't understand why that mechanism is not
something that has a standard interface that is simply overridden by the
client.
I see the whole pthread_cancel mechanism to be far more complex than it
needs to be, especially for C++ code which already has a well defined
cleanup mechanism.
The pthread_cancel mechanism has the advantage of being
implemented in kernel code, and thus being able to interrupt
blocking system requests.
|
I assume that an application calling pthread_testcancel (or the
equivalent in the proposed standard) will have the stack rolled back
similar to when an exception is thrown ?
G
---
[ comp.std.c++ is moderated. To submit articles, try just posting with ]
[ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ]
[ --- Please see the FAQ before posting. --- ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html ] |
|
| Back to top |
|
|
Howard Hinnant
Guest
|
| Posted: Mon Nov 13, 2006 10:51 pm Post subject: Re: Trip Report: Ad-Hoc Meeting on Threads in C++ |
|
|
In article <1163341583.146827.43850 (AT) f16g2000cwb (DOT) googlegroups.com>,
"James Kanze" <james.kanze (AT) gmail (DOT) com> wrote:
| Quote: | What behavior would you suggest for ~thread() where someone forgot to
explicitly join or detach? Error not detected? Throw an exception?
Terminate?
Fatal error. The true C/C++ tradition would be "undefined
behavior". But I think we both agree that that's not a
particularly good tradition, if we can avoid it.
|
I'm a little concerned about having an object on the stack that you
can't just destruct at any moment without risking a fatal error. I'm
not sure there exists any other object in the std::lib whose use is that
finicky:
template <class T>
void foo(T t)
{
// If we don't make the right call to t, foo() ends in terminate()
}
| Quote: | As for
throwing an exception... I expect that most of the time the
problem occurs, it will be because we are unwinding the stack as
a result of an exception, so throwing an exception becomes just
another case of terminate.
|
Actually I expect most calls to ~thread() will execute in a
non-exceptional context. Having ~thread() throw would pretty much mean
that std::container<thread> doesn't behave well.
| Quote: |
Note that it's not really a killer critera, since I can always
write my own wrapper which does whatever is necessary (except
that I don't know how to detach without calling the destructor
of the thread); that's what I do with boost::threads currently.
|
Assuming std::thread looked something like boost::thread, I think adding
a detach() member to it would be a fine idea. I'm a little disconcerted
that this would leave the client with a handle to a thread which can no
longer reliably do things like fetch thread identity. Otoh, and again
assuming boost::thread semantics plus move semantics, it is trivial to
get the detach() functionality anyway, so might as well standardize it:
// Without a member detach()
inline void detach_thread(std::thread) {}
void foo()
{
std::thread(f) t;
// ...
detach_thread(std::move(t));
// t detached here
}
-------------------------
// With a member detach()
void foo()
{
std::thread(f) t;
// ...
t.detach();
// t detached here
}
<shrug> Making boost::thread movable will really change the way people
use it (much for the better). It will put a lot more power in the
client's hands. E.g.
std::vector<boost::thread> net_io_thread_group;
net_io_thread_group.push_back(boost::thread(f)); // ok!
| Quote: | The issue concerning joinable depends somewhat on how and if
futures are supported, but I don't think a destructor should
ever perform an action that can wait for a potentially infinite
amount of time.
nod> That is one disadvantage to join on destruct.
A very big one, IMHO. While I don't expect thread unwinding due
to an exception to be blazingly fast, I don't normally expect it
to sit there for hours in one destructor, either.
|
The thought of cancel-on-destruct just crossed my mind. I haven't yet
decided whether I like that idea or hate it. I'm currently leaning
towards the latter... :-)
| Quote: | Sounds like a good use case for a thread pool, or thread group.
Possibly. I've generally used thread groups, when I've used
them, at a lower level of granularity. Most of the thread
classes I've used (and designed) have also contained a static
registry with *all* threads, and a static function shutdown,
which signaled all running threads, and waited for them to
finish. I'm not at all sure that my solutions, however, would
be appropriate in the standard; they almost always involved some
very application specific aspects. (And they've all counted on
collaborating threads, which explicitly checked from time to
time whether shutdown had been requested or not.)
|
Agreed. I would like to make it easy for user's to create such
registries if they want to (multiple registries even). But I'm loath to
mandate them.
| Quote: | Are you saying we should outlaw the following behavior?
#include <thread
#include <cstdio
#include <cstdlib
bool stop_now = false;
std::mutex mut;
extern "C" void bye()
{
std::exclusive_lock<std::mutex> lk(mut);
std::printf("stopping\n");
stop_now = true;
}
void f()
{
while (true)
{
std::exclusive_lock<std::mutex> lk(mut);
if (stop_now)
return;
std::printf("f running\n");
}
}
int main()
{
(std::thread(f)); // detaches f on destruct
std::atexit(bye);
}
..
f running
f running
f running
stopping
The program above detaches a thread, lets it continue running
after main returns, but arranges for it to cleanly shut down
via other communication sometime during the atexit chain.
Definitly. If the thread uses any global resources, this is
irremediably broken.
|
Not if the global resource is not yet destructed. The current atexit /
destructor sequence is well defined. Today globals can safely be
manipulated at atexit time if the global construction is properly
coordinated with the atexit registration.
| Quote: | Valid program or not?
I wouldn't like to have to formulate the guarantees necessary
for it to be valid.
|
I think that work has already been done:
3.6.3p3: Interleave static destructors with atexit registered functions.
-Howard
---
[ comp.std.c++ is moderated. To submit articles, try just posting with ]
[ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ]
[ --- Please see the FAQ before posting. --- ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html ] |
|
| Back to top |
|
|
Howard Hinnant
Guest
|
| Posted: Tue Nov 14, 2006 4:02 am Post subject: Re: Trip Report: Ad-Hoc Meeting on Threads in C++ |
|
|
In article <1163342424.043179.87490 (AT) f16g2000cwb (DOT) googlegroups.com>,
"James Kanze" <james.kanze (AT) gmail (DOT) com> wrote:
| Quote: | That's not the way I read C++03, 3.6.3p3:
Agreed. But you'll also agree that this only talks of single
threaded behavior.
|
Sure. But imho 3.6.3p3 shouldn't say one thing for single threaded and
something completely different for multithreaded. Rather the model for
multithreading should be a natural extension of what it says for single
threaded.
| Quote: | Suppose you have several
threads running. You execute bye(). No problem. You continue
execution in the main thread, executing further destructors.
For example, the destructor for the mutex. What happens to the
thread waiting on the mutex?
The order of functions registered with atexit is defined. The
order things happen in a thread relative to them isn't.
Presumably, the main thread could actually finish executing all
of the destructors, and start cleaning up global memory before
one of the waiting threads woke up.
|
You're right. I had a bug. Here I've attempted to correct it.
Well defined?
#include <thread>
#include <cstdio>
#include <cstdlib>
bool stop_now = false;
std::mutex mut;
std::condition<std::exclusive_lock<std::mutex> > cv;
extern "C" void bye()
{
std::exclusive_lock<std::mutex> lk(mut);
std::printf("stopping f\n");
stop_now = true;
while (stop_now)
cv.wait(lk);
std::printf("f stopped\n");
}
void f()
{
while (true)
{
std::exclusive_lock<std::mutex> lk(mut);
if (stop_now)
{
stop_now = false;
cv.notify_one();
return;
}
std::printf("f running\n");
}
}
int main()
{
(std::thread(f));
std::atexit(bye);
}
Above bye() is guaranteed to start *and* finish executing before mut and
cv destruct. The "f" thread is detached, and continues running after
main exits. The "f" thread is guaranteed to shut down after bye()
starts executing and before bye() finishes executing (or at least
guaranteed not to touch any global state after bye() returns).
Do we have sufficient motivation to call this program ill-formed just
because it lets a detached thread run for awhile after main returns?
I don't think we do.
I think the most we can say is similar to what we already say in 3.6.3p2:
| Quote: | If a function contains a local object of static storage duration that has
been destroyed and the function is called during the destruction of an object
with static storage duration, the program has undefined behavior if the flow
of control passes through the definition of the previously destroyed local
object.
|
(needs to be expanded to all objects with static storage duration, not
just function-locals)
Oh, wait a sec, I think namespace scope objects are covered in 3.8p5.
-Howard
---
[ comp.std.c++ is moderated. To submit articles, try just posting with ]
[ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ]
[ --- Please see the FAQ before posting. --- ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html ] |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
