C++Talk.NET C++ language newsgroups   Archives   FAQ   Search   Memberlist   Usergroups   Register   Profile   Log in to check your private messages   Log in  concurrent use of std::stringstream          C++Talk.NET Forum Index -> C++ Language (Moderated) View previous topic :: View next topic   Author Message marcel.lanz@gmail.com Guest Posted: Wed Jun 29, 2005 2:51 pm    Post subject: concurrent use of std::stringstream I use the std::stringstream class for interger <-> string conversion in an embedded threaded environment. There is no concurrent use of std::stringstream instances. Is it safe to use std::stringstream instances in different threads ? I've read that the standard doesn't says anything about thread safety, but I think this is relevant for concurrent access of library classes by different threads. I think if the std::stringstream has no statics it should be safe ? [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top tony_in_da_uk@yahoo.co.uk Guest Posted: Wed Jun 29, 2005 7:10 pm    Post subject: Re: concurrent use of std::stringstream In general, STL/stream implementations shouldn't cope with a thread reading or writing an object while another thread is writing to it. I say shouldn't, because it is impractical to do so as iterators are often invalidated by container insert/erase, and for containers and streams the performance costs are prohibitive. They should cope with concurrent readers. Different threads can take turns reading and write subject to these restrictions. Not all libraries implement these standards, but the SGI paved the way for this as a defacto standard for good libraries - check your doco. Cheers, Tony [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Maxim Yegorushkin Guest Posted: Wed Jun 29, 2005 7:14 pm    Post subject: Re: concurrent use of std::stringstream On Wed, 29 Jun 2005 18:51:23 +0400, <marcel.lanz (AT) gmail (DOT) com> wrote: Quote: I use the std::stringstream class for interger <-> string conversion in an embedded threaded environment. There is no concurrent use of std::stringstream instances. Is it safe to use std::stringstream instances in different threads ? Without locking it is safe only if all threads call nonmutating members of std::stringstream, such as std::stringstream::str(). Quote: I've read that the standard doesn't says anything about thread safety, but I think this is relevant for concurrent access of library classes by different threads. Here is a SGI STL thread safety definition which also applies for most widely used STLs. http://www.sgi.com/tech/stl/thread_safety.html -- Maxim Yegorushkin <firstname.lastname (AT) gmail (DOT) com> [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top James Kanze Guest Posted: Thu Jun 30, 2005 9:45 am    Post subject: Re: concurrent use of std::stringstream Maxim Yegorushkin wrote: Quote: On Wed, 29 Jun 2005 18:51:23 +0400, <marcel.lanz (AT) gmail (DOT) com> wrote: I use the std::stringstream class for interger <-> string conversion in an embedded threaded environment. There is no concurrent use of std::stringstream instances. Is it safe to use std::stringstream instances in different threads ? Without locking it is safe only if all threads call nonmutating members of std::stringstream, such as std::stringstream::str(). It all depends on the implementation, but I'm not aware of any that have that sort of rule. The generally accepted practice (e.g. SGI, Posix standards) is that using different instances in different threads should work; using the same instance in different threads will work only if none of the threads "modify" the instance. Except that a stringstream that no one modifies isn't very useful; about all you can do *is* call str() on it. Quote: I've read that the standard doesn't says anything about thread safety, but I think this is relevant for concurrent access of library classes by different threads. Here is a SGI STL thread safety definition which also applies for most widely used STLs. http://www.sgi.com/tech/stl/thread_safety.html It explicitly doesn't apply to g++, which gives only a weaker guarantee. The implementation in Sun CC gives a lot stronger guarantee (although the additional guarantees are largely worthless in practice, and only have the effect of slowing the implementation down). -- James Kanze mailto: [email]james.kanze (AT) free (DOT) fr[/email] Conseils en informatique orientée objet/ Beratung in objektorientierter Datenverarbeitung 9 pl. Pierre Sémard, 78210 St.-Cyr-l'École, France +33 (0)1 30 23 00 34 [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top James Kanze Guest Posted: Thu Jun 30, 2005 9:50 am    Post subject: Re: concurrent use of std::stringstream [email]marcel.lanz (AT) gmail (DOT) com[/email] wrote: Quote: I use the std::stringstream class for interger <-> string conversion in an embedded threaded environment. There is no concurrent use of std::stringstream instances. Is it safe to use std::stringstream instances in different threads ? It depends on the implementation. Quote: I've read that the standard doesn't says anything about thread safety, but I think this is relevant for concurrent access of library classes by different threads. I think if the std::stringstream has no statics it should be safe ? Maybe, maybe not. The standard makes no requirements with regards to threading. Basically, as soon as you call pthread_create, or it's equivalent, you have to depend on system specific guarantees, or some other standard (like Posix). (And be very careful who is guaranteeing what. Solaris guarantees that destructors will be called when a thread is cancelled, but that guarantee is only effective if you are using Sun CC.) As a quality of implementation issue, I would expect objects in the standard library to obey the normal thread safety guarantees for objects in the system. But there too, you have to be very careful; some compilers for Posix systems, for example, explicitly give less guarantee. As a basic rule, unless your compiler documentation explicitly defines what it does in a multithreaded environment, you cannot use the compiler in a multithreaded environment. And if it does define what it does, it should also define what guarantees it gives for the library. -- James Kanze mailto: [email]james.kanze (AT) free (DOT) fr[/email] Conseils en informatique orientée objet/ Beratung in objektorientierter Datenverarbeitung 9 pl. Pierre Sémard, 78210 St.-Cyr-l'École, France +33 (0)1 30 23 00 34 [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Martin Bonner Guest Posted: Thu Jun 30, 2005 5:27 pm    Post subject: Re: concurrent use of std::stringstream Maxim Yegorushkin wrote: Quote: On Wed, 29 Jun 2005 18:51:23 +0400, <marcel.lanz (AT) gmail (DOT) com> wrote: I use the std::stringstream class for interger <-> string conversion in an embedded threaded environment. There is no concurrent use of std::stringstream instances. Is it safe to use std::stringstream instances in different threads ? Without locking it is safe only if all threads call nonmutating members of std::stringstream, such as std::stringstream::str(). That's a bit extreme. The OP is concerned about different threads using DIFFERENT std::stringstream instances. Theoretically that is still dangerous because the different threads could be sharing class statics (or other statics). Quote: I've read that the standard doesn't says anything about thread safety, but I think this is relevant for concurrent access of library classes by different threads. Here is a SGI STL thread safety definition which also applies for most widely used STLs. http://www.sgi.com/tech/stl/thread_safety.html and that says "The SGI implementation of STL is thread-safe only in the sense that simultaneous accesses to distinct containers are safe". In other words, the OP would be OK if he is using SGI. He needs to consult the documentation of his implementation. [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Maxim Yegorushkin Guest Posted: Thu Jun 30, 2005 6:28 pm    Post subject: Re: concurrent use of std::stringstream On Thu, 30 Jun 2005 13:45:01 +0400, James Kanze <kanze (AT) none (DOT) news.free.fr> wrote: [] Quote: Here is a SGI STL thread safety definition which also applies for most widely used STLs. http://www.sgi.com/tech/stl/thread_safety.html It explicitly doesn't apply to g++, which gives only a weaker guarantee. I guess you mean that it does not apply to std::basic_string<>, or..? -- Maxim Yegorushkin <firstname.lastname (AT) gmail (DOT) com> [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top marcel.lanz@gmail.com Guest Posted: Fri Jul 01, 2005 11:04 am    Post subject: Re: concurrent use of std::stringstream I don't know yet if my stringstream implementation uses a static member. The problem seems as expected to be " OS/library implementation specific" My environment is an embedded OS (threadX) where each thread has its own stack. all threads use the same heap; hence a stack allocated class uses the standard STL allocator on the common heap. I use the libstdc++3 from the GCC. If I serialize all access to any std::stringstream instance I have no disruption with several concurrent threads (they have all their own stack allocated stringstream) libstdc++ provides different allocators. The OS comes with a patched libc, which adds a pair of malloc_Lock/malloc_Unlock functions, called in any call of malloc's lock function (__malloc_lock) the libstdc++ uses a default allocator for all its stl templates which is based on a so called __mem_interface. This mem_interface is new_handler based, which is itself malloc based; so any std::stringstream / std::string shouldn't be invalidated by its memory allocators. Unfortunately, without serialized access to any stringstream the application disrupts itself to nowhere (with no common failure point over different runs) It would be nice to have an interface from the standard which defines common hooks/ constraints about concurrent access to library classes. An object-context (ctors/dtors, allocator memory, statics) bound to context users like a thread (whatever a thread is) The more I read about the topic, the more I have the impression the standard is not interested to solve this ? [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top kanze@gabi-soft.fr Guest Posted: Fri Jul 01, 2005 11:04 am    Post subject: Re: concurrent use of std::stringstream Maxim Yegorushkin wrote: Quote: On Thu, 30 Jun 2005 13:45:01 +0400, James Kanze [email]kanze (AT) none (DOT) news.free.fr[/email]> wrote: [] Here is a SGI STL thread safety definition which also applies for most widely used STLs. http://www.sgi.com/tech/stl/thread_safety.html It explicitly doesn't apply to g++, which gives only a weaker guarantee. I guess you mean that it does not apply to std::basic_string<>, or..? Technically, g++ does not give the SGI guarantee (which corresponds to the Posix requirements) for anything in its library; the documentation explicitly says so (if you look hard enough, and know how to read between the lines). In practice, most of the STL in the g++ implementation IS the SGI code, so obviously, it will work, formal guarantee or not. (At least until someone modifies it -- g++ does not guarantee that it will always be the SGI code.) For things like iostream, the SGI guarantee is not given; practically, of course, the implementors don't go out of their way to make it not work, and I suspect that you can get away with a lot. The one case I have explicitly analysed is std::basic_string<>, and there very definitly are cases where the SGI guarantee fails. The cases are very specific, and very limited, and there are workarounds if you know them. (Just keeping two permanent copies of the same string is sufficient.) They are not documented, however (since the implementation meets the looser guarantees which are explicitly given), and formally, there is no guarantee that the workarounds will be valid in the next release. It's also worth pointing out that at least some of the people working on g++ think that g++ should give the SGI guarantee. I think some of them also agree with me that the documentation could be clearer with regards to what is and what is not guaranteed. The two problems are actually related. If, in fact, the only case where the current implementation doesn't actually meet the SGI guarantee is std::basic_string, then it may be easier to fix std::basic_string, and to fix the documentation to simply point to the SGI guarantee, rather than try and explain a different policy in detail. Especially knowing that the SGI guarantee corresponds to the Posix requirements, its implications are discussed in detail in such books as the Butenhof, and that many users just suppose such guarantees, without reading the documentation. -- James Kanze GABI Software Conseils en informatique orientée objet/ Beratung in objektorientierter Datenverarbeitung 9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34 [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top kanze@gabi-soft.fr Guest Posted: Mon Jul 04, 2005 11:05 am    Post subject: Re: concurrent use of std::stringstream [email]marcel.lanz (AT) gmail (DOT) com[/email] wrote: Quote: I don't know yet if my stringstream implementation uses a static member. The problem seems as expected to be " OS/library implementation specific" My environment is an embedded OS (threadX) where each thread has its own stack. all threads use the same heap; hence a stack allocated class uses the standard STL allocator on the common heap. That's the usual situation. Quote: I use the libstdc++3 from the GCC. In which case, the guarantees given by GCC hold (modulo errors, of course, but that's always the case). The official guarantee currently given by g++ is that as long as an object is only accessed from a single thread, external, user provided synchronization is not necessary. As soon as an object might be accessed from more than one thread, even if all of those accesses are non-modifying, external synchronization is necessary. Quote: If I serialize all access to any std::stringstream instance I have no disruption with several concurrent threads (they have all their own stack allocated stringstream) As long as each thread has its own stringstream, there is no need for external synchronization. At least with g++. Quote: libstdc++ provides different allocators. The OS comes with a patched libc, which adds a pair of malloc_Lock/malloc_Unlock functions, called in any call of malloc's lock function (__malloc_lock) the libstdc++ uses a default allocator for all its stl templates which is based on a so called __mem_interface. This mem_interface is new_handler based, which is itself malloc based; so any std::stringstream / std::string shouldn't be invalidated by its memory allocators. Unfortunately, without serialized access to any stringstream the application disrupts itself to nowhere (with no common failure point over different runs) This could be due to several things: -- Something funny is going on with your malloc_lock/malloc_unlock business, above. Normally, on all multithreaded systems I've seen, malloc and free ensure whatever locks they need internally, so that no external synchronization is necessary on the users part. (This is a requirement for Posix.) The g++ documentation doesn't say so explicitly, but I think it pretty obvious that they base their library on lower level C and system libraries, and if these libraries do not give adequate guarantees, then the g++ library guarantees are off as well. -- There could be a bug in the implementation of stringstream in the g++ library. -- There could be a problem somewhere else in your code; introducing the external synchronization around the stringstream usage changes the timing of your program, in a way that, purely by chance, makes it work. (At least until you change something else in the program.) All in all, I'd say that the third possibility is by far the most likely. Quote: It would be nice to have an interface from the standard which defines common hooks/ constraints about concurrent access to library classes. An object-context (ctors/dtors, allocator memory, statics) bound to context users like a thread (whatever a thread is) It's coming. The standards committee is definitly looking into threading. But it isn't easy; the Java people have rewritten their specifications two or three times, and we're still not sure that they have it right. Because C++ is ruled by an ISO standard, and is not just a corporate standard that can be changed on a whim, anytime we find something that doesn't work as expected, it is very important to get it right the first time. Quote: The more I read about the topic, the more I have the impression the standard is not interested to solve this ? The current standard was formally adopted in 1998. What was to go into it was more or less fixed some time around 1995 (I'm guessing a little bit about this date); many details changed since then, but no major points were added. In 1995, threading wasn't a subject well enough understood to be considered, and it wasn't widely enough used to be considered absolutely essential. Even today, I'm not 100% sure about the well enough understood part, but its ubiquity today means that the standards committee probably have to at least consider it. As they are doing. -- James Kanze GABI Software Conseils en informatique orientée objet/ Beratung in objektorientierter Datenverarbeitung 9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34 [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First         C++Talk.NET Forum Index -> C++ Language (Moderated) All times are GMT Page 1 of 1   Jump to: Select a forum C/C++----------------C LanguageC Language (Moderated)C++ Language (Moderated)C++ language (comp.lang.c++)C++ language, library and standards Non-English C/C++ Newsgroups----------------C++ (German)C++ (French)  You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum Powered by phpBB © 2001, 2006 phpBB Group SEO toolkit © 2004-2006 webmedic.