C++Talk.NET C++ language newsgroups   Archives   FAQ   Search   Memberlist   Usergroups   Register   Profile   Log in to check your private messages   Log in  STL allocators and containers of containers Goto page 1, 2  Next          C++Talk.NET Forum Index -> C++ Language (Moderated) View previous topic :: View next topic   Author Message apm Guest Posted: Fri Apr 23, 2004 12:33 am    Post subject: STL allocators and containers of containers Suppose I have a custom STL allocator called apm_alloc that does something special like allocate from shared memory. Suppose that I also have an object 'a' which is another allocator which might be quite different to an apm_alloc. Take a look at the decl below: std::vector<std::string, apm_alloc> myvec(a); Which allocator will be used to allocate the memory required by the string objects? Does the std say which allocator will be used? My gut feeling is that the default allocator will be used for allocating the strings. I haven't the faintest idea which allocator will be used by vector's ctor for any dynamic memory it needs other than that for the string objects themselves. Based on my meagre understanding of the std, when one uses an allocator as a ctor argument it is just an optimisation hint that could be ignored, basically just boiling down to a quality of implementation (QOI) issue. But I expect that the allocator specified as part of the variables type WILL be used for those container types that do use their allocator(I understand from Meyers III item 10 that many do not). This would suggest that the decl above has myvec using apm_alloc to allocate vector memory not associated with the string objects themselves and it might use allocator 'a' depending on QOI. Any thoughts? Regards, Andrew M. [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top moswald Guest Posted: Fri Apr 23, 2004 11:09 pm    Post subject: Re: STL allocators and containers of containers If 'a' doesn't derive from apm_alloc, then it will not match the constructor vector(const Allocator &), and will probably generate a compiler error instead. If it *does* derive from apm_alloc, then when myvec allocates memory it will use 'a' as it's allocator. [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top llewelly Guest Posted: Sat Apr 24, 2004 12:01 am    Post subject: Re: STL allocators and containers of containers [email]apm35 (AT) student (DOT) open.ac.uk[/email] (apm) writes: Quote: Suppose I have a custom STL allocator called apm_alloc that does something special like allocate from shared memory. Suppose that I also have an object 'a' which is another allocator which might be quite different to an apm_alloc. Take a look at the decl below: std::vector<std::string, apm_alloc> myvec(a); Which allocator will be used to allocate the memory required by the string objects? There are typically two (or more) pieces of memory required for the string object. One is the 'region of storage' where the literal string object resides in memory. The second (and successive) are dynamicly allocated buffers which are used to store most or all of the characters which make up the string. There may also be a dynamicly allocated reference count, etc. The region of storage used by the literal string object will be allocated with apm_alloc. AFAICT, the standard makes no requirement on where the dynamicly allocated chuncks for characters, reference count, etc, get allocated. Even replacing operator new and delete does not necessarily give you control over that memory allocation. [snip] Quote: This would suggest that the decl above has myvec using apm_alloc to allocate vector memory not associated with the string objects themselves and it might use allocator 'a' depending on QOI. Any thoughts? [snip] I think you are correct. [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Ben Hutchings Guest Posted: Sat Apr 24, 2004 12:35 am    Post subject: Re: STL allocators and containers of containers apm wrote: Quote: Suppose I have a custom STL allocator called apm_alloc that does something special like allocate from shared memory. Suppose that I also have an object 'a' which is another allocator which might be quite different to an apm_alloc. Take a look at the decl below: std::vector<std::string, apm_alloc> myvec(a); Which allocator will be used to allocate the memory required by the string objects? Does the std say which allocator will be used? My gut feeling is that the default allocator will be used for allocating the strings. The memory in which the string objects are constructed will be allocated with apm_alloc. Any additional memory they require will be allocated with std::allocator<char> or possibly some other specialisation of std::allocator. Quote: I haven't the faintest idea which allocator will be used by vector's ctor for any dynamic memory it needs other than that for the string objects themselves. It will be apm_alloc::rebind<something>::other, which should be an alias for a related allocator. (I suspect that you actually meant to write apm_alloc<std::string> above, in which case this should be an alias for apm_alloc<something>.) Quote: Based on my meagre understanding of the std, when one uses an allocator as a ctor argument it is just an optimisation hint that could be ignored, basically just boiling down to a quality of implementation (QOI) issue. But I expect that the allocator specified as part of the variables type WILL be used for those container types that do use their allocator(I understand from Meyers III item 10 that many do not). This would suggest that the decl above has myvec using apm_alloc to allocate vector memory not associated with the string objects themselves and it might use allocator 'a' depending on QOI. Any thoughts? Whether it always uses a single allocator of each type or uses several copies is a QoI issue. However all the allocators it uses will be copies of 'a' or converted copies of a type found through apm_alloc's rebind member. [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top llewelly Guest Posted: Sat Apr 24, 2004 11:22 am    Post subject: Re: STL allocators and containers of containers [email]apm35 (AT) student (DOT) open.ac.uk[/email] (apm) writes: Quote: Suppose I have a custom STL allocator called apm_alloc that does something special like allocate from shared memory. Suppose that I also have an object 'a' which is another allocator which might be quite different to an apm_alloc. Take a look at the decl below: std::vector<std::string, apm_alloc> myvec(a); Which allocator will be used to allocate the memory required by the string objects? Does the std say which allocator will be used? My gut feeling is that the default allocator will be used for allocating the strings. I haven't the faintest idea which allocator will be used by vector's ctor for any dynamic memory it needs other than that for the string objects themselves. This is one place you might prefer vector<char, apm_alloc> to string, becuase you can pass it your own allocator. [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Pete Becker Guest Posted: Sun Apr 25, 2004 1:52 am    Post subject: Re: STL allocators and containers of containers llewelly wrote: Quote: This is one place you might prefer vector<char, apm_alloc> to string, becuase you can pass it your own allocator. You can pass your own allocator to basic_string. -- Pete Becker Dinkumware, Ltd. (http://www.dinkumware.com) [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top apm Guest Posted: Sun Apr 25, 2004 2:22 am    Post subject: Re: STL allocators and containers of containers llewelly <llewelly.at (AT) xmission (DOT) dot.com> wrote Quote: apm35 (AT) student (DOT) open.ac.uk (apm) writes: This is one place you might prefer vector<char, apm_alloc> to string, becuase you can pass it your own allocator. TO get your own string that uses a customer allocator, typedef basic_string with your own allocator. After all, std::string is a typedef for basic_string with the default allocator. -apm [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top kanze@gabi-soft.fr Guest Posted: Mon Apr 26, 2004 2:25 pm    Post subject: Re: STL allocators and containers of containers llewelly <llewelly.at (AT) xmission (DOT) dot.com> wrote Quote: apm35 (AT) student (DOT) open.ac.uk (apm) writes: Suppose I have a custom STL allocator called apm_alloc that does something special like allocate from shared memory. Suppose that I also have an object 'a' which is another allocator which might be quite different to an apm_alloc. Take a look at the decl below: std::vector<std::string, apm_alloc> myvec(a); Which allocator will be used to allocate the memory required by the string objects? Interally allocated memory for the strings will use the standard allocator. Quote: There are typically two (or more) pieces of memory required for the string object. One is the 'region of storage' where the literal string object resides in memory. The second (and successive) are dynamicly allocated buffers which are used to store most or all of the characters which make up the string. There may also be a dynamicly allocated reference count, etc. The region of storage used by the literal string object will be allocated with apm_alloc. AFAICT, the standard makes no requirement on where the dynamicly allocated chuncks for characters, reference count, etc, get allocated. Even replacing operator new and delete does not necessarily give you control over that memory allocation. String is a typedef for: std::basic_string< char, std::char_traits< char >, std::allocator< char > ) Any memory used internally must be allocated using the standard allocator, which must, in term, use the global ::operator new and ::operator delete functions. -- James Kanze GABI Software mailto:kanze (AT) gabi-soft (DOT) fr Conseils en informatique orientée objet/ http://www.gabi-soft.fr Beratung in objektorientierter Datenverarbeitung 9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34 [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top apm Guest Posted: Tue Apr 27, 2004 10:48 am    Post subject: Re: STL allocators and containers of containers "moswald" <moswald (AT) glasseye (DOT) net> wrote Quote: If 'a' doesn't derive from apm_alloc, then it will not match the constructor vector(const Allocator &), and will probably generate a compiler error instead. If it *does* derive from apm_alloc, then when myvec allocates memory it will use 'a' as it's allocator. Unfortunately not. apm_alloc needs to derive from std:allocator but std:allocator is not designed for use as a base class; the allocate and deallocate functions are not virtual. This means that when std::vector uses its allocator it will use the allocate and deallocate functions on the base class. This leads me to believe that the allocator ctor argument is a complete waste of time. Does anyone know why it's there? It seems to me that it buys you nothing. I think the only reliable way to have a custom allocator is for it to be in the type. This is horrible because if a class called Foo, say, has STL private data member, like vector<int>, say, and we want Foo to be able to use a custom allocator. Then Foo will need to be a template class that can specify the type when it declares its vector of int. But because the allocator type will itself be a template to do this requires that the compiler supports template templates. So it appears that the only way to design objects that have custom allocators is with a compiler that supports template templates. Deep sigh. Regards, Andrew M. [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top apm Guest Posted: Wed Apr 28, 2004 12:41 am    Post subject: Re: STL allocators and containers of containers Pete Becker <petebecker (AT) acm (DOT) org> wrote Quote: llewelly wrote: This is one place you might prefer vector<char, apm_alloc> to string, becuase you can pass it your own allocator. You can pass your own allocator to basic_string. After alot of thought I have come to the conclusion that custom allocators have some serious problems that cannot be overcome with the std as it is. The problem is that AFAICS there is no reliable way to have custom allocators whose behaviour can be configured at runtime. Here are the 3 choices for customisable behaviour: 1) replace global new/delete Not good enough. This is the all-or-nothing approach. I want to be able to specify the allocation policy on a per-object basis. 2) Specify as custom allocator in the type. Not portable. From a practical point of view the custom allocator needs to be stateless (ESTL item 10) but having runtime configuration means it needs to be stateful. This would only work for a particular STL implementation that you knew supported stateful allocators. 3) Supply the allocator as a ctor argument. One would need to derive from std:allocator to avoid a compilation error when passing the allocator to STL variables such as std::vector<int>. But std:allocator is not designed as a base class. The derived allocate and deallocate methods would not get called because they are not virtual. So this simply doesn't work. So has anyone out there used custom allocators with runtime customisable behaviour? How can these problems be overcome? Regards, Andrew Marlow [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Pete Becker Guest Posted: Wed Apr 28, 2004 7:29 pm    Post subject: Re: STL allocators and containers of containers apm wrote: Quote: Pete Becker <petebecker (AT) acm (DOT) org> wrote llewelly wrote: This is one place you might prefer vector<char, apm_alloc> to string, becuase you can pass it your own allocator. You can pass your own allocator to basic_string. After alot of thought I have come to the conclusion that custom allocators have some serious problems that cannot be overcome with the std as it is. The problem is that AFAICS there is no reliable way to have custom allocators whose behaviour can be configured at runtime. That's not a problem for all custom allocators, just those that maintain non-static state. Quote: 2) Specify as custom allocator in the type. Not portable. From a practical point of view the custom allocator needs to be stateless (ESTL item 10) but having runtime configuration means it needs to be stateful. This would only work for a particular STL implementation that you knew supported stateful allocators. Again, that's only true if you need stateful allocators. If you do, and your vendor doesn't support 'em, complain. Or get our library -- available for the most commonly used compilers. And while you're at it, take a look at our CoreX library, which is a toolkit for building custom allocators. Quote: 3) Supply the allocator as a ctor argument. One would need to derive from std:allocator to avoid a compilation error when passing the allocator to STL variables such as std::vector<int>. But std:allocator is not designed as a base class. The derived allocate and deallocate methods would not get called because they are not virtual. So this simply doesn't work. Sweeping generalizations again. Same comments as before. To customize the allocator you pass your allocator type as one of the template arguments to the container. If you need some specific form of initialization you can then pass an allocator to the container's constructors. -- Pete Becker Dinkumware, Ltd. (http://www.dinkumware.com) [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top apm Guest Posted: Fri May 14, 2004 6:26 pm    Post subject: Re: STL allocators and containers of containers Pete Becker <petebecker (AT) acm (DOT) org> wrote Quote: After alot of thought I have come to the conclusion that custom allocators have some serious problems that cannot be overcome with the std as it is. The problem is that AFAICS there is no reliable way to have custom allocators whose behaviour can be configured at runtime. That's not a problem for all custom allocators, just those that maintain non-static state. Indeed. But sometimes a custom allocator will want to do that. Quote: 2) Specify as custom allocator in the type. Not portable. From a practical point of view the custom allocator needs to be stateless (ESTL item 10) but having runtime configuration means it needs to be stateful. This would only work for a particular STL implementation that you knew supported stateful allocators. Again, that's only true if you need stateful allocators. I happen to have that need. Quote: If you do, and your vendor doesn't support 'em, complain. I think my complaint would be judged invalid. Compiler/STL vendors are not obliged by the std to provide support for stateful allocators. Quote: Or get our library -- available for the most commonly used compilers. That's what I meant about not being portable. The code would then only work with particular implementations of the STL that offered more than the minimum the std says. Quote: And while you're at it, take a look at our CoreX library, which is a toolkit for building custom allocators. That does sound nice. Do you have a URL? Quote: 3) Supply the allocator as a ctor argument. One would need to derive from std:allocator to avoid a compilation error when passing the allocator to STL variables such as std::vector<int>. But std:allocator is not designed as a base class. The derived allocate and deallocate methods would not get called because they are not virtual. So this simply doesn't work. To customize the allocator you pass your allocator type as one of the template arguments to the container. So we agree then. Just supplying it as a ctor arg doesn't work. As you say, one needs to pass the allocator type as a template argument. That was my point. -Andrew M. [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Pete Becker Guest Posted: Sat May 15, 2004 2:23 pm    Post subject: Re: STL allocators and containers of containers apm wrote: Quote: If you do, and your vendor doesn't support 'em, complain. I think my complaint would be judged invalid. Compiler/STL vendors are not obliged by the std to provide support for stateful allocators. Compiler/STL vendors are obliged to provide what customers want. Quote: That does sound nice. Do you have a URL? www.dinkumware.com -- Pete Becker Dinkumware, Ltd. (http://www.dinkumware.com) [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top James Kanze Guest Posted: Sun May 16, 2004 11:41 pm    Post subject: Re: STL allocators and containers of containers Pete Becker <petebecker (AT) acm (DOT) org> writes: Quote: apm wrote: If you do, and your vendor doesn't support 'em, complain. I think my complaint would be judged invalid. Compiler/STL vendors are not obliged by the std to provide support for stateful allocators. Compiler/STL vendors are obliged to provide what customers want. I want full ISO compliance, including export. I agree with what you say, but I have the impression that a lot of vendors aren't living up to their obligations. (To be frank, while waiting for export, I'd be happy if I could just find two compilers which implement the same name lookup rules. I always thought that that was the reason we had standards.) -- James Kanze Conseils en informatique orientée objet/ Beratung in objektorientierter Datenverarbeitung 9 place Sémard, 78210 St.-Cyr-l'École, France +33 (0)1 30 23 00 34 [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top llewelly Guest Posted: Tue May 18, 2004 3:31 pm    Post subject: Re: STL allocators and containers of containers James Kanze <kanze (AT) gabi-soft (DOT) fr> writes: Quote: (To be frank, while waiting for export, I'd be happy if I could just find two compilers which implement the same name lookup rules. I always thought that that was the reason we had standards.) It looks like this will be rough one for the near future. Newly-minted gcc 3.4 has two-phase name lookup. gcc < 3.4 (much more widespread) does not. VC .net 2003 does not, but I hear microsoft plans to implement it someday. EDG does two-phase, but some compilers with EDG front-ends disable it by default (or so I hear). Ironically, this situation exists in part because compilers are changing in order to more closely match the standard. [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First         C++Talk.NET Forum Index -> C++ Language (Moderated) All times are GMT Goto page 1, 2  Next Page 1 of 2   Jump to: Select a forum C/C++----------------C LanguageC Language (Moderated)C++ Language (Moderated)C++ language (comp.lang.c++)C++ language, library and standards Non-English C/C++ Newsgroups----------------C++ (German)C++ (French)  You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum Powered by phpBB © 2001, 2006 phpBB Group SEO toolkit © 2004-2006 webmedic.