C++Talk.NET C++ language newsgroups   Archives   FAQ   Search   Memberlist   Usergroups   Register   Profile   Log in to check your private messages   Log in  realloc's design          C++Talk.NET Forum Index -> C++ Language (Moderated) View previous topic :: View next topic   Author Message baibaichen Guest Posted: Thu Nov 17, 2005 11:05 am    Post subject: realloc's design In sutter's C++ Coding Standards, he has gave realloc as an example in item5, he said "In Standard C, realloc is a infamous example of bad design. it has to do too many things: ..., it is not easily extensible. it is widely viewed as a shortsighted design failure." why the realloc is the bad design? From the user's view, this design is quite useful! For void *realloc(pblock, newsize), these are the Special ANSI Requirements (1) realloc(NULL, newsize) is equivalent to malloc(newsize) (2) realloc(pblock, 0) is equivalent to free(pblock) (except that NULL is returned) (3) if the realloc() fails, the object pointed to by pblock is left unchanged could someone give me more inforamation or reference about this issue? Thanks Chang [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Ulrich Eckhardt Guest Posted: Thu Nov 17, 2005 11:56 pm    Post subject: Re: realloc's design baibaichen wrote: Quote: In sutter's C++ Coding Standards, he has gave realloc as an example in item5, he said "In Standard C, realloc is a infamous example of bad design. it has to do too many things: ..., it is not easily extensible. it is widely viewed as a shortsighted design failure." why the realloc is the bad design? From the user's view, this design is quite useful! I know that there are several criticisms on realloc/malloc/free: 1. There is no way to query the size of an allocated block. This information must be present, but it can't be queried by the user that therefore always needs to be passed alongside, either explicitly or implied by e.g. the type. 2. There is no way to use realloc for C++ objects in general because it might contain pointers to itself. It would therefore be much better if one could e.g. request a resize that does not involve reallocation and copying. Also, querying how large this memory block could be made would be useful. These are only deficiencies, not straight design bugs though, so I can't comment on what exactly he meant. Uli [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Hyman Rosen Guest Posted: Thu Nov 17, 2005 11:58 pm    Post subject: Re: realloc's design baibaichen wrote: Quote: why the realloc is the bad design? It fails to separate the following two possible outcomes when increasing the size: 1) Lenghthen the size of the memory area pointed to by pblock. 2) Allocate a separate area of the new length, and copy over the old data. Ideally, you would want to increase length in two phases: 1) Increase size in place. 2) If (1) fails, allocate new area and copy and free old one. Especially in C++, operation (2) needs to be done at the language level, not with a memcpy. As it stands, you cannot even check to see what happened. The following code void *oldp = p; p = realloc(p, newsize); if (p != oldp) ... yields undefined behavior if in fact p is different than the old value! I suppose you could tinker with arrays of char, unsigned char oldp[sizeof(p)]; memcpy(oldp, &p, sizeof(p)); p = realloc(p, newsize); if (memcmp(oldp, &p) == 0) ... but that's not portable because pointers which are == are not necessarily the same bit-for-bit. [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Antoun Kanawati Guest Posted: Fri Nov 18, 2005 12:00 am    Post subject: Re: realloc's design baibaichen wrote: Quote: In sutter's C++ Coding Standards, he has gave realloc as an example in item5, he said "In Standard C, realloc is a infamous example of bad design. it has to do too many things: ..., it is not easily extensible. it is widely viewed as a shortsighted design failure." why the realloc is the bad design? From the user's view, this design is quite useful! For void *realloc(pblock, newsize), these are the Special ANSI Requirements (1) realloc(NULL, newsize) is equivalent to malloc(newsize) (2) realloc(pblock, 0) is equivalent to free(pblock) (except that NULL is returned) (3) if the realloc() fails, the object pointed to by pblock is left unchanged could someone give me more inforamation or reference about this issue? <imho> The missing piece is "bool resize_in_place(void *,size_t)" which would allow you to adjust the size of a block, without potentially moving it to a new address. This can reduce pointer-related headaches siginificantly, and allows the programmer a wider range of options to handle relocation of data. </imho> -- A. Kanawati [email]NO.antounk.SPAM (AT) comcast (DOT) net[/email] [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Thomas Richter Guest Posted: Fri Nov 18, 2005 12:00 am    Post subject: Re: realloc's design Hi, Quote: "In Standard C, realloc is a infamous example of bad design. it has to do too many things: ..., it is not easily extensible. it is widely viewed as a shortsighted design failure." why the realloc is the bad design? The problematic part of realloc() is that it may try to "move" the object in the memory it contains. That is, realloc() might allocate new memory, at a new position, and will then perform a byte-wise copy of the object layout in the old memory to the new memory ("shallow copy"). As such, it is of course completely blind against any C++ mechanism that might or might not exist to perform this copy: I.e. copy constructors or placements are not used. Most likely, realloc() will leave you with unusable and corrupt objects. Even worse, pointers to the objects that have been moved will not be fixed up (how?), and thus even more damage might happen. Short: Don't do that. So long, Thomas [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Ben Hutchings Guest Posted: Fri Nov 18, 2005 12:02 am    Post subject: Re: realloc's design <baibaichen (AT) gmail (DOT) com> wrote: Quote: In sutter's C++ Coding Standards, he has gave realloc as an example in item5, he said "In Standard C, realloc is a infamous example of bad design. it has to do too many things: ..., it is not easily extensible. it is widely viewed as a shortsighted design failure." why the realloc is the bad design? From the user's view, this design is quite useful! For void *realloc(pblock, newsize), these are the Special ANSI Requirements (1) realloc(NULL, newsize) is equivalent to malloc(newsize) (2) realloc(pblock, 0) is equivalent to free(pblock) (except that NULL is returned) (3) if the realloc() fails, the object pointed to by pblock is left unchanged could someone give me more inforamation or reference about this issue? The special cases (1) and (2) duplicate existing functions. In case (3), the obvious usage pblock = realloc(pblock, newsize) will leak memory; one must use an additional temporary variable to avoid this. In the usual case, realloc may succeed either by extending the block or allocating a new one. You can't tell in advance which it will do, so there is no way to do anything other than a simple memcpy in case of reallocation. In C it may be possible to fix up the results afterwards, but this is a kluge. In C++ this should be done by copy-constructors (or in future move-constructors), and it's too late to call them when realloc returns. -- Ben Hutchings Having problems with C++ templates? Your questions may be answered by <http://womble.decadentplace.org.uk/c++/template-faq.html>. [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Christopher Yeleighton Guest Posted: Fri Nov 18, 2005 12:07 am    Post subject: Re: realloc's design Suppose you have a pointer that points into your buffer. After the buffer is reallocated, the pointer may get invalidated. You have to notify all owners of such pointers that a reallocation has happened and to make them adjust their pointers. This is not easy in general; the easiest remedy is to deny access by pointers to your objects altogether. That means your data cannot contain internal pointers (where object A and object B are in the same buffer and a member of object A points to object B or to a member thereof) as well. And moreover, the copy constructors of your data must 1) exist 2) have the same effect as memcpy. However, there are also pros of realloc: it is cheap to do a series of realloc by one object where there are no allocations in between that could cause heap fragmentation. But moveable memory handles are even better because the memory can be defragmented behind the scenes. How about a comparison of a vector and a reallocating buffer? Chris "baibaichen" <baibaichen (AT) gmail (DOT) com> wrote Quote: In sutter's C++ Coding Standards, he has gave realloc as an example in item5, he said "In Standard C, realloc is a infamous example of bad design. it has to do too many things: ..., it is not easily extensible. it is widely viewed as a shortsighted design failure." why the realloc is the bad design? From the user's view, this design is quite useful! For void *realloc(pblock, newsize), these are the Special ANSI Requirements (1) realloc(NULL, newsize) is equivalent to malloc(newsize) (2) realloc(pblock, 0) is equivalent to free(pblock) (except that NULL is returned) (3) if the realloc() fails, the object pointed to by pblock is left unchanged could someone give me more inforamation or reference about this issue? Thanks Chang [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top johnchx2@yahoo.com Guest Posted: Fri Nov 18, 2005 12:17 am    Post subject: Re: realloc's design baibaichen wrote: Quote: (1) realloc(NULL, newsize) is equivalent to malloc(newsize) Therefore redundant and potentially confusing. Quote: (2) realloc(pblock, 0) is equivalent to free(pblock) (except that NULL is returned) Also redundant and potentially confusing. Given a source code line: q = realloc( p, s ); what does it do? It might allocate like malloc(), it might de-allocate like free(), or it might try-to-expand-in-place (the "real" purpose of realloc()). And if p and s are variables, you can't tell what it's supposed to do until run-time...it might do any one of the three. The biggest problem with realloc(), though, is that it "implements a policy, rather than providing a mechanism." That is, realloc() says...try to expand this block in place by a fixed amount, and if that fails, malloc() at least that much someplace else, memcpy() the existing object(s) to that new location, and free() the old memory. The whole "decision tree" is wrapped up inside one opaque function. Suppose we want to try expanding a block in memory by x bytes, but if that fails, we'll settle for y bytes... you can't tell that to realloc(). Or suppose we want to ask whether we can expand a block in place at all, and, if so, by how much? You can't ask realloc(). And, of course, for C++ users, realloc()'s use of bitwise copying is unhelpful...and there's no "natural" way to extend it to be copy-ctor-aware. Fundamentally, if realloc() happens to do exactly what you want, then it's great...but if you want something even slightly different, you're out of luck. Of course, some C implementations now provide access to the "building block" functions which underlie realloc() as non-portable extensions (e.g. to allow you to query the maximum amount by which a block can grow in place). But a better design would have been to offer those in the first place, possibly with something like realloc() as a convenience function. [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top baibaichen Guest Posted: Fri Nov 18, 2005 10:32 am    Post subject: Re: realloc's design As i can image, when we need apply semantic realloc to c++ object, it should be in std::vector like situation. so that's only situation that i can think to show the realloc's design problem, is there any ohter case where realloc meet c++ object? thanks [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top kanze Guest Posted: Fri Nov 18, 2005 4:42 pm    Post subject: Re: realloc's design baibaichen wrote: Quote: In sutter's C++ Coding Standards, he has gave realloc as an example in item5, he said "In Standard C, realloc is a infamous example of bad design. it has to do too many things: ..., it is not easily extensible. it is widely viewed as a shortsighted design failure." I think he's overstating it. Realloc is simply a function which has outlived its usefulness. It doesn't work with modern programming paradigms. At the time when it was first invented (late 1970's?), it did make sense, and back in the days when I started using C, I found it quite useful. Other's have pointed out that the ISO C version of realloc is also a recepe for memory leaks when it fails. It is, perhaps, worth pointing out that the original (K&R) realloc didn't have this defect. Originally, the idiom was to free the memory, then realloc it, and the pointer you passed to realloc was required to be the same as the last pointer passed to free. If realloc failed, the memory remained freed. (A very early change, present in the first C compilers I used, had realloc call free on the memory if the pointer wasn't the last one freed.) Perhaps ISO should have dropped realloc when standardizing C. While it was widespread, different implementations had significantly different semantics, and the ISO rules broke all of the code which depended on calling realloc with an already free'd pointer. On the other hand, even as late as 1990, it was still relevant and useful for some applications on some platforms. Even today, if I were writing in C, I'd use it. Although more because it's idiomatic and convenient, rather than because I needed for performance reasons or whatever. -- James Kanze GABI Software Conseils en informatique orientée objet/ Beratung in objektorientierter Datenverarbeitung 9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34 [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top johnchx2@yahoo.com Guest Posted: Sat Nov 19, 2005 10:26 am    Post subject: Re: realloc's design baibaichen wrote: Quote: As i can image, when we need apply semantic realloc to c++ object, it should be in std::vector like situation. That would probably be the most common situation. I think the "realloc problem" really came to prominence in the context of trying to improve the efficiency of std::vector implementations and/or std::vector "replacements." For example: http://www.cuj.com/documents/s=7988/cujcexp1912alexandr/alexandr.htm Quote: so that's only situation that i can think to show the realloc's design problem, is there any ohter case where realloc meet c++ object? It could crop up anywhere that involves low-level memory management -- custom allocators, for instance. And realloc()'s redundant uses ( as a synonym for malloc() or as a synonym for free() ) can make any code that uses them less readable. [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Howard Hinnant Guest Posted: Sat Nov 19, 2005 3:52 pm    Post subject: Re: realloc's design In article <1132253004.026578.302150 (AT) g14g2000cwa (DOT) googlegroups.com>, "johnchx2 (AT) yahoo (DOT) com" <johnchx2 (AT) yahoo (DOT) com> wrote: Quote: Of course, some C implementations now provide access to the "building block" functions which underlie realloc() as non-portable extensions (e.g. to allow you to query the maximum amount by which a block can grow in place). But a better design would have been to offer those in the first place, possibly with something like realloc() as a convenience function. This is currently being proposed to the C committee here: http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1085.htm It was first reviewed just this last Fall (2005). The result of that review was mixed interest, and to pass it over to the C++ committee for interest. The C++ committee has not yet had the chance to review it. If you are interested in this proposal, let your standards body know (either at the C or C++ level). If you have suggestions on how to better this proposal, let me know. The C++ committee will see a version of this paper rewritten from a C++ point of view (featuring vector, and an augmented allocator interface). However I strongly suspect that the C++ bits won't fly unless this functionality is adopted at the C level. The above proposal is highly backwards compatible, allowing for a smooth api and abi transition. It also allows for a zero-effort transition on the part of clients (if you don't want it, ignore it), and a minimum effort transition effort on the part of vendors (there is a trivial and useless, but conforming implementation detailed in the paper (written mostly in terms of the existing malloc interface). It is a small proposal, consisting only of these 4 additional functions to the malloc family (constrained to interoperate with the existing malloc functions): size_t sizeof_alloc(void* ptr); void* request_malloc(size_t size_requested, size_t* size_received); int resize_alloc(void* ptr, size_t size_requested, size_t* size_received); int negotiate_alloc(void* ptr, size_t min_size, size_t preferred_size, size_t* size_received); -Howard [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Niklas Matthies Guest Posted: Sat Nov 19, 2005 7:26 pm    Post subject: Re: realloc's design On 2005-11-19 15:52, Howard Hinnant wrote: : Quote: http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1085.htm : int negotiate_alloc(void* ptr, size_t min_size, size_t preferred_size, size_t* size_received); The specification for negotiate_alloc() is badly written, in my opinion. From the first sentence, "The intention of this function is to expand a block of memory in place to preferred_size, but failing to do that, expand to min_size", one would assume that on a non-zero return the size of the memory block will be either preferred_size or else min_size. But then the postcondition only states that it will be at least min_size. In general, it should be made clearer that "expand/shrink to x" actually means "expand/shrink to x plus some epsilon", where epsilon is imposed by the allocator (e.g. alignment padding). Secondly, and that's the important point, the intention should be specified as _maximizing_ the expanded size within the range [min_size, preferred_size + epsilon]. If preferred_size is not available, but min_size is, then in most situations (such as the char_array_push_back() example given in the notes) one still wants to expand as much as possible within the available range, and not just fall back to min_size. Note that the example implementation given for negotiate_alloc() in the notes actually corresponds with that intent. Also, the sentence "In all cases, resize_alloc will have been called a maximum of two times" with regard to the example implementation is not necessarily true for the multi-threaded case. (That's the reason why it is important to have the maximization as an atomic primitive.) -- Niklas Matthies [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Krzysztof ¯elechowski Guest Posted: Mon Nov 21, 2005 9:50 am    Post subject: Re: realloc's design U¿ytkownik "kanze" <kanze (AT) gabi-soft (DOT) fr> napisa³ w wiadomo¶ci news:1132317935.768137.253880 (AT) g43g2000cwa (DOT) googlegroups.com... Quote: Other's have pointed out that the ISO C version of realloc is also a recepe for memory leaks when it fails. It is, perhaps, worth pointing out that the original (K&R) realloc didn't have this defect. Originally, the idiom was to free the memory, then realloc it, and the pointer you passed to realloc was required to be the same as the last pointer passed to free. If realloc failed, the memory remained freed. (A very early change, present in the first C compilers I used, had realloc call free on the memory if the pointer wasn't the last one freed.) It would break MicrosoftVC's debugging heap because it overwrites the freed memory with a freed memory signature bit pattern. Chris [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top kanze Guest Posted: Tue Nov 22, 2005 11:32 am    Post subject: Re: realloc's design Krzysztof ¯elechowski wrote: Quote: U¿ytkownik "kanze" <kanze (AT) gabi-soft (DOT) fr> napisa³ w wiadomo¶ci news:1132317935.768137.253880 (AT) g43g2000cwa (DOT) googlegroups.com... Other's have pointed out that the ISO C version of realloc is also a recepe for memory leaks when it fails. It is, perhaps, worth pointing out that the original (K&R) realloc didn't have this defect. Originally, the idiom was to free the memory, then realloc it, and the pointer you passed to realloc was required to be the same as the last pointer passed to free. If realloc failed, the memory remained freed. (A very early change, present in the first C compilers I used, had realloc call free on the memory if the pointer wasn't the last one freed.) It would break MicrosoftVC's debugging heap because it overwrites the freed memory with a freed memory signature bit pattern. It would doubtlessly break a lot of implementations today. I believe that the C committee changed it because it was already breaking implementations back then. (Some were returning memory directly to the system, which was marking it as inaccessible.) But I'm afraid that back in the early seventies, when realloc was first invented, the authors didn't take MSVC's debugging heap into consideration. -- James Kanze GABI Software Conseils en informatique orientée objet/ Beratung in objektorientierter Datenverarbeitung 9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34 [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First         C++Talk.NET Forum Index -> C++ Language (Moderated) All times are GMT Page 1 of 1   Jump to: Select a forum C/C++----------------C LanguageC Language (Moderated)C++ Language (Moderated)C++ language (comp.lang.c++)C++ language, library and standards Non-English C/C++ Newsgroups----------------C++ (German)C++ (French)  You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum Powered by phpBB © 2001, 2006 phpBB Group SEO toolkit © 2004-2006 webmedic.