C++Talk.NET C++ language newsgroups   Archives   FAQ   Search   Memberlist   Usergroups   Register   Profile   Log in to check your private messages   Log in  retrieve size of array Goto page 1, 2  Next          C++Talk.NET Forum Index -> C++ Language (Moderated) View previous topic :: View next topic   Author Message ralph Guest Posted: Wed Aug 30, 2006 5:18 pm    Post subject: retrieve size of array I wonder why it is not possible to retrieve the size of an array from a pointer to that array. Must not at least the delete[] operator know how big the array is? Or is this information inaccessibly burried in the OS? Thanks, Ralpe [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Victor Bazarov Guest Posted: Wed Aug 30, 2006 5:47 pm    Post subject: Re: retrieve size of array ralph wrote: Quote: I wonder why it is not possible to retrieve the size of an array from a pointer to that array. Must not at least the delete[] operator know how big the array is? Or is this information inaccessibly burried in the OS? For some reason you assume that the pointer is to a dynamically allocated array. It doesn't have to be, does it? What if it isn't? If the system is going to try to use the same mechanism as 'delete[]' to retrieve the size of the array, it should either utterly fail for static or automatic arrays or it has to use provide the means for size retrieval for those arrays just as if they were allocated with 'new[]'. It's most likely inefficient and wasteful (from the memory standpoint). If you need to know the size, pass it to your function along with the pointer. V -- Please remove capital 'A's when replying by e-mail I do not respond to top-posted replies, please don't ask [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Andrei Polushin Guest Posted: Thu Aug 31, 2006 12:47 am    Post subject: Re: retrieve size of array ralph wrote: Quote: I wonder why it is not possible to retrieve the size of an array from a pointer to that array. Must not at least the delete[] operator know how big the array is? Or is this information inaccessibly burried in the OS? See http://www.parashift.com/c++-faq-lite/freestore-mgmt.html#faq-16.14 and note that the delete[] operator needs to know how many destructors to run, but not how big the array is. If the destructor is trivial (e.g. for char[] or int[] arrays), the size of the dynamically allocated array need not to be stored. -- Andrei Polushin [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top AlbertSSj Guest Posted: Thu Aug 31, 2006 10:11 pm    Post subject: Re: retrieve size of array Quote: Must not at least the delete[] operator know how big the array is? Delete do not need to know the size of all the arrays in c/c++ code. For example this is legal: int buffer[100]; std::fill( buffer + 10, buffer + 20, 1); std::fill( buffer + 20, buffer + 30, 3); but then, wich is the size of the array (buffer + 10)? 90? not 10? neither The lenght can be only be defined in context where you can't place the array head everywhere. Slicing arrays in C is trivial (at least in a read only case), but as a consequnce know the size of an array is not. [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Ron Natalie Guest Posted: Fri Sep 01, 2006 12:20 am    Post subject: Re: retrieve size of array Andrei Polushin wrote: Quote: ralph wrote: I wonder why it is not possible to retrieve the size of an array from a pointer to that array. Must not at least the delete[] operator know how big the array is? Or is this information inaccessibly burried in the OS? See http://www.parashift.com/c++-faq-lite/freestore-mgmt.html#faq-16.14 and note that the delete[] operator needs to know how many destructors to run, but not how big the array is. If the destructor is trivial (e.g. for char[] or int[] arrays), the size of the dynamically allocated array need not to be stored. The size of the allocated block (in bytes most likely) must be stored somewhere. What isn't required is that the memory allocators make that information available. [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Andrei Polushin Guest Posted: Fri Sep 01, 2006 5:46 am    Post subject: Re: retrieve size of array Ron Natalie wrote: Quote: Andrei Polushin wrote: ralph wrote: I wonder why it is not possible to retrieve the size of an array from a pointer to that array. Must not at least the delete[] operator know how big the array is? Or is this information inaccessibly burried in the OS? See http://www.parashift.com/c++-faq-lite/freestore-mgmt.html#faq-16.14 and note that the delete[] operator needs to know how many destructors to run, but not how big the array is. If the destructor is trivial (e.g. for char[] or int[] arrays), the size of the dynamically allocated array need not to be stored. The size of the allocated block (in bytes most likely) must be stored somewhere. What isn't required is that the memory allocators make that information available. Yes, but not so simple. Allocator almost always knows the size of the allocated block, but the allocated size may differ from the requested size. Allocator may allocate more memory than requested, due to allocator's internals. Some allocators store the size as a multiple of minimum block size (e.g. MS-DOS stores it as a multiple of 16). Some allocators (like bitmapped fit allocators) need not to store the size of the block, but they usually assume the size is a multiple of several bytes (4, 8, 16). Thus this method is not portable, though one can use MSVC-specific _msize() function or like in non-portable situations. -- Andrei Polushin [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top John Moeller Guest Posted: Fri Sep 01, 2006 5:47 pm    Post subject: Re: retrieve size of array ralph wrote: Quote: Victor Bazarov schrieb: ralph wrote: I wonder why it is not possible to retrieve the size of an array from a pointer to that array. Must not at least the delete[] operator know how big the array is? Or is this information inaccessibly burried in the OS? For some reason you assume that the pointer is to a dynamically allocated array. It doesn't have to be, does it? What if it isn't? UB. Just like applying delete[] to an automatic array. If the system is going to try to use the same mechanism as 'delete[]' to retrieve the size of the array, it should either utterly fail for static or automatic arrays That would be ok with me. delete[] doesn't work on static or automatic arrays either. or it has to use provide the means for size retrieval for those arrays just as if they were allocated with 'new[]'. It's most likely inefficient and wasteful (from the memory standpoint). If you need to know the size, pass it to your function along with the pointer. This is error prone. And unneccessary if the information is already there somewhere. It's there in a dynamically-allocated array, but the layout of said information is implementation-defined. It may be at the front of the array, but who's to say how far ahead of the memory location? It also may be in a table somewhere else. It's not even up to the implementation of the platform; it's up to the implementation of the standard library. For example, you could write code that works with the MSVC++ runtime, but breaks with cygwin's runtime. A language-level solution would have to discern auto/static memory from dynamic memory, and throw some kind of exception if it wasn't dynamic. That would add unneeded overhead to compiled code, in addition to breaking compatibility with C code (not to mention being completely useless in a general sense). The only reliable way to know the size is to pass it with the pointer, or have it be "zero-terminated". That's why every API that operates on arbitrary arrays requires one of those two things; and you're right, it is error-prone. That's one reason why the use of arrays is discouraged, among other reasons. If you want the size along with your data, you can always use an STL container, which always has size() available. A std::vector<> is an excellent choice to replace an array, since it's compatible in many respects. [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top kanze Guest Posted: Fri Sep 01, 2006 5:59 pm    Post subject: Re: retrieve size of array ralph wrote: Quote: I wonder why it is not possible to retrieve the size of an array from a pointer to that array. It is: that's what std::vector<>::size() does. (And of course, it's more frequent to pass arrays by reference than by pointer, so you would write array.size(), rather than array->size().) Quote: Must not at least the delete[] operator know how big the array is? Maybe. I can't think of any case where I'd use it, however. The restrictions as to what one can pass it are draconian. Quote: Or is this information inaccessibly burried in the OS? It's in the std::vector, just like the other information concerning arrays. If you're concerned with C style arrays: I can't think of a case where I'd ever allocated a C style array dynamically, other than with type char[] or unsigned char[]. And delete[] doesn't need any information concerning the size of the array there. -- James Kanze GABI Software Conseils en informatique orientée objet/ Beratung in objektorientierter Datenverarbeitung 9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34 [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Allan W Guest Posted: Wed Sep 06, 2006 5:47 am    Post subject: Re: retrieve size of array Quote: ralph wrote: I wonder why it is not possible to retrieve the size of an array from a pointer to that array. Must not at least the delete[] operator know how big the array is? Or is this information inaccessibly burried in the OS? As others have tried to say: The need for an "array" data structure that knows it's own size, is EXACTLY the reason that std::vector was created. And this is NOT especially error-prone, but it also doesn't have to rely on any particular implementation of memory management, and when you're done with it, you don't have to remember to delete[] it... Ron Natalie wrote: Quote: The size of the allocated block (in bytes most likely) must be stored somewhere. What isn't required is that the memory allocators make that information available. Why must the size be stored somewhere? Suppose I am trying to write a program where speed is an overriding concern... I don't want it to take 190 seconds to complete, if it's possible to complete in 189 seconds. And further suppose that I happen to know that the computer where this system will run has 1 Gigabyte of physical RAM available, but the grand total of all memory allocations is expected to be well under 45 Megabytes. Let's also suppose that on my hardware, cache currency isn't a relevant issue. I should be able to write my own version of ::operator new, malloc(), and so on... I believe that (almost?) every compiler supports this. I could decide to make ::operator delete, free(), and so on, no-op functions. This is legal, isn't it? And if I do this, is there any reason why I need to keep track of the sizes of currently-allocated memory blocks? This hypothetical example is extremely unlikely, I grant you... but it's legal, right? Real-life general-purpose systems of course don't work this way... but my point (I do have one, I'm not just trying to be difficult is that the memory allocation library is completely independant of anything that understands what either a C-style array is, or what a std::vector is. When we ask ::operator new (or malloc) for 400 bytes of storage, it has no idea if this is for 100 4-byte integers, or 4 100-byte structures, or anything else... and this is the way things SHOULD be, not just so that we can replace ::operator new with our own version, but for other reasons as well. [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Andrei Polushin Guest Posted: Wed Sep 06, 2006 5:21 pm    Post subject: Re: retrieve size of array Allan W wrote: Quote: Ron Natalie wrote: The size of the allocated block (in bytes most likely) must be stored somewhere. What isn't required is that the memory allocators make that information available. Why must the size be stored somewhere? Suppose I am trying to write a program where speed is an overriding concern... I don't want it to take 190 seconds to complete, if it's possible to complete in 189 seconds. And further suppose that I happen to know that the computer where this system will run has 1 Gigabyte of physical RAM available, but the grand total of all memory allocations is expected to be well under 45 Megabytes. Let's also suppose that on my hardware, cache currency isn't a relevant issue. I should be able to write my own version of ::operator new, malloc(), and so on... I believe that (almost?) every compiler supports this. I could decide to make ::operator delete, free(), and so on, no-op functions. This is legal, isn't it? And if I do this, is there any reason why I need to keep track of the sizes of currently-allocated memory blocks? This hypothetical example is extremely unlikely, I grant you... but it's legal, right? Not so hypothetical example: in CGI programming, it is even desirable not to free memory explicitly, because it will be freed by the system when the short-living process exits. The 'allocation' could be done extremely fast by incrementing pointer to the free memory, and the size of the block is completely useless and inefficient. The opposite example: many generic allocators do store the exact size of the requested block, so it might be wise to reuse it, right? In particular, if the allocator knows the size of the allocated block, why does compiler need to overallocate arrays to store the size twice? Because the compiler is constrained to new/delete interface from the allocator, which is rather weak for this case. The same issue could be identified with std::allocator - if it stores the size already, then it need not to be stored by std::vector. The implementation of std::vector::capacity() could simply query for allocator's block size and relax. Quote: Real-life general-purpose systems of course don't work this way... but my point (I do have one, I'm not just trying to be difficult is that the memory allocation library is completely independant of anything that understands what either a C-style array is, or what a std::vector is. When the "independence" is the issue, then we might look deeper and further, and notice that many public interfaces avoid dependence on std::vector, but prefer C-style arrays, because of their predictable binary representation. If you need to be binary independent from the particular STL implementation, you should use C-style arrays. This contradicts with the recommendation to use std::vector everywhere. Even more, sometimes it is recommended to depend on a particular allocator implementation, but not to depend on STL implementation. If our allocator is known to provide the size-of-block, we can treat arrays similarly to std::vector - pass them to functions and so on. To summarize that, a small auto_array_ptr as a replacement for vector: size_t memory_size(const void* p); // provided by our allocator template<class T> class auto_array_ptr { T* p; public: explicit auto_array_ptr(T* p = NULL) : p(p) { } ~auto_array_ptr() { delete[] p; } size_t size() const { return memory_size(p) / sizeof(T); } // other methods like in std::vector<> }; size_t usage_example(const int* p) // independent from vector { auto_array_ptr<int> ap(p); return ap.size(); // as convenient as vector } int main() { std::cout << usage_example(new int[168]); return 0; } -- Andrei Polushin [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Francis Glassborow Guest Posted: Wed Sep 06, 2006 5:22 pm    Post subject: Re: retrieve size of array In article <1157492178.928617.308670 (AT) m79g2000cwm (DOT) googlegroups.com>, Allan W <allan_w@my-dejanews.com> writes Quote: As others have tried to say: The need for an "array" data structure that knows it's own size, is EXACTLY the reason that std::vector was created. No vector was created because of the need for dynamic arrays. Vector does not meet all the needs of those requiring an array which is why Boost (and TR1) have an array type. Quote: And this is NOT especially error-prone, but it also doesn't have to rely on any particular implementation of memory management, and when you're done with it, you don't have to remember to delete[] it... Which assumes you are creating a dynamic array to start with and need mechanisms for changing the size after construction. Statically dimensioned arrays are fine and sometimes desirable. The problem is that they cannot be passed to functions that handle a general array unless the size is also passed. Template techniques allow arrays whose size is known at compile time but that can be passed to functions that recover the size at execution time. -- Francis Glassborow ACCU Author of 'You Can Do It!' and "You Can Program in C++" see http://www.spellen.org/youcandoit For project ideas and contributions: http://www.spellen.org/youcandoit/projects [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Victor Bazarov Guest Posted: Wed Sep 06, 2006 7:46 pm    Post subject: Re: retrieve size of array Allan W wrote: Quote: [...] I should be able to write my own version of ::operator new, malloc(), and so on... I believe that (almost?) every compiler supports this. 'malloc', no. But you can define your own allocation function for the 'new operator', and call it '::operator new', yes. Quote: I could decide to make ::operator delete, free(), and so on, no-op functions. This is legal, isn't it? And if I do this, is there any reason why I need to keep track of the sizes of currently-allocated memory blocks? The requirement could be added so that 'new operator' (not your 'operator new' or 'operator new[]' functions) stores the size of an array if the array form is used. Where and how, is immaterial. Your allocation and deallocation functions wouldn't even know the size is actually stored. Now, whether it's a good idea and why it's not been done yet, is what is worth discussing, maybe, in comp.std.c++. Quote: This hypothetical example is extremely unlikely, I grant you... but it's legal, right? Yes. It's missing the point, but it's legal. Quote: [..] V -- Please remove capital 'A's when replying by e-mail I do not respond to top-posted replies, please don't ask [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top kanze Guest Posted: Wed Sep 06, 2006 10:51 pm    Post subject: Re: retrieve size of array Andrei Polushin wrote: [...] Quote: When the "independence" is the issue, then we might look deeper and further, and notice that many public interfaces avoid dependence on std::vector, but prefer C-style arrays, because of their predictable binary representation. Most such public interfaces avoid dependence on C++ completely, and use a public interface defined in C. In practice, if std::vector changes its layout, there's a pretty good chance that vtable layout will change as well, as will name mangling, and who knows what all else. Either the compiler guarantees binary compatibility (which includes the layout of all of the standard classes), or it doesn't. Of course, even if you need to pass a C style array to a function, you can (and normally should) still use std::vector to allocate it. -- James Kanze GABI Software Conseils en informatique orientée objet/ Beratung in objektorientierter Datenverarbeitung 9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34 [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Alan McKenney Guest Posted: Wed Sep 06, 2006 10:53 pm    Post subject: Re: retrieve size of array ralph wrote: Quote: Victor Bazarov schrieb: ralph wrote: I wonder why it is not possible to retrieve the size of an array from a pointer to that array. Must not at least the delete[] operator know how big the array is? Or is this information inaccessibly burried in the OS? For some reason you assume that the pointer is to a dynamically allocated array. It doesn't have to be, does it? What if it isn't? UB. Just like applying delete[] to an automatic array. This would really limit its usefulness. The point of being able to retrieve the size is when a user calls your function with a pointer to an array, subarray, or scalar (= array of size 1) of indeterminant origin, and you want to know how far it is to the end of the array. If it would only work if the pointer pointed to the first element of a dynamically allocated array, you would still need a separate version of the function for all the other cases, where you need to explicitly pass the size. And remember to never use the "implicit size" version except when you could guarrantee that the pointer value was pointing to the first element of a dynamically allocated array. And once you figure out how to set things up so that you could use this call safely, you will most likely realize that you could just as easily remember the size somewhere. This doesn't apply to delete[], because if you're using dynamic allocation, you have to set things up to match deallocations to allocations, anyway, to avoid memory leaks. For example, each allocation in a constructor is usually paired with a deallocation in the destructor. Doing delete -- or delete[] -- on a pointer whose origins you don't really know is not something people do for very long. Operating on an array referenced through a pointer whose origins you don't really know is quite common. -- Alan McKenney [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top James Dennett Guest Posted: Thu Sep 07, 2006 11:32 pm    Post subject: Re: retrieve size of array Paolo wrote: Quote: Isn't it true that OS stores the lenght of a dynamic allocated array in a memory block before the actual data? No, for a number of reasons: (a) The OS generally isn't aware of "arrays" at all; it is responsible only for giving raw memory to the C++ runtime environment; (b) For arrays of PODs, or more generally objects with trivial destructors, the runtime doesn't need to store the length of the array; (c) Even in cases where the length is needed, there is no requirement that it be stored *before* the array in memory. -- James [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First         C++Talk.NET Forum Index -> C++ Language (Moderated) All times are GMT Goto page 1, 2  Next Page 1 of 2   Jump to: Select a forum C/C++----------------C LanguageC Language (Moderated)C++ Language (Moderated)C++ language (comp.lang.c++)C++ language, library and standards Non-English C/C++ Newsgroups----------------C++ (German)C++ (French)  You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum Powered by phpBB © 2001, 2006 phpBB Group SEO toolkit © 2004-2006 webmedic.