C++Talk.NET C++ language newsgroups   Archives   FAQ   Search   Memberlist   Usergroups   Register   Profile   Log in to check your private messages   Log in  Endian Functions Goto page 1, 2, 3, 4  Next          C++Talk.NET Forum Index -> C++ language, library and standards View previous topic :: View next topic   Author Message Goalie_Ca Guest Posted: Sun Jul 02, 2006 6:53 pm    Post subject: Endian Functions I have done some searching and it appears that there isn't any standard way to convert numbers from big endian to little endian and determine if it is a big endian or little endian machine. There appears to be a version for short and int in various network libraries. People appear to invent their own every time for every data type from short to double. Some are high performance and some are naive. Some machines like itanium are actually capable of running natively in either but must be compiled with a specific flag. Is there perhaps a reason why it is not in the standard libraries or in any of the technical reports? --- [ comp.std.c++ is moderated. To submit articles, try just posting with ] [ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ] [ --- Please see the FAQ before posting. --- ] [ FAQ: http://www.comeaucomputing.com/csc/faq.html ] Back to top Frederick Gotham Guest Posted: Mon Jul 03, 2006 4:42 am    Post subject: Re: Endian Functions Goalie_Ca posted: Quote: I have done some searching and it appears that there isn't any standard way to convert numbers from big endian to little endian and determine if it is a big endian or little endian machine. I recently posted fully-portable code for determining the endianness of a machine. Here's the google link: http://groups.google.ie/group/comp.std.c++/msg/320642c7b4a21366?hl=en& If you want fully-portable code to convert between endiannesses, then post to comp.lang.c++ and I'll help you out. -- Frederick Gotham --- [ comp.std.c++ is moderated. To submit articles, try just posting with ] [ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ] [ --- Please see the FAQ before posting. --- ] [ FAQ: http://www.comeaucomputing.com/csc/faq.html ] Back to top Steve Clamage Guest Posted: Mon Jul 03, 2006 4:44 am    Post subject: Re: Endian Functions Goalie_Ca wrote: Quote: I have done some searching and it appears that there isn't any standard way to convert numbers from big endian to little endian and determine if it is a big endian or little endian machine. There appears to be a version for short and int in various network libraries. People appear to invent their own every time for every data type from short to double. Some are high performance and some are naive. Some machines like itanium are actually capable of running natively in either but must be compiled with a specific flag. Is there perhaps a reason why it is not in the standard libraries or in any of the technical reports? The data transfer problem is seldom limited to endianness. The sizes of basic types might differ, and other aspects of data format -- especially for floating-point numbers, and the padding of structs or arrays. There is a standard Unix facility, also available in Linux, called XDR (eXternal Data Representation). It allows transfer of data between different systems. Run "man xdr" for a description. Briefly, the sender marshalls the data values into a standard linear interchange format, and the receiver un-marshalls the data into its internal format. The problem of data transfer is not limited to one programming language, so it seems more appropriate for it to be standardized by something like the Single Unix Standard. --- Steve Clamage, stephen.clamage (AT) sun (DOT) com --- [ comp.std.c++ is moderated. To submit articles, try just posting with ] [ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ] [ --- Please see the FAQ before posting. --- ] [ FAQ: http://www.comeaucomputing.com/csc/faq.html ] Back to top Goalie_Ca Guest Posted: Mon Jul 03, 2006 9:10 am    Post subject: Re: Endian Functions Steve Clamage wrote: Quote: Goalie_Ca wrote: I have done some searching and it appears that there isn't any standard way to convert numbers from big endian to little endian and determine if it is a big endian or little endian machine. There appears to be a version for short and int in various network libraries. People appear to invent their own every time for every data type from short to double. Some are high performance and some are naive. Some machines like itanium are actually capable of running natively in either but must be compiled with a specific flag. Is there perhaps a reason why it is not in the standard libraries or in any of the technical reports? The data transfer problem is seldom limited to endianness. The sizes of basic types might differ, and other aspects of data format -- especially for floating-point numbers, and the padding of structs or arrays. There is a standard Unix facility, also available in Linux, called XDR (eXternal Data Representation). It allows transfer of data between different systems. Run "man xdr" for a description. Briefly, the sender marshalls the data values into a standard linear interchange format, and the receiver un-marshalls the data into its internal format. The problem of data transfer is not limited to one programming language, so it seems more appropriate for it to be standardized by something like the Single Unix Standard. --- Steve Clamage, stephen.clamage (AT) sun (DOT) com --- [ comp.std.c++ is moderated. To submit articles, try just posting with ] [ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ] [ --- Please see the FAQ before posting. --- ] [ FAQ: http://www.comeaucomputing.com/csc/faq.html ] What about the simple case where you need to read/mmap a large binary file. Perhaps this file was downloaded from a server or retrieved off of a network drive. Some binary files have the endianess in the spec itself and users are required to convert depending on the machine. Many libraries, including vtk [1], reivent the wheel to solve the problem. I know I have a library of my own handy but it is not high-performance. Having a standard library could possibly make use of machine instructions for speed. I suppose if c++ would like a stardard networking class, like on the wishlist [2], then endianess features would be required as well. [1] http://www.vtk.org/doc/nightly/html/classvtkByteSwap.html [2] http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1901.html --- [ comp.std.c++ is moderated. To submit articles, try just posting with ] [ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ] [ --- Please see the FAQ before posting. --- ] [ FAQ: http://www.comeaucomputing.com/csc/faq.html ] Back to top Carl Barron Guest Posted: Mon Jul 03, 2006 8:21 pm    Post subject: Re: Endian Functions Goalie_Ca <goalieca (AT) gmail (DOT) com> wrote: Quote: I have done some searching and it appears that there isn't any standard way to convert numbers from big endian to little endian and determine if it is a big endian or little endian machine. There appears to be a version for short and int in various network libraries. People appear to invent their own every time for every data type from short to double. Some are high performance and some are naive. Some machines like itanium are actually capable of running natively in either but must be compiled with a specific flag. Is there perhaps a reason why it is not in the standard libraries or in any of the technical reports? Caring about the byte order of integeral types meens the source is for/from an external device since otherwise the order is unimportant as long as it is fixed. If from some other source in the wrong order then A use text any integer of size N bytes can be sent as a text string in Hex with M *k --- [ comp.std.c++ is moderated. To submit articles, try just posting with ] [ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ] [ --- Please see the FAQ before posting. --- ] [ FAQ: http://www.comeaucomputing.com/csc/faq.html ] Back to top John Nagle Guest Posted: Mon Jul 03, 2006 8:27 pm    Post subject: Re: Endian Functions Goalie_Ca wrote: Quote: I have done some searching and it appears that there isn't any standard way to convert numbers from big endian to little endian and determine if it is a big endian or little endian machine. There appears to be a version for short and int in various network libraries. People appear to invent their own every time for every data type from short to double. Some are high performance and some are naive. Some machines like itanium are actually capable of running natively in either but must be compiled with a specific flag. Is there perhaps a reason why it is not in the standard libraries or in any of the technical reports? It's in the Single Unix Specification, Version 2. See http://www.opengroup.org/onlinepubs/007908799/xns/htonl.html for "arpa/inet.h". Whether those primitives belong in the language specification is an interesting issue. John Nagle --- [ comp.std.c++ is moderated. To submit articles, try just posting with ] [ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ] [ --- Please see the FAQ before posting. --- ] [ FAQ: http://www.comeaucomputing.com/csc/faq.html ] Back to top kanze Guest Posted: Tue Jul 04, 2006 6:20 pm    Post subject: Re: Endian Functions Steve Clamage wrote: Quote: There is a standard Unix facility, also available in Linux, called XDR (eXternal Data Representation). It allows transfer of data between different systems. XDR is, I believe, also an Internet standard. As such, I expect you can find some support for it on any machine which you can connect to the Internet. How much, on the other hand, might depend, as it isn't the most widely used standard:-). Note that xdr is a representation layer protocol, used by higher layers, like RPC. Quote: Run "man xdr" for a description. Briefly, the sender marshalls the data values into a standard linear interchange format, and the receiver un-marshalls the data into its internal format. The problem of data transfer is not limited to one programming language, so it seems more appropriate for it to be standardized by something like the Single Unix Standard. Also, of course, it isn't a universal protocol. Having XDR support in the language (or elsewhere) doesn't help if the other side is speaking BER encoded ASN.1. Perhaps the biggest argument against standardizing such things is the fact that there are so many to choose from. (Of course, XDR has the advantage of being one of the simplest, and a lot of protocols use some subset of it, officially as such, or because what they specify happens to coincide with XDR, at least for the common types.) -- James Kanze GABI Software Conseils en informatique orientée objet/ Beratung in objektorientierter Datenverarbeitung 9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34 --- [ comp.std.c++ is moderated. To submit articles, try just posting with ] [ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ] [ --- Please see the FAQ before posting. --- ] [ FAQ: http://www.comeaucomputing.com/csc/faq.html ] Back to top Bob Bell Guest Posted: Wed Jul 05, 2006 10:49 pm    Post subject: Re: Endian Functions Frederick Gotham wrote: Quote: Goalie_Ca posted: I have done some searching and it appears that there isn't any standard way to convert numbers from big endian to little endian and determine if it is a big endian or little endian machine. I recently posted fully-portable code for determining the endianness of a machine. Here's the google link: http://groups.google.ie/group/comp.std.c++/msg/320642c7b4a21366?hl=en& The way you use reinterpret_cast is undefined behavior; the code is not portable. Bob --- [ comp.std.c++ is moderated. To submit articles, try just posting with ] [ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ] [ --- Please see the FAQ before posting. --- ] [ FAQ: http://www.comeaucomputing.com/csc/faq.html ] Back to top Frederick Gotham Guest Posted: Thu Jul 06, 2006 6:32 am    Post subject: Re: Endian Functions Bob Bell posted: Quote: The way you use reinterpret_cast is undefined behavior; the code is not portable. It is indeed portable, because any object's address can reliably be stored in a char*. But if you'd like to use old-style casts, then go ahead: char const *p = (char const*)&guinea_pig; -- Frederick Gotham --- [ comp.std.c++ is moderated. To submit articles, try just posting with ] [ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ] [ --- Please see the FAQ before posting. --- ] [ FAQ: http://www.comeaucomputing.com/csc/faq.html ] Back to top Bob Bell Guest Posted: Thu Jul 06, 2006 3:28 pm    Post subject: Re: Endian Functions Frederick Gotham wrote: Quote: Bob Bell posted: The way you use reinterpret_cast is undefined behavior; the code is not portable. It is indeed portable, because any object's address can reliably be stored in a char*. But if you'd like to use old-style casts, then go ahead: char const *p = (char const*)&guinea_pig; I was talking about the fact that after reinterpret_cast'ing the pointer, you then dereference it. Bob --- [ comp.std.c++ is moderated. To submit articles, try just posting with ] [ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ] [ --- Please see the FAQ before posting. --- ] [ FAQ: http://www.comeaucomputing.com/csc/faq.html ] Back to top Frederick Gotham Guest Posted: Thu Jul 06, 2006 8:24 pm    Post subject: Re: Endian Functions Bob Bell posted: Quote: I was talking about the fact that after reinterpret_cast'ing the pointer, you then dereference it. A char has no trap representations, and so it is safe to access the bytes of any object in memory as if it were a char, signed char or unsigned char: #include <iostream> #include <string> template<class T> void PrintObjectBytes( T const &obj ) { unsigned char const *p = (unsigned char const *)&obj; unsigned char const * const p_over = (unsigned char const *)(&obj + 1); do std::cout << (unsigned)*p++; while(p != p_over); } int main() { std::string obj; PrintObjectBytes(obj); std::cout << '\n'; unsigned char array[] = {1,2,3,4,5,6,7,8,9}; PrintObjectBytes(array); } -- Frederick Gotham --- [ comp.std.c++ is moderated. To submit articles, try just posting with ] [ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ] [ --- Please see the FAQ before posting. --- ] [ FAQ: http://www.comeaucomputing.com/csc/faq.html ] Back to top Bob Bell Guest Posted: Thu Jul 06, 2006 9:27 pm    Post subject: Re: Endian Functions Frederick Gotham wrote: Quote: Bob Bell posted: I was talking about the fact that after reinterpret_cast'ing the pointer, you then dereference it. A char has no trap representations, and so it is safe to access the bytes of any object in memory as if it were a char, signed char or unsigned char: I'm glad to hear that that's the case on your platform of choice. Nevertheless, reinterpret_cast'ing a pointer to a different type of pointer and then dereferencing it is undefined behavior. See section 5.2.10 in the standard. Bob --- [ comp.std.c++ is moderated. To submit articles, try just posting with ] [ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ] [ --- Please see the FAQ before posting. --- ] [ FAQ: http://www.comeaucomputing.com/csc/faq.html ] Back to top Alberto Ganesh Barbati Guest Posted: Thu Jul 06, 2006 11:25 pm    Post subject: Re: Endian Functions Frederick Gotham ha scritto: Quote: Bob Bell posted: I was talking about the fact that after reinterpret_cast'ing the pointer, you then dereference it. A char has no trap representations, and so it is safe to access the bytes of any object in memory as if it were a char, signed char or unsigned char: Could you please quote the clause in the C++ standard that guarantees that? I could not find it. The best I could find is this one, which is quite close but still doesn't say that you can directly dereference reinterpret_cast<char*>(ptr): 3.9/2: For any object (other than a base-class subobject) of POD type T, whether or not the object holds a valid value of type T, the underlying bytes (1.7) making up the object can be copied into an array of char or unsigned char. So, unless I'm missing something, in order to be fully-portable you have to *copy* the object in a char array before inspecting it. Notice that doing so does not require a reinterpret_cast. Ganesh PS: C-style casts are of no help, as they are defined in terms of const_cast/static_cast/reinterpret_cast (5.4/5). --- [ comp.std.c++ is moderated. To submit articles, try just posting with ] [ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ] [ --- Please see the FAQ before posting. --- ] [ FAQ: http://www.comeaucomputing.com/csc/faq.html ] Back to top Guest Posted: Fri Jul 07, 2006 2:23 am    Post subject: Re: Endian Functions Alberto Ganesh Barbati wrote: Quote: Frederick Gotham ha scritto: A char has no trap representations, and so it is safe to access the bytes of any object in memory as if it were a char, signed char or unsigned char: Could you please quote the clause in the C++ standard that guarantees that? 3.9.1/1, which says, "For character types, all bits of the object representation participate in the value representation." In addition, 3.10/15 (7th bullet) allows access to the value of an object of any type through an lvalue of type char or unsigned char. --- [ comp.std.c++ is moderated. To submit articles, try just posting with ] [ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ] [ --- Please see the FAQ before posting. --- ] [ FAQ: http://www.comeaucomputing.com/csc/faq.html ] Back to top Guest Posted: Fri Jul 07, 2006 2:28 am    Post subject: Re: Endian Functions Bob Bell wrote: Quote: I'm glad to hear that that's the case on your platform of choice. Nevertheless, reinterpret_cast'ing a pointer to a different type of pointer and then dereferencing it is undefined behavior. See section 5.2.10 in the standard. I did, and 5.2.10/10 says exactly the opposite. In particular, it says: That is, a reference cast reinterpret_cast<T&>(x) has the same effect as the conversion *reinterpret_cast<T*>(&x) with the built-in & and * operators. The result is an lvalue that refers to the same object as the source lvalue, but with a different type. If reinterpret_cast<T&>(x) has a well-defined effect, as described here, and this is the "same effect" as reinterpret_cast<T*>(&x), I don't see why we should think that *reinterpret_cast<T*>(&x) yields undefined behavior. --- [ comp.std.c++ is moderated. To submit articles, try just posting with ] [ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ] [ --- Please see the FAQ before posting. --- ] [ FAQ: http://www.comeaucomputing.com/csc/faq.html ] Back to top Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First         C++Talk.NET Forum Index -> C++ language, library and standards All times are GMT Goto page 1, 2, 3, 4  Next Page 1 of 4   Jump to: Select a forum C/C++----------------C LanguageC Language (Moderated)C++ Language (Moderated)C++ language (comp.lang.c++)C++ language, library and standards Non-English C/C++ Newsgroups----------------C++ (German)C++ (French)  You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum Powered by phpBB © 2001, 2006 phpBB Group SEO toolkit © 2004-2006 webmedic.