std::ios::binary? - C++ language (comp.lang.c++)

C++Talk.NET
C++ language newsgroups

Archives

FAQ

Memberlist

Usergroups

Profile

std::ios::binary?
Goto page 1, 2, 3, 4, 5, 6, 7 Next

C++Talk.NET Forum Index -> C++ language (comp.lang.c++)

View previous topic :: View next topic

Author

Message

Steven T. Hatton
Guest

Posted: Thu Jul 28, 2005 7:01 am Post subject: std::ios::binary?

§27.4.2.1.4 Type ios_base::openmode Says this about the std::ios::binary openmode flag: binary: perform input and output in binary mode (as opposed to text mode) And that is basically _all_ it says about it. What the heck does the binary flag mean? -- If our hypothesis is about anything and not about some one or more particular things, then our deductions constitute mathematics. Thus mathematics may be defined as the subject in which we never know what we are talking about, nor whether what we are saying is true.-Bertrand Russell

Alf P. Steinbach
Guest

Posted: Thu Jul 28, 2005 7:25 am Post subject: Re: std::ios::binary?

* Steven T. Hatton:

Quote:

§27.4.2.1.4 Type ios_base::openmode
Says this about the std::ios::binary openmode flag:
*binary*: perform input and output in binary mode (as opposed to text mode)

And that is basically _all_ it says about it. What the heck does the binary
flag mean?

Basically it means that the i/o functions should not do any translation
to/from external representation of the data: they should behave as they
really should have behaved by default, but unfortunately do not.

For example, a Windows text file has each line terminated by carriage return
+ linefeed, as opposed to just linefeed in Unix, and in text mode 'n' is
translated accordingly for output, and these sequences are translated to
'n' on input -- in practice 'r' is carriage return and 'n' is linefeed.

Also, for example, in a Windows text file Ctrl Z denotes end-of-file.
That's useful for including a short descriptive text snippet at the start of
a binary file, but I suspect it was originally a misunderstanding of the
Unix shell command to send the current line immediately (which for an empty
line means zero bytes, which in Unix indicates end-of-file). So in text
mode in Windows, a Ctrl Z might be translated to end-of-file on input.

Interestingly the C++ iostream utilities are so extremely badly designed
that you can't make a simple copy-standard-input-to-standard-output-exactly
program using only the standard C++ library, on systems where this is
meaningful but text translation occurs in text mode.

Of course, the religious C++'ers maintain that that shouldn't be possible
anyway because you can't do it on, say, a mobile phone, where C++ could be
used for something, but then they forget that i/o is there for a reason.

--
A: Because it messes up the order in which people normally read text.
Q: Why is it such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?

Gianni Mariani
Guest

Posted: Thu Jul 28, 2005 7:25 am Post subject: Re: std::ios::binary?

Steven T. Hatton wrote:

Quote:

On some platforms "n" translates to "rn" when written (and the
reverse when read) to a file while on others it does not. binary mode
will make no such translation.

Steven T. Hatton
Guest

Posted: Thu Jul 28, 2005 7:43 am Post subject: Re: std::ios::binary?

Gianni Mariani wrote:

Quote:

Steven T. Hatton wrote:
§27.4.2.1.4 Type ios_base::openmode
Says this about the std::ios::binary openmode flag:
*binary*: perform input and output in binary mode (as opposed to text
mode)

And that is basically _all_ it says about it. What the heck does the
binary flag mean?

On some platforms "n" translates to "rn" when written (and the
reverse when read) to a file while on others it does not. binary mode
will make no such translation.

Is that per the Standard, or "implementation imposed"? I know that what you
describe is what I typically think of when I think of binary I/O. For
example, in ancient times we used to have to explicitly tell ftp to use
binary mode transfer.

--
If our hypothesis is about anything and not about some one or more
particular things, then our deductions constitute mathematics. Thus
mathematics may be defined as the subject in which we never know what we
are talking about, nor whether what we are saying is true.-Bertrand Russell

Steven T. Hatton
Guest

Posted: Thu Jul 28, 2005 10:24 am Post subject: Re: std::ios::binary?

Alf P. Steinbach wrote:

Quote:

Yes. I am aware of DOS spiders. ^M is what it looks like in Emacs, and it
is never a nice thing to have in a tarball. dos2unix is a great tool.

Quote:

Also, for example, in a Windows text file Ctrl Z denotes end-of-file.
That's useful for including a short descriptive text snippet at the start
of a binary file, but I suspect it was originally a misunderstanding of
the Unix shell command to send the current line immediately (which for an
empty
line means zero bytes, which in Unix indicates end-of-file). So in text
mode in Windows, a Ctrl Z might be translated to end-of-file on input.

Interestingly the C++ iostream utilities are so extremely badly designed
that you can't make a simple
copy-standard-input-to-standard-output-exactly program using only the
standard C++ library, on systems where this is meaningful but text
translation occurs in text mode.

I'm now wondering if I really understood. If I read "characters" from a
std::istream, it goes into an error state when it hits an EOF. That's why
stuff like this works (when it works)

std::vector<float_pair> positions
(istream_iterator<float_pair> (file),
(istream_iterator<float_pair> ()));

I don't believe binary files are terminated by a special character, but I
could be wrong (again).

Take this example:
####################################
Thu Jul 28 06:03:39:> cat main.cpp
#include <fstream>
#include <vector>
#include <iterator>
#include <iostream>
#include <sstream>

using namespace std;

main(int argc, char* argv[]){
if(argc<2) { cerr << "give me a file name" << endl; return -1; }

ifstream file (argv[1],ios::binary);

if(!file) { cerr << "couldn't open the file:"<< argv[1] << endl; return
-1; }

std::vector
copy(istream_iterator<unsigned char>(file)
, istream_iterator<unsigned char>()
, back_inserter(data));

cout<<"read "<
file.clear();
file.seekg(0,ios::beg);
ostringstream oss;
oss << file.rdbuf();
cout<<"read "<
}

Thu Jul 28 06:10:21:> g++ -obinio main.cpp

Thu Jul 28 06:13:04:> ./binio binio
read 22075bytes of data
read 22470bytes of data

##########################

Notice the second output is larger than the first.

Quote:

I suspect there are "political" reasons things turned out the way they did.
I really don't know how much of a performance hit it would be if certain
platforms had to do some extra endian shuffling. I do believe the lack of
real binary I/O in the Standard Library is an unexpected inconvenience.
Stroustrup bluntly states that binary I/O is beyond the scope of C++
Standard, and beyond the scope of TC++PL(SE). §21.2.1

Here's an interesting observation:
compiled with gcc 3.3.5
-rwxr-xr-x 1 hattons users 40830 2005-07-28 06:22 binio-3.3.5
compiled with gcc 4.0.1
-rwxr-xr-x 1 hattons users 22470 2005-07-28 06:23 binio-4.0.1

And 4.0.1 produces (much) faster code as well.
--
If our hypothesis is about anything and not about some one or more
particular things, then our deductions constitute mathematics. Thus
mathematics may be defined as the subject in which we never know what we
are talking about, nor whether what we are saying is true.-Bertrand Russell

Alf P. Steinbach
Guest

Posted: Thu Jul 28, 2005 10:52 am Post subject: Re: std::ios::binary?

* Steven T. Hatton:

Quote:

* Alf P. Steinbach:

[snip]
For example, a Windows text file has each line terminated by carriage
return + linefeed, as opposed to just linefeed in Unix, and in text mode
'n' is translated accordingly for output, and these sequences are
translated to
'n' on input -- in practice 'r' is carriage return and 'n' is
linefeed.

Yes. I am aware of DOS spiders. ^M is what it looks like in Emacs,

That's because a carriage return is ASCII 13, Ctrl M.

It would be different using EBCDIC, I imagine.

;-)

[snip]

Quote:

Interestingly the C++ iostream utilities are so extremely badly designed
that you can't make a simple
copy-standard-input-to-standard-output-exactly program using only the
standard C++ library, on systems where this is meaningful but text
translation occurs in text mode.

I'm now wondering if I really understood. If I read "characters" from a
std::istream, it goes into an error state when it hits an EOF. That's why
stuff like this works (when it works)

std::vector<float_pair> positions
(istream_iterator<float_pair> (file),
(istream_iterator<float_pair> ()));

I don't believe binary files are terminated by a special character, but I
could be wrong (again).

Not in Unix, and not in Windows. However a binary file can contain any byte
values, and those that look like end-of-line markers will be translated in
text mode, and the first that looks like an end-of-file marker may be
translated. All depending on the implementation.

[snip]

Quote:

ifstream file (argv[1],ios::binary);

if(!file) { cerr << "couldn't open the file:"<< argv[1] << endl; return
-1; }

std::vector
copy(istream_iterator<unsigned char>(file)
, istream_iterator<unsigned char>()
, back_inserter(data));

cout<<"read "<
file.clear();
file.seekg(0,ios::beg);
ostringstream oss;
[snip]

Quote:

##########################

Notice the second output is larger than the first.

That's probably because the ostringstream does some text mode shenanigans;
although I haven't checked.

Quote:

Of course, the religious C++'ers maintain that that shouldn't be possible
anyway because you can't do it on, say, a mobile phone, where C++ could be
used for something, but then they forget that i/o is there for a reason.

I suspect there are "political" reasons things turned out the way they did.
I really don't know how much of a performance hit it would be if certain
platforms had to do some extra endian shuffling.

? The _default_ is translation. That is, the default is the overhead &
performance hit (+ other much more evil effects) you're mentioning.

Quote:

I do believe the lack of
real binary I/O in the Standard Library is an unexpected inconvenience.
Stroustrup bluntly states that binary I/O is beyond the scope of C++
Standard, and beyond the scope of TC++PL(SE). §21.2.1

There is no §21.2.1.

--
A: Because it messes up the order in which people normally read text.
Q: Why is it such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?

Steven T. Hatton
Guest

Posted: Thu Jul 28, 2005 11:11 am Post subject: Re: std::ios::binary?

Alf P. Steinbach wrote:

Quote:

* Steven T. Hatton:

I don't believe binary files are terminated by a special character, but I
could be wrong (again).

Not in Unix, and not in Windows. However a binary file can contain any
byte values, and those that look like end-of-line markers will be
translated in text mode, and the first that looks like an end-of-file
marker may be
translated. All depending on the implementation.

And they call it a "standard"? :/

Quote:

[snip]
ifstream file (argv[1],ios::binary);

if(!file) { cerr << "couldn't open the file:"<< argv[1] << endl; return
-1; }

std::vector
copy(istream_iterator<unsigned char>(file)
, istream_iterator<unsigned char>()
, back_inserter(data));

cout<<"read "<
file.clear();
file.seekg(0,ios::beg);
ostringstream oss;
[snip]

##########################

Notice the second output is larger than the first.

That's probably because the ostringstream does some text mode shenanigans;
although I haven't checked.

Sorry, I forgot one important point.
$ls -l binio-3.3.5
-rwxr-xr-x 1 hattons users 40830 2005-07-28 06:22 binio-3.3.5

###### note the file size above, and the second output value below:

$./binio-3.3.5 binio-3.3.5
read 40034bytes of data
read 40830bytes of data

As I understand things, when I do `file.rdbuf() >> oss' I am getting a raw
stream. That, too, may be implementation defined for all I know.

Quote:

I suspect there are "political" reasons things turned out the way they
did. I really don't know how much of a performance hit it would be if
certain platforms had to do some extra endian shuffling.

? The _default_ is translation. That is, the default is the overhead &
performance hit (+ other much more evil effects) you're mentioning.

So is it the case that ios::binary may still not produce real binary
streams? That is, the stream could still act in some ways like a text
stream, e.g., eof?

Quote:

--
If our hypothesis is about anything and not about some one or more
particular things, then our deductions constitute mathematics. Thus
mathematics may be defined as the subject in which we never know what we
are talking about, nor whether what we are saying is true.-Bertrand Russell

P.J. Plauger
Guest

Posted: Thu Jul 28, 2005 11:45 am Post subject: Re: std::ios::binary?

"Steven T. Hatton" <chattengau (AT) germania (DOT) sup> wrote

Quote:

§27.4.2.1.4 Type ios_base::openmode
Says this about the std::ios::binary openmode flag:
*binary*: perform input and output in binary mode (as opposed to text
mode)

And that is basically _all_ it says about it. What the heck does the
binary
flag mean?

"Binary" and "text" are both terms of art from the C Standard, which
is included by reference in the C++ Standard. Binary I/O is byte
transparent, with the possible exception of padding NUL bytes.
Text I/O endeavors to translate between internal newline-delimited
text lines and however the system commonly represents text outside
the program.

P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com

P.J. Plauger
Guest

Posted: Thu Jul 28, 2005 11:49 am Post subject: Re: std::ios::binary?

"Alf P. Steinbach" <alfps (AT) start (DOT) no> wrote

Quote:

* Steven T. Hatton:
§27.4.2.1.4 Type ios_base::openmode
Says this about the std::ios::binary openmode flag:
*binary*: perform input and output in binary mode (as opposed to text
mode)

And that is basically _all_ it says about it. What the heck does the
binary
flag mean?

Basically it means that the i/o functions should not do any translation
to/from external representation of the data: they should behave as they
really should have behaved by default, but unfortunately do not.

For example, a Windows text file has each line terminated by carriage
return
+ linefeed, as opposed to just linefeed in Unix, and in text mode 'n' is
translated accordingly for output, and these sequences are translated to
'n' on input -- in practice 'r' is carriage return and 'n' is
linefeed.

Also, for example, in a Windows text file Ctrl Z denotes end-of-file.
That's useful for including a short descriptive text snippet at the start
of
a binary file, but I suspect it was originally a misunderstanding of the
Unix shell command to send the current line immediately (which for an
empty
line means zero bytes, which in Unix indicates end-of-file).

No, the usage originated in early systems that couldn't describe the
length of a file to the nearest byte. A CTL-Z delimited the logical
end of text, so your program didn't read trailing garbage.

Quote:

So in text
mode in Windows, a Ctrl Z might be translated to end-of-file on input.

Interestingly the C++ iostream utilities are so extremely badly designed
that you can't make a simple
copy-standard-input-to-standard-output-exactly
program using only the standard C++ library, on systems where this is
meaningful but text translation occurs in text mode.

Well, yes you can. Open files in binary mode and use read/write.

Quote:

Nonsense.

P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com

P.J. Plauger
Guest

Posted: Thu Jul 28, 2005 11:51 am Post subject: Re: std::ios::binary?

"Steven T. Hatton" <chattengau (AT) germania (DOT) sup> wrote

Quote:

Gianni Mariani wrote:

Steven T. Hatton wrote:
§27.4.2.1.4 Type ios_base::openmode
Says this about the std::ios::binary openmode flag:
*binary*: perform input and output in binary mode (as opposed to text
mode)

And that is basically _all_ it says about it. What the heck does the
binary flag mean?

On some platforms "n" translates to "rn" when written (and the
reverse when read) to a file while on others it does not. binary mode
will make no such translation.

Is that per the Standard, or "implementation imposed"?

The C and C++ Standards both require that text mode I/O do whatever
is necessary to convert between the universal internal form of text
streams and whatever the execution environment requires instead.

Quote:

I know that what you
describe is what I typically think of when I think of binary I/O. For
example, in ancient times we used to have to explicitly tell ftp to use
binary mode transfer.

And you still do, sometimes, if your FTP utility can't make an
intelligent guess.

P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com

P.J. Plauger
Guest

Posted: Thu Jul 28, 2005 12:02 pm Post subject: Re: std::ios::binary?

"Steven T. Hatton" <chattengau (AT) germania (DOT) sup> wrote

Quote:

Correct. But the I/O subsystem on every OS has *some* way to tell
you when you run out of input characters.

Quote:

...
Of course, the religious C++'ers maintain that that shouldn't be possible
anyway because you can't do it on, say, a mobile phone, where C++ could
be
used for something, but then they forget that i/o is there for a reason.

I suspect there are "political" reasons things turned out the way they
did.

Nonsense. In the early 1970s, Unix pioneered the notion of a universal
format for text streams, by pushing any needed mappings out to the
device drivers. That text stream model became an integral part of C,
which first evolved under Unix. In the late 1970s and early 1980s,
Whitesmiths, Ltd. ported C to several dozen operating systems. We
elaborated the text/binary I/O model as a way of preserving both the
universal text stream format and the transparent text stream, as
needed. All that technology was captured in the C Standard in the
mid 1980s. It was then incorporated by reference in the C++ Standard
in the mid 1990s. It's there because it works.

Quote:

I really don't know how much of a performance hit it would be if certain
platforms had to do some extra endian shuffling. I do believe the lack of
real binary I/O in the Standard Library is an unexpected inconvenience.

Might be, if there were such a lack.

Quote:

Stroustrup bluntly states that binary I/O is beyond the scope of C++
Standard, and beyond the scope of TC++PL(SE). §21.2.1

Stroustrup is not nearly as familiar with the Standard C++ library
as he is with the language he invented.

P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com

P.J. Plauger
Guest

Posted: Thu Jul 28, 2005 12:08 pm Post subject: Re: std::ios::binary?

"Steven T. Hatton" <chattengau (AT) germania (DOT) sup> wrote

Quote:

Alf P. Steinbach wrote:

* Steven T. Hatton:

I don't believe binary files are terminated by a special character, but
I
could be wrong (again).

Not in Unix, and not in Windows. However a binary file can contain any
byte values, and those that look like end-of-line markers will be
translated in text mode, and the first that looks like an end-of-file
marker may be
translated. All depending on the implementation.

And they call it a "standard"? :/

Yes they do. The C Standard describes how to impose order over a
diverse range of operating systems. You can describe practically
every car made today as having a steering wheel, an accelerator,
and a brake -- if you want to emphasize what's standard about
them. Or you can discuss at great length the different kinds of
linkages and braking systems -- if you want to emphasize how
they differ. Depends on your "political" goal, I suppose.

P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com

Alf P. Steinbach
Guest

Posted: Thu Jul 28, 2005 12:11 pm Post subject: Re: std::ios::binary?

* P.J. Plauger:

Quote:

That should be only a few lines; could you please present the code?

Quote:

See above. ;-)

--
A: Because it messes up the order in which people normally read text.
Q: Why is it such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?

P.J. Plauger
Guest

Posted: Thu Jul 28, 2005 12:24 pm Post subject: Re: std::ios::binary?

"Alf P. Steinbach" <alfps (AT) start (DOT) no> wrote

Quote:

* P.J. Plauger:

Interestingly the C++ iostream utilities are so extremely badly
designed
that you can't make a simple
copy-standard-input-to-standard-output-exactly
program using only the standard C++ library, on systems where this is
meaningful but text translation occurs in text mode.

Well, yes you can. Open files in binary mode and use read/write.

That should be only a few lines; could you please present the code?

#include <fstream>

int main(int argc, char **argv)
{ // copy a file
if (2 < argc)
{ // copy argv[1] to argv[2] transparently
std::ifstream ifs(argv[1],
std::ios_base::in | std::ios_base::binary);
std::ofstream ofs(argv[2],
std::ios_base::out | std::ios_base::binary);

ofs << ifs.rdbuf();
}
return (0);
}

(I was wrong about needing read and write.)

P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com

Alf P. Steinbach
Guest

Posted: Thu Jul 28, 2005 12:46 pm Post subject: Re: std::ios::binary?

* P.J. Plauger:

Quote:

* Alf P. Steinbach
* P.J. Plauger:

Interestingly the C++ iostream utilities are so extremely badly
designed that you can't make a simple copy-standard-input-to-
standard-output-exactly program using only the standard C++
library, on systems where this is meaningful but text translation
occurs in text mode.

Well, yes you can. Open files in binary mode and use read/write.

That should be only a few lines; could you please present the code?

#include
int main(int argc, char **argv)
{ // copy a file
if (2 < argc)
{ // copy argv[1] to argv[2] transparently
std::ifstream ifs(argv[1],
std::ios_base::in | std::ios_base::binary);
std::ofstream ofs(argv[2],
std::ios_base::out | std::ios_base::binary);

ofs << ifs.rdbuf();
}
return (0);
}

(I was wrong about needing read and write.)

Thanks for the code.

Since I (naturally) don't use iostreams much -- and in fact it's been some
time since I did C++ development -- I learned a new idiom.

And you weren't wrong about read and write: it can be done that way.

What you were wrong about:

The above doesn't copy standard input to standard output. ;-)

Cheers,

- Alf

--
A: Because it messes up the order in which people normally read text.
Q: Why is it such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?

Display posts from previous:

	C++Talk.NET Forum Index -> C++ language (comp.lang.c++)	All times are GMT Goto page 1, 2, 3, 4, 5, 6, 7 Next
Page 1 of 7

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum