C++Talk.NET Forum Index C++Talk.NET
C++ language newsgroups
 
Archives   FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Replacing blank lines in my .txt data file
Goto page 1, 2  Next
 
Post new topic   Reply to topic    C++Talk.NET Forum Index -> C++ Language (Moderated)
View previous topic :: View next topic  
Author Message
Guest
PostPosted: Tue Aug 22, 2006 5:32 am    Post subject: Replacing blank lines in my .txt data file Reply with quote


Hi all,

How could I replace the blank lines in my .txt data file? It has many
empty lines between groups of "text". In truth, the file is not a
well-formated one, in strict sense. I would like to change blank lines
for "special caracter", for example "$". I supose this is easy to do,
but is taking my time in such a way that I'm thinking about to do
manually.

"Things that only a beginner can't see, you know"

Thanks a lot.


[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]
Back to top
Carl Barron
Guest
PostPosted: Tue Aug 22, 2006 7:21 pm    Post subject: Re: Replacing blank lines in my .txt data file Reply with quote


In article <1156193214.293686.97930 (AT) 74g2000cwt (DOT) googlegroups.com>,
<diazhenrique (AT) gmail (DOT) com> wrote:

Quote:
Hi all,

How could I replace the blank lines in my .txt data file? It has many
empty lines between groups of "text". In truth, the file is not a
well-formated one, in strict sense. I would like to change blank lines
for "special caracter", for example "$". I supose this is easy to do,
but is taking my time in such a way that I'm thinking about to do
manually.

"Things that only a beginner can't see, you know"

sed? Easiest if you have access to sed:)

sed 's/^[ \t]*$/$/p' <infile >outfile
will replace each blank line with $\n. Smile Smile
if no sed can you script your editor to do it? Smile Smile
I don't have debugged boundary corner problem safe way
to do it handy but it involves using the free function std::getline
from <string> header most likely.

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]
Back to top
kevin cline
Guest
PostPosted: Tue Aug 22, 2006 7:22 pm    Post subject: Re: Replacing blank lines in my .txt data file Reply with quote


Possibly off-topic, but tasks like this one can be done is a few
seconds with any halfway decent text editor, or with scripting
languages like Perl:

perl -pi -e 's/^$/$/' somefile.txt

diazhenrique (AT) gmail (DOT) com wrote:
Quote:
Hi all,

How could I replace the blank lines in my .txt data file? It has many
empty lines between groups of "text"....


[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]
Back to top
Ulrich Eckhardt
Guest
PostPosted: Tue Aug 22, 2006 7:23 pm    Post subject: Re: Replacing blank lines in my .txt data file Reply with quote
diazhenrique (AT) gmail (DOT) com wrote:
Quote:
How could I replace the blank lines in my .txt data file? It has many
empty lines between groups of "text". In truth, the file is not a
well-formated one, in strict sense. I would like to change blank lines
for "special caracter", for example "$". I supose this is easy to do,
but is taking my time in such a way that I'm thinking about to do
manually.

Well, the stream editor 'sed' could do what you want with a simple
invocation, but I guess you want to do it in C++. Therefore, there are a
few steps you need to take:

1. Read a file line by line. You can output the lines to std::cout then.
2. Write a file line by line. This is pretty much the same as writing to
std::cout, but you need to check that opening and writing succeeds.
3. Remove the original file and replace it with the generated one. For that,
there are no functions in (the very limited standardlibrary of) C++, so you
will have to resort to platform-dependent ones.
4. Implement a filter that replaces empty lines with whatever marker you
want.

Note that all four tasks can solved independently, which you should do.
For solving them, you will need both IOStreams and strings of the C++
standardlibrary (classes std::string, std::i[f]stream, std:Surprised[f]stream)
and related functions. In case you have problems with either step, show
some code and describe the problems you are having.

Uli




[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]
Back to top
Hrayr BABAJANYAN
Guest
PostPosted: Tue Aug 22, 2006 7:25 pm    Post subject: Re: Replacing blank lines in my .txt data file Reply with quote
Hi!
diazhenrique (AT) gmail (DOT) com wrote:
Quote:
Hi all,

How could I replace the blank lines in my .txt data file? It has many
empty lines between groups of "text". In truth, the file is not a
well-formated one, in strict sense. I would like to change blank lines
for "special caracter", for example "$". I supose this is easy to do,
but is taking my time in such a way that I'm thinking about to do
manually.

"Things that only a beginner can't see, you know"

Thanks a lot.



Have fun!!

// begin
#include <fstream>
#include <iostream>
#include <string>

bool isBalnk(const std::string& str, const std::string& blanks = "
\t\v")
{
if (str.length() == 0)
{
return (true);
}

for (std::string::const_iterator i = str.begin(); i != str.end();
++i)
{
if (blanks.find_first_of(*i) == std::string::npos)
{
return (false);
}
}

return (true);
}

void stripBlankLines(const std::string& fileName, const std::string
replace = std::string())
{
std::ifstream is(fileName.c_str());
if (!is)
{
std::cerr << "error: opening file '" + fileName + "' for reading."
<< std::endl;
return;
}

const std::string str = replace.length() == 0 ? replace : replace +
'\n';
std::string contents;
std::string line;
while (is.good() && !is.eof())
{
std::getline(is, line);
if (isBalnk(line))
{
contents += str;
}
else
{
contents += line + '\n';
}
}

is.close();

std::ofstream os(fileName.c_str());
if (!os)
{
std::cerr << "error: opening file '" + fileName + "' for writing"
<< std::endl;
return;
}

os << contents;

return;
}

int main(int argc, char* argv[])
{
if (argc == 1)
{
std::cerr << "usage: " << argv[0] << " <files>" << std::endl;
// can further add an option to specify the hardcoded "$"
return (EXIT_FAILURE);
}

while (--argc)
{
std::cout << "stripping: " << argv[argc] << " ..." << std::endl;
stripBlankLines(argv[argc], "$"); // stripBlankLines(argv[argc]);
to just remove the blank lines
}

return (EXIT_SUCCESS);
}
// end

Cheers!
--
Hrayr


[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]
Back to top
Karthick
Guest
PostPosted: Tue Aug 22, 2006 7:26 pm    Post subject: Re: Replacing blank lines in my .txt data file Reply with quote
diazhenrique (AT) gmail (DOT) com wrote:
Quote:
Hi all,

How could I replace the blank lines in my .txt data file? It has many
empty lines between groups of "text". In truth, the file is not a
well-formated one, in strict sense. I would like to change blank lines
for "special caracter", for example "$". I supose this is easy to do,
but is taking my time in such a way that I'm thinking about to do
manually.
[snip]

Are you *really* looking for a C++ solution for this problem?
Won't the following work..?
$ sed s/^$/YOUR_SPECIAL_CHAR/ your_file_name
Where a "blank line" is an empty line with no characters. If you have
spaces and/or tabs, you simply have to modify the search string.

Or do this in your editor.

- Karthick


[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]
Back to top
kanze
Guest
PostPosted: Tue Aug 22, 2006 8:50 pm    Post subject: Re: Replacing blank lines in my .txt data file Reply with quote
Michiel.Salters (AT) tomtom (DOT) com wrote:
Quote:
diazhenrique (AT) gmail (DOT) com wrote:

How could I replace the blank lines in my .txt data file?

Read in all the lines into a container, do it in-memory, then
overwrite the file?

It depends on what the goal of the program is, but I would never
do this professionally. Write to a temporary file, then
"remove" the original, and "rename" (or copy, then delete, if
the rename fails) to the orginal file.

Otherwise, any problems when writing, and you loose your data.

There's also the issue of whether the file fits in memory.

Quote:
That usually is the easiest way to manipulate text files. One
std::copy in, one std::replace() in memory and one std::copy
out. Basically, that's all you need with the correct
iterators.

Manipulating random data in memory is a lot easier than on disk,
and any time I could be sure that the file would fit, and I
needed some sort of random access as well, I'd do this. For
something as simple as he's doing however, it seems like
overkill. Just use transform and the corresponding
[io]stream_iterators. (You will have to define a Line type,
since the << operator on a string reads words, and not lines.)

--
James Kanze GABI Software
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34


[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]
Back to top
kanze
Guest
PostPosted: Tue Aug 22, 2006 8:51 pm    Post subject: Re: Replacing blank lines in my .txt data file Reply with quote
Ulrich Eckhardt wrote:
[...]
Quote:
3. Remove the original file and replace it with the generated
one. For that, there are no functions in (the very limited
standardlibrary of) C++, so you will have to resort to
platform-dependent ones.

What about std::remove and std::rename (in <cstdio>)?

For typical editing jobs, it is usual to create the "temporary"
file in the same directory as the source, appending something
like ".new" to get the name. In such cases, std::rename almost
always works.

--
James Kanze GABI Software
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34


[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]
Back to top
Ulrich Eckhardt
Guest
PostPosted: Wed Aug 23, 2006 6:28 pm    Post subject: Re: Replacing blank lines in my .txt data file Reply with quote
Note up front: you might have felt challenged with this task, but it
obviously was a homework assignment or similar job, so you shouldn't have
posted complete code here. See also the FAQ about homework assignments.

However, let's look at the code...

Hrayr BABAJANYAN wrote:
Quote:
bool isBalnk(const std::string& str,
const std::string& blanks = " \t\v")

Ignoring the typo (Balnk vs Blank), there is a function called isspace()
that tells whitespace apart from characters which could have been used
here. Also, using the right second argument it could have twisted the
meaning of this function completely around, e.g. if I call it
with "0123456789" as second argument. Then it would in fact return if the
first string is a number.

Quote:
if (str.length() == 0)
{
return (true);
}

- std::string::empty() is much more expressive than checking the length
against zero. Also I'd use size() as that's the name used in all other
containers.
- 'return' is not a function, no need for brackets.


Quote:
for (std::string::const_iterator i = str.begin();
i != str.end();
++i)
{
if (blanks.find_first_of(*i) == std::string::npos)
{
return (false);
}
}
return (true);

std::string has a find_first_not_of() function, IIRC, which does just that.
This function also returns string::npos when the searched string is empty
so even the check for an empty string can be removed. In fact the whole
function then boils down to

str.find_first_not_of(blanks)==std::string::npos


Quote:
void stripBlankLines(const std::string& fileName,
const std::string replace = std::string())

You could have used a reference for the second parameter, too.


Quote:
const std::string str =
replace.length() == 0 ? replace : replace + '\n';

Again, could have used empty() instead.

Quote:
std::string contents;
std::string line;
while (is.good() && !is.eof())
{
std::getline(is, line);
if (isBalnk(line))
{
contents += str;
}
else
{
contents += line + '\n';
}
}

Did you test this? The point is that eof() only returns true after(!) an
operation reached EOF. Your last getline() call will fail but you will
still use the result. The idiomatic way is this:

while( getline(in,line))
... // handle line

if(in.eof())
... // reached EOF
else
... // failed before reaching EOF


Quote:
return;
}

Redundant, it's a void function anyway. That's a matter of taste/purism
though.

Quote:
while (--argc)
{
std::cout << "stripping: " << argv[argc] << " ..." << std::endl;
stripBlankLines(argv[argc], "$"); // stripBlankLines(argv[argc]);
to just remove the blank lines
}

Surprising that it starts at the end... You could have used the fact that
argv is a NULL-terminated array to handle the files in order. I would also
have used a for-loop because the number is fixed - that's again a matter of
taste though.

Uli



[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]
Back to top
Jiang
Guest
PostPosted: Wed Aug 23, 2006 6:28 pm    Post subject: Re: Replacing blank lines in my .txt data file Reply with quote
diazhenrique (AT) gmail (DOT) com wrote:
Quote:
Hi all,

How could I replace the blank lines in my .txt data file? It has many
empty lines between groups of "text". In truth, the file is not a
well-formated one, in strict sense. I would like to change blank lines
for "special caracter", for example "$". I supose this is easy to do,
but is taking my time in such a way that I'm thinking about to do
manually.


If you want to use C++ for your text processing jobs, I suggest you
check boost.regex at:

http://www.boost.org/libs/regex/doc/index.html

IMHO, if you do serious text processing in your daily job, you should
consider using regular expression from now.

BTW, regex will soon be added to the standard library, so you need
not worry too much about its availability.

HTH


[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]
Back to top
kanze
Guest
PostPosted: Wed Aug 23, 2006 11:35 pm    Post subject: Re: Replacing blank lines in my .txt data file Reply with quote
Ulrich Eckhardt wrote:

[...]
Quote:
The idiomatic way is this:

while( getline(in,line))
... // handle line

if(in.eof())
... // reached EOF
else
... // failed before reaching EOF

Just a detail, but the only way getline can normally fail is if
the last line is not terminated by a '\n'. And of course, this
sets EOF as well. If you want to reliably handle an incorrectly
terminated last line:

while ( in.peek() != EOF ) {
if ( ! getline( in, line ) ) {
// missing '\n'...
// If I understand correctly, line should still
// contain the characters up to the EOF...
}
// ...
}

"Abnormally", getline can fail because the streambuf raised an
exception, or because of std::bad_alloc. Both set badbit, which
can be tested with in.bad().

--
James Kanze GABI Software
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34


[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]
Back to top
Carl Barron
Guest
PostPosted: Thu Aug 24, 2006 6:12 am    Post subject: Re: Replacing blank lines in my .txt data file Reply with quote
In article <1156351815.444658.218570 (AT) p79g2000cwp (DOT) googlegroups.com>,
kanze <kanze@gabi-soft.fr> wrote:

Quote:
Ulrich Eckhardt wrote:

[...]
The idiomatic way is this:

while( getline(in,line))
... // handle line

if(in.eof())
... // reached EOF
else
... // failed before reaching EOF

Just a detail, but the only way getline can normally fail is if
the last line is not terminated by a '\n'. And of course, this
sets EOF as well. If you want to reliably handle an incorrectly
terminated last line:


if the maximum filesize of a file on an OS is greater than
string::max_size(), then it is possible to extract string::max_size()
chars before finding a '\n' and this will set the failbit while there is
no real problem with the stream and its not at 'eof'. attempting to
store the char in the string should set bad_alloc but should be caught
in getline or the special case would not occur, and not need specif

this can be simulated with a stream whose streambuffer class always
returns '1' as the char extracted.

In these days of 'super large files' and 'networked files with main
frame capacity sources' seems like a case to reckon with.

do
{
std::getline(in,line);
if(in.eof())
{
// end line missing in stream
}
if(!n)
{
if(s.size()==s.max_size())
{
// string is really truncated at max_size() chars
// delimiter not reached
}
}
}while(in);

looks like a more safe approach...

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]
Back to top
Jorgen Grahn
Guest
PostPosted: Sat Aug 26, 2006 4:26 pm    Post subject: Re: Replacing blank lines in my .txt data file Reply with quote
On 22 Aug 2006 11:50:25 -0400, kanze <kanze@gabi-soft.fr> wrote:
Quote:
Michiel.Salters (AT) tomtom (DOT) com wrote:
diazhenrique (AT) gmail (DOT) com wrote:

How could I replace the blank lines in my .txt data file?

Read in all the lines into a container, do it in-memory, then
overwrite the file?

It depends on what the goal of the program is, but I would never
do this professionally. Write to a temporary file, then
"remove" the original, and "rename" (or copy, then delete, if
the rename fails) to the orginal file.

Or read from std::cin and write to std::cout, if the audience is Unix users
and you don't want to surprise them.

/Jorgen

--
// Jorgen Grahn <grahn@ Ph'nglui mglw'nafh Cthulhu
\X/ snipabacken.dyndns.org> R'lyeh wgah'nagl fhtagn!

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]
Back to top
James Kanze
Guest
PostPosted: Sun Aug 27, 2006 4:51 am    Post subject: Re: Replacing blank lines in my .txt data file Reply with quote
Carl Barron wrote:
Quote:
In article
1156407927.493751.25300 (AT) 75g2000cwc (DOT) googlegroups.com>, kanze
kanze@gabi-soft.fr> wrote:

Try it. Read from an istringstream initialized with "abc\n",
and then from one initialized with "abc\nxyz". Using just
getline and testing after, there is no way to distinguish
between the two. If you want to be sure, you need the peek()
before.

Well this doesn't test but prints the eof ,fail and bad
conditions in that order. So if in.eof() is true the string
is read without '\n' and [in this implementation] fail() is
false, before the do while loop finally reads and end file
condition.

Interesting. Double checking, the C++98 version of the standard
does say that encountering EOF without having seen a new line
will not cause failbit to be set; this seems a bit odd, since
it is definitly what I would consider a format error. But of
course, this doesn't help us much, at least not formally, since
eofbit can also be set if we've seen the '\n' correctly
(although in this particular case, I can't imagine it happening
in practice).

--
James Kanze kanze.james (AT) neuf (DOT) fr
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France +33 (0)1 30 23 00 34


[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]
Back to top
Francis Glassborow
Guest
PostPosted: Sun Aug 27, 2006 7:53 pm    Post subject: Re: Replacing blank lines in my .txt data file Reply with quote
In article <1156627023.419958.235470 (AT) b28g2000cwb (DOT) googlegroups.com>,
James Kanze <kanze.james (AT) neuf (DOT) fr> writes
Quote:
Interesting. Double checking, the C++98 version of the standard
does say that encountering EOF without having seen a new line
will not cause failbit to be set; this seems a bit odd, since
it is definitly what I would consider a format error. But of
course, this doesn't help us much, at least not formally, since
eofbit can also be set if we've seen the '\n' correctly
(although in this particular case, I can't imagine it happening
in practice).

Indeed, and working round the problems caused by this specification has
caused me immense problems in the past. However, I think it was
motivated by the habit of sending end-of-file (CTRL D in unixes or CTRL
Z in MSDOS/WINDOWS) from the keyboard where it is simply not possible to
send them in the expected order. (newline -- eof)


--
Francis Glassborow ACCU
Author of 'You Can Do It!' and "You Can Program in C++"
see http://www.spellen.org/youcandoit
For project ideas and contributions:
http://www.spellen.org/youcandoit/projects


[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]
Back to top
Display posts from previous:   
Post new topic   Reply to topic    C++Talk.NET Forum Index -> C++ Language (Moderated) All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2006 phpBB Group
SEO toolkit © 2004-2006 webmedic.