The usefulness of application logging - C++ Language (Moderated)

C++Talk.NET
C++ language newsgroups

Archives

FAQ

Memberlist

Usergroups

Profile

The usefulness of application logging

C++Talk.NET Forum Index -> C++ Language (Moderated)

View previous topic :: View next topic

Author

Message

Guest

Posted: Thu Jul 13, 2006 3:51 am Post subject: The usefulness of application logging

I use logging as a diagnostic trace aid during development. But I leave the logging code in when I deploy to production. I have come across another way or working. This other way is to not bother logging anything until there is a problem, then add logging to shed light on the cause. The logging calls are removed once the problem has been found and fixed. The reason given for this way of working is that logging adds clutter to the code and it is not necessary when the code contains no bugs. The reason I work the way I do is that the programmers that come after me may not know that I always develop bug-free code <grin> so they may need light shed when investigating any issues where my code gets called. Also when logging is done by most modules it helps to show the overall flow of control and intermediate calculation results. I wonder what use people make of logging generally. Perhaps people can share their experiences and opinions here. Surely there is more than just these two ways of doing it. I am not particularly interested in which logger is used to do the job. There are a number of commonly available loggers for C++. Also I think there is nothing wrong in developing a home-grown logger if, say log4cxx is found to be not good enough in a particular situation. Regards, Andrew Marlow [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ]

Greg Herlihy
Guest

Posted: Fri Jul 14, 2006 2:07 am Post subject: Re: The usefulness of application logging

apm35 (AT) student (DOT) open.ac.uk wrote:

Quote:

I use logging as a diagnostic trace aid during development. But I leave
the logging code in when I deploy to production. I have come across
another way or working. This other way is to not bother logging
anything until there is a problem, then add logging to shed light on
the cause. The logging calls are removed once the problem has been
found and fixed. The reason given for this way of working is that
logging adds clutter to the code and it is not necessary when the code
contains no bugs. The reason I work the way I do is that the
programmers that come after me may not know that I always develop
bug-free code <grin> so they may need light shed when investigating any
issues where my code gets called. Also when logging is done by most
modules it helps to show the overall flow of control and intermediate
calculation results.

I wonder what use people make of logging generally. Perhaps people can
share their experiences and opinions here. Surely there is more than
just these two ways of doing it.

This following is not meant to be a criticism of anyone, but I have
found that inexperienced programmers are usually the ones whose code
makes heavy or systemic use of application logging as a QA technique.
Inexperienced programmers in particular often want to "see what the
program is doing" and thereby be able to reassure themselves that the
program is working as intended. And while application logging can be
used to verify a program's correct operation - more experienced
programmers know that application logging is almost always too
haphazard, error-prone and labor-intensive a techinque to be worth the
effort. Instead, experienced developers generally employ a much more
efficient method of ensuring program correctness; and for that reason
the code of experienced programmers usually contains relatively little
in the way of logging the program's execution.

There are two principal shortcomings with application logging as a QA
technique: first, adding just the "right" amount of logging to a
program is often quite difficult. Log too thoroughly and the volume of
output quickly becomes too voluminous to examine in any detail - but
log too sparingly and problems in the program's execution may not be
captured. Second, interpreting the output can be a tedious and
error-prone task and - worse - is a task that depends upon knowledge
not present in the code but known only to a particular programmer. In
fact the XP school of programming even has a name for this approach:
the "guru checks output" anti-pattern.

To find a better QA technique than application logging for ensuring a
program's correctness, it is first helpful to take a step back and
consider the "bigger picture". There are, in general, two ways to
determine whether a program is correct - the first, and for which
application logging is an example - is to verify that everything that
the program does is correct - which necessarily entails finding out
everything that the program is doing. The other way is to approach the
problem is from the other direction - that is, to make sure that the
program - whatever it is doing - is not doing anything wrong. After
all, a program that does nothing wrong has to be a correct program.

Concentrating on detecting errors turns out to have several advantages
as a QA technique over an approach that tries to detect correctness.
First, the programmer can automate the error detection by placing
asserts in the code. A failed assertion indicates a programming error.
Asserts do more than merely detect errors, they also preempt errors by
formalizing and regularizing the program's implicit operating
assumptions, particularly at the interface layer between a function and
its caller. And unlike application logging, there is little penalty for
making heavy use of asserts. And I would go so far as to say that there
are probably very few - if any - situations in which application
logging could not be more effectively and efficiently replaced with one
or more asserts.

I would point out that there are a few instances in which application
logging is useful: in particular, test code often logs a program's
state or output in order to compare against the expected state or
output. Note that logging in this situation is being used in an
automated context - and is really more akin to an assert than it is a
general execution log. So as a rule of thumb, I would state that any
type of application logging in which the expected output is either not
tested programatically for correctness - is probably not a worthwhile
QA technique.

Greg

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

kanze
Guest

Posted: Fri Jul 14, 2006 2:11 am Post subject: Re: The usefulness of application logging

apm35 (AT) student (DOT) open.ac.uk wrote:

Quote:

What if the problem isn't reproduceable? One of the most
important things I find about logging is that shows how you got
there. Even in actual production use, low level logging is
active, so that in case of a problem, I know what input
triggered it.

Even if the problem is reproduceable on the production machine,
it may depend on the environment, and not be reproduceable on
your machine. So you want to be able to turn on full logging on
the production machine, in order to find the problem.

Quote:

The reason given for this way of working is that logging adds
clutter to the code and it is not necessary when the code
contains no bugs.

By definition, the latter case doesn't exist in any non-trivial
software.

Quote:

The reason I work the way I do is that the programmers that
come after me may not know that I always develop bug-free code
grin> so they may need light shed when investigating any
issues where my code gets called. Also when logging is done by
most modules it helps to show the overall flow of control and
intermediate calculation results.

I wonder what use people make of logging generally. Perhaps
people can share their experiences and opinions here. Surely
there is more than just these two ways of doing it.

I suspect that it largely depends on the application. I
consistenly use fairly complete logging facilities, which are
active (albeit at a lower level) even in the production code. I
work on large scale servers for in house applications, though;
I'm not sure I'd follow the same policies if I were writing a
shrink wrapped game program. (I don't think I'd like it if I
bought a game, and it filled my hard disk with log files:-).)

Basically, I'd say that if you won't be able to get at the logs
from deployed programs, use classical asserts and logging which
can be compiled out. Anytime you can get at log files from
deployed code, however, it would be foolish to not take
advantage of this. Unless the profilers says otherwise, of
course:-).

--
James Kanze GABI Software
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

W. J. La Cholter
Guest

Posted: Sat Jul 15, 2006 2:06 am Post subject: Re: The usefulness of application logging

apm35 (AT) student (DOT) open.ac.uk wrote in news:1152715284.018894.264310
@m79g2000cwm.googlegroups.com:

Quote:

There are several potential audiences for logging: the developer, the
maintainer, the end user/operator, the tester.

The developer might add short-term noisy "debug" level logging to
pinpoint a problem discovered in unit test or later. Logging is a
complement to other tools such as debuggers and is especially useful on
a system that has complex interactions. For example my project at work
has 27 active multithreaded processes that have very timing-sensitive
communications. Logging makes it easier to track some problems than
using an interactive debugger.

The maintainer cannot deal with the noise and instead needs to see
specific anomalous behavior. Good filtering, such as by component or
level, is vital. Otherwise there's too much information. More valuable
than logs are design documentation, including sequence diagrams.

On my project, my colleagues and I use the logging to look for certain
things during integration test. It enables one or two people to
complete a test, rather than the whole team.

The end user/operator needs to know when things have started
successfully or when an error has occurred. This logging should be
minimal and not too technical to understand. Because the end user is
dealing with a production environment, release builds are a good way to
filter much logging.

The tester would look for separate types of logs (again, relying on
filtering). If the system is complex enough, it can be a good way to do
white box testing without having to develop as much test harness
software.

Another application is profiling. It's not good for fine-grained but
may meet your needs if you can't get an intrusive off-the-shelf package
to work.

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

James Kanze
Guest

Posted: Sun Jul 16, 2006 5:07 am Post subject: Re: The usefulness of application logging

apm35 (AT) student (DOT) open.ac.uk wrote:

Quote:

kanze wrote:
apm35 (AT) student (DOT) open.ac.uk wrote:
I use logging as a diagnostic trace aid during development.
But I leave the logging code in when I deploy to production.

Even if the problem is reproduceable on the production machine,
it may depend on the environment, and not be reproduceable on
your machine. So you want to be able to turn on full logging on
the production machine, in order to find the problem.

Basically, I'd say that if you won't be able to get at the logs
from deployed programs, use classical asserts and logging which
can be compiled out. Anytime you can get at log files from
deployed code, however, it would be foolish to not take
advantage of this. Unless the profilers says otherwise, of
course:-).

Thanks James. This is very similar to the view I hold. How
pleasant it is to find agreement.

I suspect that you'll find agreement from anyone who has worked
on large projects where there was access to the deployed
programs. In all of the projects I've worked on, I've never had
to argue for, or even suggest logging. It has always been
assumed by everyone involved that there would be some logging.
It seems to be pretty much of an established engineering
principle.

Quote:

I would add that whilst I leave the logging in the production
version there is a way to turn it off and it is off by
default. Where this can't easily be done (e.g the logger isn't
good enough) then I arrange is so that the logging is still
present in the development version (a compile-time switch
makes the logging calls no-ops for production) then I try to
find the problem in development. If it only occurs in
production where no light can be shed then, ahem, I am in
trouble.... Sad

In all of the larger projects I've been involved in, logging has
been configured by means of a multi-line file, where each line
specifies the subsystems and the logging levels concerned, and
where to log to: a file, syslog, email... There has also been a
means of triggering the reconfiguration dynamically, without
stopping the system.

Designing such a system isn't that hard, but it could be
overkill for smaller projects. All systems I've worked on,
however, have had multi-level logs, with a configurable log
level, and at least minimum logging active at all times. (But
all of the projects I've worked on have either been in house, or
for a single client with whom we had a priviledged
relationship---enough so that we could modify the configuration
files on his systems. Also, the projects have had very strict
quality requirements: contractual penalties for downtime, etc.)

--
James Kanze kanze.james (AT) neuf (DOT) fr
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France +33 (0)1 30 23 00 34

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

kanze
Guest

Posted: Wed Jul 19, 2006 4:00 am Post subject: Re: The usefulness of application logging

Greg Herlihy wrote:

Quote:

James Kanze wrote:
Greg Herlihy wrote:
apm35 (AT) student (DOT) open.ac.uk wrote:
I wonder what use people make of logging generally.
Perhaps people can share their experiences and opinions
here. Surely there is more than just these two ways of
doing it.

This following is not meant to be a criticism of anyone,
but I have found that inexperienced programmers are
usually the ones whose code makes heavy or systemic use of
application logging as a QA technique.

I've found just the contrary. It's inexperienced
programmers who don't realize just how useful it can be, and
simply suppose that it will be too expensive. Experienced
programmers generally log pretty heavily. (I know that I've
consistently increased the amount I log as I've gained
experience.)

Regardless of the amount of application logging a programmer
may perform while writing a program (which is a matter of
personal preference),

More likely a question of programming guidelines (and in some
extreme cases, legal issues).

Quote:

I would expect that the more experienced programmer would be
more likely to remove all logging code once work on the
program had completed.

Your expectations don't conform with reality. In practice, the
more experienced a programmer is, the more likely he is to leave
a maximum of logging in the deployed program. (Provided there
exists some possibility for him to access the log files later.)

Quote:

After all, the purpose of the logging code is aid development,
so once development is over, the logging code has necessarily
outlived its usefulness.

That's superficially true. But development is only over when
you can be 100% sure that there are no more errors in the code.
That is: never. And logging, as a development aid, is most
important in deployed code, where there are very few other
development aids.

Quote:

The logging code in a finished program becomes simply a
liability: potentially impairing the program's efficiency, its
stability or even its security.

You'll have to explain that one to me. I've always designed my
logging systems so that they can be compiled out if necessary,
but I've never had to compile them out---a well designed logging
system does not create significant additional overhead when it
is configured to lower levels of logging. As for stability or
security: changing anything from the development version (the
version which was tested) greatly reduces both stability and
security. If you have logging in development versions, then you
should only remove it from the production version under dire
necessity.

Quote:

Just to be clearthat this
discussion concerns only developer-added logging - that is,
logging that is not part of the program's specification. Many
programs do maintain logs of their own activities as part of
their routine, planned operation - and that is not the type of
logging being discussed here.

I'm not sure where, or even if, you can draw the line. My
logging systems typically have 9 or 10 different levels. (No
great philosophical reason; you really only need 4 or 5. 9 or
10 just happens to be what you get if you use a single digit to
specify the level.)

Quote:

In reality, the problem is not so much leaving extraneous
logging code in a finished program - the problem is really
ever adding logging code the program in the first place.
Logging code really belongs somewhere else. After all, the
purpose of logging is to document a program's state at a
particular point - and to capture that information should be
possible without having to change the program itself.

Exactly. And since we have to be able to do it in a deployed
program, what other solution do you recommend?

Quote:

By modifying the program in order to log its execution, the
program being logged is no longer identical to original
program or the program as it will ship.

Exactly. That is, IMHO, a killer argument for leaving the
logging in a deployed program.

Quote:

In order to obtain logs as accurate as possible, the logging
should be conducted as part of the program's execution
enviroment.

You can't usually control the execution environment.

Quote:

Running the program under a debugger, and configuring the
debugger to output the information of interest accomplishes
that goal quite handily.

Asking your users to run the program under a debugger isn't an
option on the programs I've worked on.

Quote:

Having the debugger generate the program's execution log
allows the programmer maximum flexiblity in obtaining the
information of interest - and to do so without ever having to
alter the program under observation. And by keeping separate
the data collection and the program's execution logic, there
is no danger of any logging code winding up in the finished
product where - at the very least - it is of no value to the
user running the program (since a user generally wants to run
a shipping program for some purpose other than to debug it).

And experienced programmer know that regardless of what they
want, the users always do end up doing some of your debugging.
At the very least, you can never be 100% sure that this isn't
the case.

--
James Kanze GABI Software
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

kanze
Guest

Posted: Wed Jul 19, 2006 4:00 am Post subject: Re: The usefulness of application logging

A Marlow wrote:

Quote:

On Sat, 15 Jul 2006 06:35:02 -0400, John Hickin wrote:

apm35 (AT) student (DOT) open.ac.uk> wrote in message
news:1152715284.018894.264310 (AT) m79g2000cwm (DOT) googlegroups.com...
I use logging as a diagnostic trace aid during development.
But I leave the logging code in when I deploy to
production.

Even if logging is left in the code you may be refused
permission to turn it on. You may need the equivalent of a
_flight recorder_, whose contents will be dumped and the
file sent to you by the customer.

I am not sure what you mean. Can you elaborate please?

Logging that is automatically turned on, and that the user
doesn't know about. If the user refuses you permission to use
logging, you call it something else:-).

--
James Kanze GABI Software
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Dejan.Mircevski@gmail.com
Guest

Posted: Wed Jul 19, 2006 4:30 am Post subject: Re: The usefulness of application logging

Greg Herlihy wrote:

Quote:

After all, the purpose of logging is to document a program's
state at a particular point - and to capture that information should be
possible without having to change the program itself. By modifying the
program in order to log its execution, the program being logged is no
longer identical to original program or the program as it will ship.

I agree with your sentiment, but practice may be lagging behind theory
here. Until aspect-oriented programming becomes standard, I don't know
of any good way to log a program's state without adding logging code to
the program.

Quote:

Running the
program under a debugger, and configuring the debugger to output the
information of interest

I have found this to be difficult in practice. GDB often keels over
when I put extensive command lists at breakpoints, and the Visual
Studio debugger didn't even have the notion of command lists last time
I looked. (These are not the only debuggers available, but they are
the most commonly used ones.)

If the problem is only reproducible by running live in production, most
customers will allow logging to help diagnose it, but no customer I
know will consider running live under a debugger.

And running the program under a debugger changes the program's behavior
even more than adding logging code does. Many tough problems that
require logging won't even show up under a debugger, for a variety of
reasons (timing, process image layout, memory capacity, etc, etc).

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

kanze
Guest

Posted: Thu Jul 20, 2006 3:59 am Post subject: Re: The usefulness of application logging

Hendrik Schober wrote:

[...]

Quote:

I wrote a logging lib that allows filtering of log
messages at compile-time and at run-time, both based on
what's the target audience of the messages and their
severity. At least VC8's optimizer is able to completely
remove log statements that are filtered at compile-time.
Run-time filtered messages are optimized so far as they
generate very little code besides an 'if' statement that
determines whether the message is to be discarded.

Presumably, the if is on a single, scalar variable.

Is this library available somewhere on the network. All of my
logging code has supported significant runtime configurability,
but the compile time option was always all or nothing. It
sounds like you've gone one step further.

--
James Kanze GABI Software
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Hendrik Schober
Guest

Posted: Thu Jul 20, 2006 9:29 pm Post subject: Re: The usefulness of application logging

kanze <kanze@gabi-soft.fr> wrote:

Quote:

Hendrik Schober wrote:

[...]
I wrote a logging lib that allows filtering of log
messages at compile-time and at run-time, both based on
what's the target audience of the messages and their
severity. At least VC8's optimizer is able to completely
remove log statements that are filtered at compile-time.
Run-time filtered messages are optimized so far as they
generate very little code besides an 'if' statement that
determines whether the message is to be discarded.

Presumably, the if is on a single, scalar variable.

if( std::vector<...>::empty() )

Quote:

Is this library available somewhere on the network. [...]

No it's not. Somone whose opinion I) value high already
tried to make me write an article about it and publish
the code, but I so far haven't got around doing it... Sad

If you want I could outline my solution. I don't think
it's all that inventive and you shouldn't have any
problems to come up with your own. (Although it heavily
relies on template-meta stuff and I remember you saying
the compilers you use aren't very good with templates.
It's a while since I regularily read here, though, so
meanwhile this might have changed.)

Schobi

--
SpamTrap (AT) gmx (DOT) de is never read
I'm Schobi at suespammers dot org

"The sarcasm is mightier than the sword."
Eric Jarvis

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

kanze
Guest

Posted: Sat Jul 22, 2006 8:23 am Post subject: Re: The usefulness of application logging

Hendrik Schober wrote:

Quote:

kanze <kanze@gabi-soft.fr> wrote:
Hendrik Schober wrote:

[...]
I wrote a logging lib that allows filtering of log
messages at compile-time and at run-time, both based on
what's the target audience of the messages and their
severity. At least VC8's optimizer is able to completely
remove log statements that are filtered at compile-time.
Run-time filtered messages are optimized so far as they
generate very little code besides an 'if' statement that
determines whether the message is to be discarded.

Presumably, the if is on a single, scalar variable.

if( std::vector<...>::empty() )

That shouldn't be too expensive. In my own implementations, it
usually comes down to something like:
if ( someStaticArray[ N ] != NULL )
where N is the log level---normally a constant.

I've still not found a good solution for this when there is a
requirement to be able to change the configuration on the fly in
a multithreaded environment. You certainly don't want a lock in
the if (although at least under Solaris, it isn't that
expensive), so you've got to find some way of suspending all
threads other than the one doing the modification, AND ensuring
that all of the other threads correctly synchronize their memory
after the suspention. The only thing I've found so far is for
the updating thread to post a request in a well known place, and
all other threads to periodically poll and suspend themselves if
the request is present. The updating thread then waits until
all threads are suspended, does the update, and then signals the
threads to continue. While this is fairly easy to do using
conditions in Posix, it results in a lot of coupling; in
particular, the updating thread must know how many other threads
there are, and the other threads must explicitly collaborate.

Quote:

I understand the problem. All too well, in fact.

In my own case, I've finally resorted to just making the code
and what documentation I've had the chance to write available
(at http://kanze.james.neuf.fr, at least for the moment), as is,
on the odd chance that it might be useful. What I'd like to do
is update at least the important parts to Boost's standards, and
move it into Boost, but if I wait until I've the time for that,
who knows when it would be available.

Quote:

It's not so much a question of my needing it myself. I've
obviously got a system that works, and since we don't turn
logging completely off in the production code, we obviously
aren't going to compile it out. But it seems like the sort of
thing that would be generally useful. For various reasons, the
logging code in all of the applications I've worked on has
contained some proprietary code which I can not make public.
And yours sounds more flexible than mine anyway.

--
James Kanze GABI Software
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

James Kanze
Guest

Posted: Mon Jul 24, 2006 5:55 am Post subject: Re: The usefulness of application logging

Yechezkel Mett wrote:

Quote:

kanze wrote:
I've still not found a good solution for this when there is a
requirement to be able to change the configuration on the fly
in a multithreaded environment. You certainly don't want a
lock in the if (although at least under Solaris, it isn't
that expensive), so you've got to find some way of suspending
all threads other than the one doing the modification, AND
ensuring that all of the other threads correctly synchronize
their memory after the suspention. The only thing I've found
so far is for the updating thread to post a request in a well
known place, and all other threads to periodically poll and
suspend themselves if the request is present. The updating
thread then waits until all threads are suspended, does the
update, and then signals the threads to continue. While this
is fairly easy to do using conditions in Posix, it results in
a lot of coupling; in particular, the updating thread must
know how many other threads there are, and the other threads
must explicitly collaborate.

The question is how timely it needs to be. Does it matter if
not all threads change configuration immediately? You could
have a thread-local variable caching the current logging state
and update it in each thread (using a lock or similar) when
convenient.

It's an idea. If thread local variables can be accessed as
quickly as other variables, it is probably a good idea. I'm not
sure that this is always the case, however. The basic code for
each log statement resolves to something along the lines of:
if ( table[ N ] != NULL ) ...
where N is a constant. So the runtime cost when logging at
level N is turned of is very, very low. If putting table into
thread local memory ups the cost here, it might not be
acceptable.

--
James Kanze kanze.james (AT) neuf (DOT) fr
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France +33 (0)1 30 23 00 34

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Hendrik Schober
Guest

Posted: Fri Jul 28, 2006 12:00 am Post subject: Re: The usefulness of application logging

kanze <kanze@gabi-soft.fr> wrote:

Quote:

Hendrik Schober wrote:
kanze <kanze@gabi-soft.fr> wrote:
[...]
I've still not found a good solution for this when there is a
requirement to be able to change the configuration on the fly in
a multithreaded environment. [...]

Ahh. No MT here yet. But this is an upcoming requirement,
so I have to deal with this sooner or later. The fact
that locking is required does give me a headache and I
have been thinking a lot about how to minimize that -- so
far to no avail.

There are two distinct issues with respect to threading. If
the logging is configured once at start-up (before threading
starts), then you can read the configuration variables without
locking, and it is relatively simple to only lock when actually
generating a log message. Given that generating a log message
will almost certainly involve a certain number of allocations
and some physical output, the extra lock seems acceptable.

Ah. One less thing to worry then.

Quote:

The
problem occurs if you want to be able to reconfigure the log
dynamically, while the threads are running, because then, you
have to somehow synchronize all accesses to the configuration
variables. And you certainly don't want a lock in your test
whether a particular level is active or not.

I see.

Quote:

Is this library available somewhere on the network. [...]

No it's not. Somone whose opinion I value high already
tried to make me write an article about it and publish
the code, but I so far haven't got around doing it... :(

I understand the problem. All too well, in fact.

In my own case, I've finally resorted to just making the
code and what documentation I've had the chance to write
available (at http://kanze.james.neuf.fr, at least for the
moment), as is,

Where? I quickly looked around, but haven't found the
logging stuff.

Sorry. I was talking in general. All of the actual logging
systems I've done have involved proprietary code, so I can't
make them available per se. [...]

You said this elsewhere, yes. Sorry, I got confused.

Quote:

[extensive description snipped]

Thanks for that explanation. In general, I don't like
to use streams for this.
For one, I have a parameter inserting algorithm that
works by parameter names (not position) which I use.
So log messages can also be translated, which is nigh
impossible to do with streams. Also, I frequently log
to Windows' 'OutputDebugStr()' API function that takes
an 'const char*' (and lets me see this in the debugger,
where clicking that log message takes me to the line
that produced it). I don't see a reason to spend time
building a stream buffer that writes into this. And
even if I did -- creating your own stream buffers
requires an amount of knowledge of streams interna
which few C++ developers have. Requiring users of the
logging lib to write their own stream buffers would
ver much limit the number of potential users.

Quote:

[...]
We compile out /some/ of it. Let me explain:
The lib allows to specify different targets for logging.
Currently, there is "debugger" (what developers want to
see during debugging), "logger" (giving an idea what the
app is currently doing), and "user" (what should be shown
the user). Also, there is several severities ("debug",
"log", "info", "warn", "error", "fatal"). Targets and
severities can be freely combined for each log message.
(Both are passed as template parameters to allow compile-
time evaluation).

That's an interesting comment. I usually use macros for
logging, in order to be able to automatically insert __FILE__
and __LINE__. Since I'm already using macros, I've not been
very motivated to consider templating it. (Also, of course,
much of it was developped before I had very good template
support.)

I, too, use those macros. The templates came into
play when I redesigned everything one or two years
ago. I do some template meta-programming, but no
macro meta-programming. I have no idea whether what
I do would be possible using macros.
A logging statement here currently looks something
like this:
TLogger::log<TgtDebugger,SevInfo>( ORIGIN, "@(event) happened at
@(time)!", ERROR_PARAM(event)+ERROR_PARAM(time) );
In this, 'TLogger' is the logger used. It's 'log()'
function takes a target and a severity as template
parameters. 'ORIGIN' is the file name and line number
macro (actually it also carries date and time of the
compilations as a simple means to identify versions).
Then follows a C string and a parameter list. The
latter uses expression templates to delay evaluation
of the parameters until it is verified that this
specific target/severity combination is actually
logged.

Quote:

Logging is done through loggers, each of which forwards
log messages to another one. There is a global logger
singleton at the root of the resulting DAG. Each logger
defines which targets/severities it will let pass and
which it will discard. (This forwarding effectively is
a compile-time operation.) If a message makes it to the
singleton it will be discarded unless there's a listener
registered for its target and severity. (The test is the
above mentioned 'if'.)

I'm not sure that I understand the role of the intermediate
loggers. Unless...

On one very large project, I did have a "configuration" table
per sub-system. So that you could configure debug for a single
subsystem, without getting swamped with messages you weren't
interested in. (I also defined a couple of "virtual" subsystems,
which represented things like e.g. messages on a socket.)

It's close to this, except that it hasn't a given
granularity, but allows you to use any granularity
you like. For example, I have a lot of loggers that
are specific to one source file only and forward to
some sub-system-specific logger. They were in use
when I created, tested, and debugged the code in
these source files. When I was done, I switched
them so that most of the logging is now compiled
out. Whenever I need to go back and change, fix, or
add something, I might turn this on again to see in
more detail what's happening.

Quote:

[...]
Typically a release build would have target "debugger"
and severity "debug" compiled out. Everything else is
left in the code.

Left in the code, but activated by means of a configuration
file, I suppose.

It's the apllication's decision how this is controlled.
Config files are one possibility. A popular one. :)

Schobi

--
SpamTrap (AT) gmx (DOT) de is never read
I'm Schobi at suespammers dot org

"The sarcasm is mightier than the sword."
Eric Jarvis

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Display posts from previous:

	C++Talk.NET Forum Index -> C++ Language (Moderated)	All times are GMT
Page 1 of 1

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum