C++Talk.NET Forum Index C++Talk.NET
C++ language newsgroups
 
Archives   FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

idea for shared_ptr

 
Post new topic   Reply to topic    C++Talk.NET Forum Index -> C++ Language (Moderated)
View previous topic :: View next topic  
Author Message
Andrei Alexandrescu (See
Guest





PostPosted: Thu Mar 24, 2005 12:01 pm    Post subject: idea for shared_ptr Reply with quote



Actually, after that whole discussion about shared_ptr, I've had an idea
on making shared_ptr support intrusive reference counting directly
without an extra memory allocation.

So, shared_ptr currently holds two pointers in-situ: one pointer to the
pointee object, and one to the bookeeping structure sp_counted_base. A
problem is that sp_counted_base (actually a subtype thereof) must be
allocated dynamically.

So I was thinking, let's imagine a class Widget wants to implement
intrusive refcounting. So what it does it inherits sp_counted_base itself:

class Widget : public sp_counted_base {
...
};

Now, when creating a shared_ptr<T>, let's say shared_ptr's constructor
looks to see whether T inherits sp_counted_base. If it doesn't, then
it's business as usual and a second dynamic allocation is performed.

But it T does inherit sp_counted_base, then shared_ptr will store the
same pointer in *both* of its fields - the pointee field and the
bookkeeping field!

Now that's pretty cool. There's no second hit for a dynamic allocation,
and nicely enough, during destruction, a simple comparison between the
two fields will tell whether or not the refcounting was intrusive or
not. So now the overhead can be reduced a lot, plus, very nicely,
shared_ptr has passed on to the pointee class the decision on whether
the refcounting is multithreaded or not.

Of course, there's a ton of details to be figured out, such as casts,
offsets, and all that jazz. But I believe it can be made to work. How
would that play with the other features of shared_ptr?


Andrei

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Back to top
Andrew Koenig
Guest





PostPosted: Thu Mar 24, 2005 2:55 pm    Post subject: Re: idea for shared_ptr Reply with quote



"Andrei Alexandrescu (See Website for Email)"
<SeeWebsiteForEmail (AT) moderncppdesign (DOT) com> wrote


Quote:
So I was thinking, let's imagine a class Widget wants to implement
intrusive refcounting. So what it does it inherits sp_counted_base itself:

class Widget : public sp_counted_base {
...
};

Wouldn't you want that to be

class Widget : public virtual sp_counted_base { ... };

?


[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Back to top
David Abrahams
Guest





PostPosted: Fri Mar 25, 2005 12:55 am    Post subject: Re: idea for shared_ptr Reply with quote



"Andrei Alexandrescu (See Website for Email)" <SeeWebsiteForEmail (AT) moderncppdesign (DOT) com> writes:

Quote:
Actually, after that whole discussion about shared_ptr, I've had an idea
on making shared_ptr support intrusive reference counting directly
without an extra memory allocation.

So, shared_ptr currently holds two pointers in-situ: one pointer to the
pointee object, and one to the bookeeping structure sp_counted_base. A
problem is that sp_counted_base (actually a subtype thereof) must be
allocated dynamically.

So I was thinking, let's imagine a class Widget wants to implement
intrusive refcounting. So what it does it inherits sp_counted_base itself:

class Widget : public sp_counted_base {
...
};

You must've missed my talk ;-)

We've been down that road with shared_ptr. It was the original
technique used to do shared_from_this. But it has really bad
implications for weak_ptrs -- the pointee isn't destroyed until the
_weak_ count goes to zero.

If you don't care about weak_ptr, you could still use this approach,
but I'm not sure it's worth the potential correctness problems -- you
can't prevent someone from using weak_ptr later. So then you could
start thinking of a design that detects this inheritance from
sp_counted_base and leaves out weak_ptr support from the interface
altogether. But then you either lose correctness during derived ->
base or shared_ptr<T> -> shared_ptr<void> conversions, or you have to
statically prevent them when the derived class is derived from
sp_counted_base. At some point you have to ask why you're trying to
squeeze this arrangement into shared_ptr instead of using a design
like intrusive_ptr.

--
Dave Abrahams
Boost Consulting
www.boost-consulting.com

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Back to top
Peter Dimov
Guest





PostPosted: Fri Mar 25, 2005 12:56 am    Post subject: Re: idea for shared_ptr Reply with quote

Andrei Alexandrescu (See Website for Email) wrote:
Quote:
Actually, after that whole discussion about shared_ptr, I've had an
idea on making shared_ptr support intrusive reference counting
directly without an extra memory allocation.

So, shared_ptr currently holds two pointers in-situ: one pointer to
the pointee object, and one to the bookeeping structure
sp_counted_base. A problem is that sp_counted_base (actually a
subtype thereof) must be allocated dynamically.

So I was thinking, let's imagine a class Widget wants to implement
intrusive refcounting. So what it does it inherits sp_counted_base
itself:

class Widget : public sp_counted_base {
...
};

Now, when creating a shared_ptr<T>, let's say shared_ptr's constructor
looks to see whether T inherits sp_counted_base. If it doesn't, then
it's business as usual and a second dynamic allocation is performed.

But it T does inherit sp_counted_base, then shared_ptr will store the
same pointer in *both* of its fields - the pointee field and the
bookkeeping field!

shared_ptr actually did that in the past, I dropped it for two reasons: one,
weak_ptr instances kept the whole Widget alive and this proved confusing;
two, sp_counted_base needs to be documented; it can no longer be an
implementation detail because users depend on it. This wasn't (and still
isn't) a good idea for an evolving component. We don't want to lock
ourselves into an inferior implementation. (A third reason was that the
"shared from this" problem was solved via other means.)



[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Back to top
Dave Harris
Guest





PostPosted: Fri Mar 25, 2005 12:57 am    Post subject: Re: idea for shared_ptr Reply with quote

[email]SeeWebsiteForEmail (AT) moderncppdesign (DOT) com[/email] (Andrei Alexandrescu (See Website
for Email)) wrote (abridged):
Quote:
So I was thinking, let's imagine a class Widget wants to implement
intrusive refcounting. So what it does it inherits sp_counted_base
itself:
[...]
How would that play with the other features of shared_ptr?

I just posted a similar idea, and also mentioned a problem: weak_ptr.
Currently weak_ptr requires sp_counted_base to out-live the managed
object. It uses it to tell whether the managed object is still there.
There is no list of weak_ptrs so no way to update them at delete time.

I suppose you could have a second control block used only by weak_ptr.
This means another pointers-worth of overhead. However, possibly the
weak_count could be moved to the new block, and it could be allocated only
if a weak_ptr is actually created, which would minimise the overhead when
weak_ptrs are not used.

I think it might be possible to avoid that pointer and allocation overhead
in the non-intrusive case, and without using virtual functions, but it's
tricky. Doing that while also keeping it thread-safe is beyond my current
competence.

-- Dave Harris, Nottingham, UK

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Back to top
Larry Evans
Guest





PostPosted: Fri Mar 25, 2005 1:04 am    Post subject: Re: idea for shared_ptr Reply with quote

On 03/24/2005 06:01 AM, Andrei Alexandrescu (See Website for Email) wrote:
[snip]
Quote:
So, shared_ptr currently holds two pointers in-situ: one pointer to the
pointee object, and one to the bookeeping structure sp_counted_base. A
problem is that sp_counted_base (actually a subtype thereof) must be
allocated dynamically.

So I was thinking, let's imagine a class Widget wants to implement
intrusive refcounting. So what it does it inherits sp_counted_base itself:

class Widget : public sp_counted_base {
...
};
Actually, since sp_counted_base has:


virtual void dispose() = 0; // nothrow

wouldn't it have to inherit one of the following:

sp_counted_impl_p <Widget>
sp_counted_impl_pd<Widget,D>

for some deleter type D? IIUC, one reason (the main reason?) for
the 2nd choice is:

- immunity from deletion in wrong heap

as mentioned by David here:

http://groups-beta.google.com/group/comp.lang.c++.moderated/msg/8fcb1efde77b85b0

But this is part of the "details to be figured out" you mention later.
Quote:

Now, when creating a shared_ptr<T>, let's say shared_ptr's constructor
looks to see whether T inherits sp_counted_base. If it doesn't, then

Seems it could work with boost's type_traits is_derived_from, or
something like that.

Quote:
it's business as usual and a second dynamic allocation is performed.

But it T does inherit sp_counted_base, then shared_ptr will store the
same pointer in *both* of its fields - the pointee field and the
bookkeeping field!

Now that's pretty cool. There's no second hit for a dynamic allocation,

Great!

[snip]

Quote:
offsets, and all that jazz. But I believe it can be made to work. How
would that play with the other features of shared_ptr?

Sounds good to me, but I'm no expert on shared_ptr.

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Back to top
arketype@myrealbox.com
Guest





PostPosted: Fri Mar 25, 2005 11:01 am    Post subject: Re: idea for shared_ptr Reply with quote

What happens here?

class Interface { };

class Concrete1 : public Interface,
public ?virtual? sp_counted_base { };

class Concrete2 : public Interface { };

shared_ptr<Interface> ptr = shared_ptr<Concrete1>(concrete1Ptr);

ptr = shared_ptr<Concrete2>(concrete2Ptr);

Or many variations thereof. I am currently using your excellent
Loki::SmartPtr to manage such a heirarchy with some custom policies
(Thanks BTW). I need to be confident of the ref counting policy in many
contexts and the smart pointer you describe subverts some of the
explicitness and static checks I enjoy with your Loki::SmartPtr.

Jeremy Jurksztowicz


[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Back to top
Jonathan Turkanis
Guest





PostPosted: Fri Mar 25, 2005 11:03 am    Post subject: Re: idea for shared_ptr Reply with quote

Andrei Alexandrescu (See Website for Email) wrote:

Quote:
Now, when creating a shared_ptr<T>, let's say shared_ptr's constructor
looks to see whether T inherits sp_counted_base. If it doesn't, then
it's business as usual and a second dynamic allocation is performed.

But it T does inherit sp_counted_base, then shared_ptr will store the
same pointer in *both* of its fields - the pointee field and the
bookkeeping field!

Now that's pretty cool. There's no second hit for a dynamic
allocation, and nicely enough, during destruction, a simple
comparison between the two fields will tell whether or not the
refcounting was intrusive or
not. So now the overhead can be reduced a lot, plus, very nicely,
shared_ptr has passed on to the pointee class the decision on whether
the refcounting is multithreaded or not.

I'm not sure whether this is nice or not. Given a type T, I'd like to know that
if I form shared_ptr<T> I get the same guarantees (incluing thread-safety of the
reference count) as I would for any other type. If T happens to derive from
sp_counted_base, which could be an implementation detail subject to change, I
might not get this guarantee. To be sure I'm getting the right behavior I might
have to write a wrapper for T or use a home-brewed smart pointer.

Jonathan




[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]


Back to top
lancediduck@nyc.rr.com
Guest





PostPosted: Sat Mar 26, 2005 9:42 am    Post subject: Re: idea for shared_ptr Reply with quote

There is an easy modification to make, in boost/detail/shared_count.hpp
(1.32)

Change line 340 from
pi_ = new sp_counted_base_impl<P, D>(p, d);
To
pi_ =sp_create_function_select<P,D>::make(p, d);

And somewhere in the file add (inside scope of boost::detail)
template <class P,class D>
boost::detail::sp_counted_base*
sp_make_counted_base(P const&p,D const&d,void*){
return new detail::sp_counted_base_impl<P,D>(p,d);
}
template <class P,class D>
struct sp_create_function_select{
static boost::detail::sp_counted_base*
make(P const&p,D const&d){
return sp_make_counted_base(p,d,static_cast<D*>(0));
}
};


Now you have a way of specializing the create function, on either T or
D, for the guts of shared_ptr, but the default is the same as it ever
was.
For example, in the case that P is already inherited from
sp_counted_base, in your file you would write
namespace boost{
namespace detail{
boost::detail::sp_counted_base*
sp_make_counted_base(sp_counted_base*p,void*,void*){
return p;
}

}}
Or something like that ( I didn't try THIS particular overload out.
That is an exercize for the reader)
OK you got me. This violates the Open/Closed Principle. If
sp_counted_base were not in "detail" then it wouldn't. But this
may be useful to hash out some ideas while we spend the next several
years figuring out what the real answer is.


[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]
Back to top
Dave Harris
Guest





PostPosted: Mon Apr 11, 2005 11:34 pm    Post subject: Re: idea for shared_ptr Reply with quote

[email]dave (AT) boost-consulting (DOT) com[/email] (David Abrahams) wrote (abridged):
Quote:
class Widget : public sp_counted_base {

We've been down that road with shared_ptr. It was the original
technique used to do shared_from_this. But it has really bad
implications for weak_ptrs -- the pointee isn't destroyed until the
_weak_ count goes to zero.

If you don't care about weak_ptr, you could still use this approach,
but I'm not sure it's worth the potential correctness problems -- you
can't prevent someone from using weak_ptr later. So then you could
start thinking of a design that detects this inheritance from
sp_counted_base and leaves out weak_ptr support from the interface
altogether. But then you either lose correctness during derived -
base or shared_ptr<T> -> shared_ptr<void> conversions, or you have to
statically prevent them when the derived class is derived from
sp_counted_base.

Isn't it possible to still support weak_ptr by allocating a separate block
for it in the intrusive case?

So you have a weak_ptr_block and a shared_ptr_block, each of which keeps a
pointer to the other as well as its weak/strong count and, for
shared_ptr_block, the support for deleting the object. At creation time
you pick one of two configurations:

Intrusive:
T inherits from shared_ptr_base so becomes the shared_ptr_block.
Its pointer to weak_ptr_block is initialised to NULL. The first
time a weak_ptr is attached, a weak_ptr_block is allocated and
shared_ptr_block updated to point to it.

When the object is destroyed, shared_ptr_block is destroyed too,
but the weak_ptr_block (if any) remains. Its shared_ptr_block
pointer now holds NULL. When its weak count reaches zero, it too
is deleted.

Non-intrusive:
Weak_ptr_block inherits from shared_ptr_block. The ptr classes
work as now, except that to access one block from the other you
have to follow the pointers and check for them being null (which
will not happen in this configuration).

All the information needed by shared_ptr is in shared_ptr_block, so the
speed of its common operations should not be affected. Shared_ptr_block
has gained an extra pointer but lost the weak count, so its size is
unchanged.

All the extra overhead is in weak_ptr support. Creating a weak_ptr from a
shared_ptr now involves an extra null check, and possibly an allocation
and some assignments the first time it happens. Extra memory overhead is
at least 2 pointers. In my opinion, weak_ptr is less performance-critical
than shared_ptr.

I believe I can see how to make this work in a naive, single-threaded
implementation. I won't bother giving code because a real solution needs
to support the multi-threaded case with all the locking (or lock-free)
optimisations, which I don't yet understand.


Quote:
At some point you have to ask why you're trying to squeeze this
arrangement into shared_ptr instead of using a design like
intrusive_ptr.

It'd be nice if we had a single pointer type which could be used for both
intrusive and non-intrusive objects. Whether a particular object supports
an intrusive count should be an implementation detail which a user of
shared_ptr<T> should not need to know about. Indeed, the smart pointer
should continue to work for incomplete types.

-- Dave Harris, Nottingham, UK.

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

Back to top
Roshan Naik
Guest





PostPosted: Wed May 04, 2005 10:37 am    Post subject: Re: idea for shared_ptr Reply with quote

Quote:

class Widget : public sp_counted_base {
...
};

Wouldn't you want that to be

class Widget : public virtual sp_counted_base { ... };

?

Why public ?
Would public inherritance be the "best" candidate here...considering liskov
substitution principle ?
I mean this seems to be an implementation detail (or is it really?) and not
necessarily intended to express "is-a".

Dont want to be pedantic here, but just curious if the choice for public
inherritance is only because it might be technically difficult (if possible)
to check for private/protected inherritance from an arbitrary type.

-Roshan


[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]


Back to top
Display posts from previous:   
Post new topic   Reply to topic    C++Talk.NET Forum Index -> C++ Language (Moderated) All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2006 phpBB Group
SEO toolkit © 2004-2006 webmedic.