This is a good question. It appears that Mr. Buffett does care about the size of his canvas and is averse to shrinking it, unless it cannot be justified at all by seriously hurting performance.

This view receives support in the fact that he resisted buying back Berkshire stock for a long, long time, despite repeated calls for it, and even when the shares traded closer to book value than now. At that time, I recall being puzzled by his behavior, given that he was a huge proponent of buying back stock when it trades below intrinsic value and actually pushed many companies in which he had invested to do so (including The Washington Post).

Even in recent years, many feel that he could have bought back larger amounts of Berkshire stock.

Although he did return money to the investors in the Buffett Partnership in 1969 when he could not find good good deals, it may be that, at least in some respects, he views those partners differently than Berkshire shareholders, despite statements that both groups are his "partners." For one, he knew most of those partners personally and would also invite them to his house during Christmas (per Alice Schroeder and Roger Loweinstein). However, most of Berkshire shareholders are nameless and faceless to him.

Just my 2 cents.