White on White
I would normally say that moderation is key here and not to go overboard, but then an exception like Sir John Soane's wonderful house comes to mind. I suppose it all depends upon balance. If you have many collections, perhaps rotate them every few weeks/months, or display them in a way that is unobtrusive, not overwhelming, for the appreciative viewer. Or, if you possess Soane's brilliant eye and unfailing attention to aesthetics, then go all out there and cover your whole house. Whatever you end up doing, your collection should express you. Every piece should be valued by you, not there just to fill a gap.
I've been thinking about this a lot lately. I know it's often been done, but I really like the idea of all-white objects. I don't have a lot of space in my apartment, but I do have three sizable windows and thus a lot of windowsill. I've been thinking about collecting an assortment of white ceramic objects - figurines, vases, etc., and putting them all in a row on a sill. I should add that my whole apartment is painted white; it's a rental and I can't change it, but it does magnify the space. They would express continuity in terms of color, but regarding form each would be unique. Remember - just because it's white doesn't mean an object has to be boring. Here are some of my favorites:
Friendly Two-Headed Dog (I always wanted to keep Cerberus for a pet...), by Melabo
FÄRM Vases, at IKEA
Large Horse Sculpture, by Jonathan Adler
Ceramic Bird Sculpture, at the Curiosity Shoppe
White Notes 3D Flower Vases at Blue Bell Bazaar
Ted Muehling's Tree Branch with Moss, at moss
Nymphenburg Porcelain Skull, at Unica Home.
Astier de Villatte Aphrodite pitcher, at Moss
Mr. and Mrs. Jones juicer set at Sprout Home
Aalto vase, at Finnstyle