Not in my experience! Not in my garden! The above photograph is of two small purple Violas (click to enlarge), with a variegated Sage on the right, a new Pineapple Sage at the top (almost killed by the early April frost, but coming back nicely), and a Canna "fan" near it. Cannas, I was told when I first started gardening, only grow at trailer parks and abandoned gas stations. Not so! I only have one, but my daughter who lives two towns over has a bunch of them in her garden. All are beautiful.
A few tiny Violas, sprinkled throughout my landscape last winter are now blooming their little hearts out. This bright yellow one, with a smaller purple one to its right, just below the clump of Thyme, is a perfect example. The red specks (top of picture) are Alternanthera 'Gail's Choice' -- cuttings from last year, kept in water to root during the winter, just added to the landscape. They will turn a deep purple/rust when summer arrives and, I hope, grow as tall as their "parents" did last year.