You should try to make them only as visible as possible, but as stated by Mike above, this causes problems with UserControls and using the VS Designer with those controls on forms or other UserControls.
So as a general rule, keep all classes and UserControls that you aren't adding using the Designer only as visible as they need to be. But if you are creating a UserControl that you want to use in the Designer (even if that's within the same assembly), you will need to make sure that the UserControl class, its default constructor, and any properties and events, are made public for the designer to work with it.
I had a problem recently where the designer would keep removing the this.myControl = new MyControl() line from the InitializeComponent() method because the UserControl MyControl was marked as internal along with its constructor.
It's really a bug I think because even if they are marked as internal they still show up in the Toolbox to add in the Designer, either Microsoft needs to only show public controls with public constructors, or they need to make it work with internal controls as well.