For audio, you must always provide a zero bandwidth alternate setting for when the device is not in use:
Whenever an AudioStreaming interface requires an isochronous data
endpoint, it shall at least provide the default Alternate Setting
(Alternate Setting 0) with zero bandwidth requirements (no isochronous
data endpoint defined) and an additional Alternate Setting that
contains the actual isochronous data endpoint.
From UAC 3.0
Same for video:
All devices that transfer isochronous video data must incorporate a
zero-bandwidth alternate setting for each VideoStreaming interface
that has an isochronous video endpoint, and it must be the default
alternate setting (alternate setting zero). A device offers to the
Host software the option to temporarily relinquish USB bandwidth by
switching to this alternate setting. The zero-bandwidth alternate
setting does not contain a VideoStreaming isochronous data endpoint
descriptor.
From UVC 1.5