Are you changing the color? If not I would not remove the panels. I forgot to mention it, however you did AWESOME by getting the sanding blocks. That is why people have ripples in their paint, because they don't use a block. Try to avoid using your fingers as much as possible.

You will also want newspaper and masking tape. I would mask off a panel at a time and work on one or two that way one can be drying as you work on the other. Also, that way you don't ahve to resand your own overspray.