Does it cost anything to do a rollover?

Your 401(k) provider may charge a nominal fee to initiate the transfer (typically ~$50) but you generally won’t pay any taxes or penalties, unless you decide to do a Roth conversion.