It's tons of fun. I've done it a few times.
As long as you're a decent swimmer, you'll be fine.
Also - listen to every option that your river guide gives you. If you end up in calm water, and he says you can hop out and swim beside the raft... do it! If he says that there's a spot where you can get out and cliff jump...do it! Just take advantage of all that the trip has to offer. Most of the river guides are good people. and will be able to answer any questions that you have.