When I choose some forums and hit update I assume that this enables those forums for karma voting, but when the list reloads it doesn't give you any indication of which are active and which aren't.
Hmmm it should select them if karma is on. I'll go and check.
Also, if there is a way to turn off karma voting for a forum?
You answered that yourself above :) ...the forums you select on the Karma Forums page will allow voting, the others won't.