The Go button is used for people who do not have javascript enabled. If you have javascript enabled, then the button should not appear.
If you *really* don't want it to appear no matter what, then you'll have to override the paging code and generate your own paging code. For that, you'll have to read the function header in Links::Utils::paging.