Moving agents to a new task server?

I installed a second task server in my environment with the intent of moving all of the agents off the NS server. I understand the Task needs to remain installed on the NS, so how can I ensure that all my clients talk to the server?