The reason why you may want custom nameservers is because if you have any sub-cPanel accounts you could then point the nameservers to the ones that you are about to set up and then you will be able to control the DNS from the WHM or cPanel. Which is very convenient, it is not necessary and may not be ideal for all situations. That though is up to you to decide.
Setting up the nameservers in your WHM. If you need help logging into the WHM. Once you are logged into the WHM you will need to search in the top left for “Basic” and then click on “Basic cPanel and WHM Setup.”
Once you are on the page Basic cPanel and WHM Setup, you will need to go to the bottom and there is a section called Nameservers. It may already have the main domain for the account already there, which means that the DNS records are already created. Like shown below.
Even if those are there you will need to know which IP address they are pointing to, so click the “Assign IP Address” button and then you will see a pop saying:
Assigning IP address for ns1.domain.com... ...Done and under that you will see:
The nameserver has been assigned the IP XXX.XXX.XXX.XXX. I would write down the IP address, because when you register the nameserver with your registrar you will need to know that. Do the same thing for the second one. You can also setup brand new ones so long as they are hosted on the same server.
Now that you have it setup on the server and have the IP addresses that they are pointing to you can go to your registrar and register the name servers with them. So it turns out that cPanel Inc. has a decent list of registrars and how to register custom nameservers with them: How to Register Private Nameservers