"We've got a selection panel of four but in all seriousness, there about 20 million out there who are on the selection panel," Nielsen told local media on Saturday. "That's what is great about our game. Everyone has their opinion.
"While we haven't had some of the results we would have liked, there's no doubt in my mind, or anybody's mind inside the group that Ricky is the right man to lead the team."
Ponting has been under fire in the wake of Australia's 2-0 test series loss to India [ Images ], a result which saw the side slump to fifth in the test rankings.
Former test bowler Geoff Lawson has suggested the 35-year-old should step down to concentrate on his batting and let vice captain Michael Clarke [ Images ] take the reins, while Shane Warne [ Images ] publicly criticised Ponting's tactics in Twitter posts on the final day of Australia's thumping seven-wicket defeat in Bangalore this week.
Nielsen also defended embattled spinner Nathan Hauritz [ Images ], who endured a poor series was carted around the ground by Indian bats on the final day in Bangalore.
"Don't lay all the blame or all the ideas on Nathan Hauritz," he said. "We didn't bowl or bat as well as we could have for the whole series."