Chasing 275 runs to win, India bowled out South Africa for 201 to pocket the penultimate match by 73 runs.
On a difficult surface, it was India's wrist spinners who once again rose to the occasion to hand India a sensational series win. South Africa were off to a good start with Aiden Markram and Hashim Amla providing the hosts a solid start.
After a 52-run opening stand, Jasprit Bumrah provided the breakthrough by removing the South Africa skipper. Hardik Pandya then snared an edge to pack off JP Duminy as India kept the pressure lid on South Africa.
Pandya then got the prized scalp of Ab de Villiers to keep India ahead.
David Miller and Amla joined hands to stabilize the chase and after a brief stand, Yuzvendra Chahal bowled a beauty to castle the southpaw.
It was then Kuldeep who stole the show as he ran through the SOuth Africa lower-order to help India take an unassilable 4-1 lead in the series.
While Kuldeep took four wickets, Pandya and Chahal shared four wickets among themselves. Bumrah took one wicket.
Earlier, opener Rohit Sharma scored his maiden century on the South African soil as India posted 274/7.
Rohit (115) who notched up his 17th ODI century hit 11 boundaries and four sixes in his 126-ball innings. It was also his second ton against South Africa, the first one came in 2015 at Kanpur when he scored 150.
The century also helped Rohit become the fourth highest century scorer in ODIs for India, following Sachin Tendulkar (49), Virat Kohli (34) and Sourav Ganguly (22).
For South Africa, Lungisani Ngidi was the pick of the bowlers returning figures of 4/51 while Kagiso Rabada picked up a wicket.
Asked to bat, openers Shikhar Dhawan (34) and Rohit got India off to a decent start before the former was removed by South African pacer Kagiso Rabada while trying to pull a good bouncer which ended straight into the arms of Andile Phehlukwayo at deep square leg.
Rohit, who struggled to score in the previous ODIs started his innings in similar fashion. However, a boundary and a hit over the fence in back-to-back overs set the momentum for the Mumbai batsman, who kept going from there.
Later, one-down batsman Virat Kohli (36) played a slow 54-ball innings. He was involved in a 105-run second wicket stand with Rohit before he was ran out by Jean-Paul Duminy.
Following the dismissal, Ajinkya Rahane (8) again fell victim to Rohit's mis-timed call and was ran out by Morne Morkel.
However, Rohit continued his flow of runs and brought up his century in the 36th over after he got a lucky escape batting on 96 being dropped by Tabraiz Shamsi.
Incoming batsman Shreyas Iyer (30) played a steady innings to keep the flow of runs going. He hit two boundaries in his 37-ball innings and was involved in a 60-run fourth wicket partnership with Rohit.
Rohit, who ended his drought of runs was cleaned up pacer Ngidi while trying to play a late cut which was easily taken by the keeper.
Incoming batsman, Hardik Pandya (0) was sent back by Ngidi in the very next delivery for a duck, which brought former Indian captain Mahendra Singh Dhoni on the crease.
Later, Iyer, who kept the momentum of runs for India also fell prey to Ngidi in a similar fashion.
Post the dismissal, Dhoni scored 13 runs while before becoming the fourth scalp to Ngidi while Bhuvneshwar Kumar (19 not out) contributed important runs in the lower order.