The touring side were 283 for six in their second innings at lunch, still 20 runs behind the hosts who lead the series 2-0. Ridley Jacobs was on nought with Vasbert Drakes unbeaten on four at the interval.
West Indies followed on 303 runs behind after they were dismissed for 301 in reply to South Africa's first innings of 604 for six declared.
The start of the morning session was delayed by rain, 11 overs being lost before West Indies resumed on 263 for three.
Gayle was dismissed in the fifth over when he lost his balance cutting a delivery from fast bowler Makhaya Ntini and was caught by Neil McKenzie in the gully for 107.
Gayle was at the crease for 226 minutes, facing 159 balls and hitting 17 fours. His dismissal ended a stand of 174 for the fourth wicket with Sarwan, a West Indian record in Tests against South Africa.
Three overs later fast bowler Shaun Pollock trapped Sarwan lbw for a career best equalling 119 with a ball that cut back at the right-hander.
Sarwan batted for 350 minutes, faced 273 balls and hit 15 fours.
Dwayne Smith lasted just six balls before he had his off-stump uprooted by Ntini without scoring as West Indies slipped to 278 for six. Ntini had figures of three for 71 at the interval.
