However, Celtic managed to claw their way back into the game, most notably when Lerato Lamola and Dumisani Zuma got on the ball.
Lamola was presented with Celtic's best opportunity midway through the opening half, but the winger's effort was blocked by Eric Mathoho.
Both sides, however, showed attacking intent in the opening 45 minutes, but were let by their passing in the final third.
Celtic should have opened the scoring in the 53rd minute but Zuma managed to head over from six-yards out, after a mix-up between Brilliant Khuzwayo and Siboniso Gaxa.
The home side continued to pile on the pressure and went close in the 60th minute, as Lamola's deflected effort forced Khuzwayo into a fine save.
Chiefs struggled to get their passing game going in the second half but nearly went ahead in the 79th minute, as Eric Mathoho's header was cleared off the line.
Baxter brought on the likes of Siphelele Mthembu and David Zulu in the hope of snatching a win, but the two forwards failed to get going in the latter stages, and they were forced to settle for a point.
Here's how they lined up:
Bloemfontein Celtic: Tignyemb, Fransman, Ndengane, Mzava, Motshegwa, Morena, Bilankulu, Zuma, Nyatama, Ritchie, Lamola
Kaizer Chiefs: Khuzwayo, Gaxa, Mashamaite, Mathoho, Masilela, Katsande, Letsholonyane, Tshabalala, Parker, Masango, Maluleka