It was Brendan Rodgers's first defeat by a Scottish club since taking over Celtic in the summer of 2016 -- and it is one he is not likely to forget.
The Parkhead club, however, remain two points ahead of Aberdeen at the top of the table with a game in hand while Hearts, following a fantastic if unexpected result, are unbeaten in six.
Despite their good form, Hearts boss Craig Levein rang the changes for the lunchtime kick-off with Michael Smith, Jamie Brandon, Don Cowie, Ross Callachan, Cochrane, Milinkovic and Lafferty all restored.
Former Hibernian striker Leigh Griffiths got his first Celtic start since November as defender Mikael Lustig, Jozo Simunovic and Kieran Tierney also returned for a match which began in typically frenetic fashion and hinted at the shock to come.
Lafferty put himself about in the early stages and he and Callachan both had efforts from distance which missed the target.
At the other end, Griffiths slipped when set up by James Forrest inside the Jambos' penalty box, then fired a similar opportunity into the side-netting.
However, Hearts' sheer enthusiasm continued to rattle Celtic again and when Tierney slipped at a Dedryck Boyata pass Milinkovic took a Cowie pass, raced down the right and cut the ball back for Cochrane.
The young midfielder then drilled a wonderful shot past Hoops goalkeeper Craig Gordon from 16 yards, to the delight of three of the four Tynecastle stands.
Minutes later Gordon prevented the hosts' doubling their lead when he forced a shot from Milinkovic over the bar for a corner which came to nothing -- but Hearts soon had their second goal.
Milinkovic robbed Callum McGregor in the middle of the park and motored forward before releasing Lafferty who, from the edge of the box and looking as if he had plenty to do, angled a shot low past Gordon and in off the far post.
Rodgers brought on striker Moussa Dembele and Stuart Armstrong for Tierney and Olivier Ntcham at the start of the second half but soon found themselves three down when Milinkovic took advantage of a Simunovic slip and rounded Gordon before knocking the ball into the net.
The champions swarmed around the Hearts penalty area and Griffiths drew a fine save from Gorgie goalkeeper Jon McLaughlin with a powerful 22-yard drive.
In the 67th minute Hearts brought on another 16-year-old, Anthony McDonald, for Prince Buaben at the same time as Rodgers used his last throw of the dice by replacing midfielder McGregor with Odsonne Edouard, before Cole Stockton took over from Lafferty.
However, it was all going wrong for Celtic.
Gordon brought down Callachan inside the box following a Gorgie counter-attack -- when the Hearts player had overrun the ball -- and Milinkovic converted the spot kick for his second goal in maroon since joining the club in the summer.
There was no way back for last season's the treble winners and their proud undefeated record ended.