The hosts, who last won five consecutive games in December 2017, have 16 points from seven games, equalling their best start to a season last achieved in the 2000-01 campaign.
Udinese levelled in the 22nd minute thanks to an own goal from Milan Skriniar after Beto's header from a Roberto Pereyra free kick bounced off the Inter defender and caught goalkeeper Samir Handanovic by surprise.
Inter's best chance to regain the lead came in the 30th minute when defender Alessandro Bastoni tried to fire in with a superb effort inside the box but Silvestri denied him with an equally brilliant save.
Handanovic also saved a long-range low shot from Sandi Lovric around the hour mark and denied Udinese several times in the second period as Inter struggled to keep the hosts outside of their own half.
However, their resolve was eventually broken in the 84th minute when Bijol scored from a Gerard Deulofeu corner.
Substitute Arslan made sure of the points in the third minute of stoppage time with a header from a Deulofeu cross.