Or HaChaim on Shemos 5:3

English: ויאמרו אלוקי העברים ..פן יפגענו "The G'd of Israel…. lest He punish us." The last words were spoken by the Israelites. Even though our sages in Shemot Rabbah 5,14 told us that the elders melted away on the way to the first confrontation with Pharaoh, the Israelites expressed fear of being punished by G'd. The elders were punished for not accompanying Moshe and Aharon. Perhaps the entire nation went to see Pharaoh and told him it would be in Pharaoh's own interest to excuse the Jewish people from work for three days and to take the chance that they would return; otherwise, if they would be punished by their G'd, Pharaoh would lose their labour altogether if they were to die of a pestilence. Perhaps they even implied that such a pestilence would also pose a problem for Pharaoh himself who might become infected. They used the well known method of describing such a plague as afflicting them rather than Pharaoh, as they did not want to threaten Pharaoh outright.