“He.” Nebuchadnezzar. The “he” is preserved in the Septuagint, Syriac, and Latin texts, the change from “he” to “I” was apparently a copyist’s error that now shows up in the accepted Hebrew text. The “he” continues in the rest of the verse in the Hebrew text.
“the houses of the gods.” That is, the temples. A victor would burn the temples and carry the gods away captive to show that their own gods were superior to the gods of the conquered nation.
“and he will burn them or carry them away captive.” This is referring to the gods of Egypt. Some were wood and would be burned, while valuable gods made of gold, silver, etc., would be carried away captive. The burning of the temples is stated in Jeremiah 43:12. The belief in the gods, even the gods of other nations, was strong, and so the idols were not considered booty, but captives.
“he will clean the land of Egypt as a shepherd cleans his garment of vermin.” This translation follows definitions preferred in the HALOT Hebrew-English lexicon (cp. HCSB; ESV; NAB; NET; NLT; NRSV; RSV).