Jesus H. Christ

Unknown. An extension of Jesus Christ with a fantastical middle initial, perhaps derived from a reading of ΙΗΣ (IĒS) (sometimes ΙΗϹ (IĒS), using a lunate sigma), a Greek-alphabet abbreviation for Jesus (ΙΗΣΟΥΣ) as three initials in Latin letters. See "Christogram" at Wikipedia.
The earliest use of the phrase is unknown, although according to Mark Twain (1835–1910) it was already common in the region of Hannibal, Missouri, when he was a boy.