Cloriviere was an officer in the French army and a counter-revolutionary implicated in an assassination plot againt Napoleon Bonaparte. He fled to Baltimore in the United States in 1803, became a priest, and ministered in Charleston, South Carolina. After involvement in a trustee controversy there, he was assigned to the Visitation Convent in Georgetown.