The phrase "losing my religion" is an expression from the southern region of the United States that means "losing one's temper or civility" or "feeling frustrated and desperate." Stipe told The New York Times the song was about romantic expression.
​⁠@MichaelStipeActual