Where there is love, Usher will BE THERE -- or so he promised earlier this week at an L.A. bar ...  during a pretty incredible karaoke rendition of the Jackson 5's "I'll Be There" ... and it was all caught on tape.

The song requires some pretty impressive falsetto -- but Usher hit ALL the notes early Tuesday morning at Rock & Reilly's on the Sunset Strip ... and the crowd loved every second of it, singing along the whole time.

Sources tell TMZ, Usher had been hanging out upstairs in a private area with friends throughout the night -- but came down around 1am to show the bar how karaoke is REALLY done.

(It's a pretty ironic song choice -- considering Usher was NOT there ... at his stepson's funeral two weeks ago ... though the singer never explained why.)

Nevertheless -- tough act to follow.