It's hard to say for sure, but I believe you're right in this case - the tape marker appears to pass underneath the headshell, and there's a pronounced "pop" sound as it does so. I believe the tape is bumping the stylus back into the previous groove, thus creating an endless loop. It's worth mentioning that the artist in this video was using that "loop" as a sound design tool, where the sound of the pop was part of the composition. That is not always desirable in looping, but it's a consequence of how he's done it here.
It is possible to cut a groove in a record such that the stylus returns back to the same point every revolution. Many records do this at the very end, so record players that do not automatically raise the tonearm (such as the one in the video) will not be damaged - they can just spin indefinitely. However, I'm not aware of many records that have done this anywhere except at the end.
As a practical matter, many cases of live "vinyl" looping are actually achieved with effects or software. You can sample a phrase playing on a vinyl record and immediately loop it, or you can use timecoded control vinyl to control audio playback in software, with attendant features such as looping and other effects. Both of these techniques are in common use today by DJs, although the latter is more common than the former.