I use my mini mill and there isn't a problem with it. I don't see it being a problem for a drill press. Not really that much pressure to press a fret in my opinion if slotted correctly and with a lead filed in.
CA and glue will size the wood if somewhat spongy and the fret will hold better and be more solid. It's good for loose frets and will prevent dead sounding frets. It will also lube the fret slot and make pressing a little easier I guess too. Whether or not you use it is up to you in the end.
I made the video because I had to press this compound radius board and didn't have the time to make or cash for a set of cauls. I had given some thought to putting a bearing on the end of a shaft and roll it over the fret as it pressed it. But what the hell, the bolt and poly took about five minutes to make and worked fine. Beats noisy hammering and seems to follow the board radius.
I was just thinking out of the box again with this one and it just happened to work out this time and thought I'd share it.