My biggest problem is that the arms on safety glasses don't extend back far enough. Because I'm already wearing glasses, the safety glasses have to sit farther down the bridge of my nose, and not where they should sit. The arms aren't designed to have them sit so far forward. This makes them unstable, and they tend to slide down. With a properly designed bridge and/or longer arms to accommodate the adjusted bridge position, they would work just fine.
The model to which I provided a link has extendable/adjustable temple pieces. As you may have noticed when we met, I have a head the size of a small planet, and they work for me okay. If you get the dark version to use as sunglasses, no one can see your other glasses under them, which minimizes the dweebosity.
Given that, I think a prescription insert with "regular" safety glasses works better.