I was looking at this discussion and started wondering "What Would Dan Savage Say?" Many of you may know him as an advice columnist for The Stranger in Seattle, syndicated nationwide, with podcasts, public appearances, an outspoken gay advocate, and a leader behind the "It Gets Better" campaign to help and encourage LGBT youth.
Anyhow, Dan is very sex-positive (e.g., in favor of sex) and is sought out for advice by all sorts of people whether they are coupled or single, gay, straight, bi, trans, or however they identify.
So I Googled for what he might have said on the subject and found these. Note that Savage is concerned about the ethical context of the commitment to your wife and whether your are in fact a monogamous couple (many are not) and whether you are being honest with yourself and with her:
Savage Love - Cheating Pieces - Columns - The Stranger
Savage Love - Head Fake - Columns - The Stranger
As I interpret his advice, it would be OK to have sex with men but you need to be safe. In this context, you're not only dealing with your own health but also your wife's.
In addition, Dan would probably suggest that you talk with your wife and be honest about your needs. I know that's hard for those in a traditional straight marriage. Perhaps that would lead to a situation where you could include your fantasies in your shared fantasies and sex life. Or perhaps she would give you permission to play with a man (stranger things happen every day). Or perhaps she would react negatively to the entire idea and it could lead to heartache.
Maybe it's presumptuous for me as a gay man in an open relationship and not married to advise you. On the other hand, you're asking what to do and as Editor of this site I see all sorts of situations. In my own life, my father had an acknowledged ongoing relationship with a woman who was not his wife (i.e., he lived part-time with my mother and part-time with his girlfriend). She was the widow of his best friend. It was "all very European," as we used to say in those days long ago. They stopped seeing each other much later when my father was elderly and had dementia, but his girlfriend came to his funeral. My point is that there's many ways to arrange your life, but you'll probably feel better and be able to live with your decisions if you can find a way, however difficult, to be open about them with the people who matter most in your life.