It’s a current painpoint. I had the same problem.
The (vomit) solution that I came up with is … touch the file (dismiss ImageMagick or whatever might get launched) then bring up the keyboard (selecting the Terminal keyboard if not already selected, and selecting the special keys (>_) if not already selected) then touch Del. Job done.
Then touch x to dismiss the Undo option (although it will disappear automatically after X seconds).
The Context menu did actually work just once when I was fumbling around but I was never able to reproduce that.