It would be hard to get by in the San Francisco area of Panama City, if you just speak English. You might be able to survive, because a lot of local people in Panama have at least some sort of idea of English, and I would say that a lot of the population of Panama in the more upper-class neighborhoods know a little bit more English than the average Panamanian.
If you were planning to move to Panama, I would highly recommend that you learn at least some basic Spanish. If you are coming to Panama as a tourist, you should at least know how to speak English, but you could also come to Panama and learn Spanish here because we have some good places in Panama to learn Spanish. We have several English / Spanish learning centers throughout Panama City, for example the Kaplan Centers, which are very popular in Panama.