Unlike some folks, I don't look down on the term "sci-fi" but it's true that it just doesn't sound like it also means fantasy, and I write both. And with all due respect to Robert J. Sawyer (who blogged in favor of having separate shelving for the two "genres"), I do think there are some works that can't be assigned either label (I wonder where he would put Silverberg's Majipoor books?).
As for the abbreviation SF, it could be said to stand for either science fiction or speculative fiction, but to most of the population of the U.S., it just means San Francisco.
My only problem with abbreviating speculative fiction is that when folks write it as one word without a hyphen (specfic) I tend to read it as "specific." Hence the new name of my blog!