Way back on Solaris I once managed to accidentally name a file backspace. Somehow I managed to finger-flub and hit "mv filename <crtl-h> <enter>" or something like that. Had to use a GUI interface to delete that file.
Not at a Unix machine right, now, but this might work:
rm \BKSP where the BKSP represents the backspace key. If it worked, it did because the backslash escapes the usual meaning of the backspace key, which is to erase the preceding character.
Also, related (though may not work for above case), to delete other files with funny characters in the name (like a leading dash):
rm -- -filename
where the -- means (end of the options), so even a leading dash after -- is not treated it as an option.
Best to use the -i option too, with such cases, for interactive confirmation of the remove.