Modern keyboards are based on those found on typewriters, and while they have several digital-specific innovations like the delete key, they also include many quirks and symbols that date from the era when typewriters were the primary means of official communication.
Typewriters were essentially fast-paced stamp machines, putting ink directly on the paper. That means it wasn’t possible to delete or undo something the way you can on a computer. But they still had a “backspace” key — why?
Originally, the space key moved the target location forward one character, creating an empty space between words or for the typist to move to a field some distance ahead on the line without manually adjusting the carriage. While we tend to think of the space key now as adding a sort of “empty character,” back then it was also a way to move the typewriter forward one space.
When typographical errors (typos) happened, or if you hit forward space one too many times, you’d use the backspace key, which did the reverse of the space key: moving back one space. Though you could not, of course, remove an error, it was common to backspace to the beginning of the word and type x’s through it, or dashes to give a strikethrough effect.)
Computers subsequently gained arrow keys to move the cursor around the digital document, so there was no longer any need for backspace to do anything but fix errors, and so removal was integrated into its backwards movement.