Paper books have hundreds of pages; if you don’t want to remember exactly which one you’re on, and if you don’t want to dog-ear (fold over) that page, you generally slip a page marker, or bookmark, into the spot. Of course anything flat enough will do: just as anything heavy enough can be a paperweight.
But older books, especially bibles (among the oldest, most widely printed, and largest of books) tended to come with a built-in bookmark in the form of a long ribbon attached to the binding at the top of the spine. The ribbon could be laid between the pages safely and would never be lost. Sometimes they were plain, at other times decorative, but small flourishes like a “swallowtail” were common, and the forked shape has become visual shorthand for bookmarking something. And the word “bookmark” has become digital shorthand for saving something for later.
Some icons include a page or the book itself, but the bookmark ribbon itself is almost always present. In contexts where the “document” aspect of the thing being saved is less prominent, one now finds stars or hearts as indicators of saved status.