I think it has more to do with representation of the character in visual media (comic books, television, movies). Showing the character from the side is better than showing just their feet. Also, flying sideways is visually distinct from simply jumping upward, which is important when you’re trying to show the action in the context of two or three comic frames.

Also, that’s how Superman took off, so that’s how a lot of superheroes designed after him took off.