The show was well-known for having its' own "commercial segments," rather than a traditional commercial break. During those breaks, Superman and Batman (with Robin) appeared in their own Filmation cartoons, featuring their voices from The Batman/Superman Hour.
The segments showed the superheroes in situations where they provide educational information to the viewer. The Superman video taught children about the letter S. And the Batman videos taught children about walking across the street using a crosswalk, which Joker was seen not doing, and the other video taught children the concept of "up, through and around."
According to Muppet Wiki, there was another Superman video - this time teaching children about the letter D, which featured the hero holding a door-like shield, and rescued Lois Lane from an ape's hands, and exclaimed "D for Door!". That was originally aired during the Friday episode of the fifth week of the series, and was rarely shown in subsequent seasons.