He was Mr. Conductor on the US version of Shining Time Theatre and the narrator (and other voices) of the US version of Thomas the Tank Engine. Both were somewhat recent roles (in the 90s). He was doing a lot of voice work around this time.
The only thing close to scandal was in 1972-1973, when he was arrested for the "Seven Words" routine and then the FCC came down on a radio station for broadcasting it.
one of the Romper Room hosts was fired for having an abortion.
A few of the early Disney stars of the late 50s/early 1960s were fired when it became too public they were homosexual.
Maybe we are desensitized, but there's no way anyone with George Carlin's stand-up routines would be allowed to host childrens programming 25 or 30 years ago.