It depends on one's definition of "silly". I have been asked if I was the Doctor's "other daughter" because of my verbal piffle, but my mind is usually living on another plane of existence, so a lot of things come out as being silly and irrelevant to the reality I'm occupying.
I try to be quieter about it around strangers and while employed, as it tends to scare people into thinking that sanity is optional to function. (The psych evaluations say I am sane, which surprised me.)
Otherwise, friends, classmates, and family know that the caustic commentary, random trivia, and weirdness will fly at any moment.