Secret societies were a traditional part of P.A. history until they were banned in 1949 by Headmaster Kemper.
Some still exist today:
T.U.B. (Truth, Unity, Brotherhood) places a bathtub of soda on the library steps every finals week.
M.S.A.S. (Madame Sarah Abbot's Society) wrote "MSAS" in chalk on nearly every building last spring term.