Elliot Murphy argues that theories of language and theories of neural computation must constrain one another. Distinguishing symbolic, hierarchical linguistic structure from sequential statistics, he uses formal tools including category theory, algebra, and statistical physics to narrow the candidate mechanisms by which sound, meaning, and grammar are implemented in the brain.