Structure and Meaning in Language

Language uses signs for communication. But how?

Communication, Signification, and Language

Communication and signification are general topics, with relatively basic theories and principles. Language is a specific topic, with many complex subtopics and many theories.

Shannon’s Mathematical Theory of Communication is one paper.

What about signs? Charles Sanders Peirce developed a comprehensive theory of signs, categorizing them into icons, indices, and symbols based on their relationship to the objects they represent. You can learn the basics in a day.

Icons

Resemble what they represent

A photo, a map, onomatopoeia

Indices

Causally connected to their referent

Smoke → fire, pointing

Symbols

Arbitrary, conventionally determined

Most words, red=stop

Human language is overwhelmingly symbolic. No other species uses symbols at scale. We’ve been doing it for a long time:

Exercise: Read The Symbolic Species by Terrence Deacon. Or ask an AI assistant to summarize it. What are the main theses of the book? (Hint: symbolic thought and language co-evolve) How does Deacon describe symbols?

Symbols are fascinating because they have no intrinsic connection to their referent. They require (a) shared convention, (b) memory, (c) the ability to link, but still keep separate, “word” and “meaning.” Symbols have unlimited generativity—you can make a symbol for anything, even abstractions: justice, infinity, recursion, love, LLM, the meaning of life, etc. With symbols, our ability to express and communicate is unbounded. It can also create lies, misinformation, and fictional worlds.

But where do symbols come from and how do they hook onto the world? Philosophers call this the symbol grounding problem. Symbols in a formal system only refer to other symbols, much as how a dictionary defines words with more words. There is no base word. Humans ground symbols through perception and action (and convention).

Exercise: LLMs manipulate symbols without perceptual grounding. What does that mean for “understanding”?

Language is far more specialized and does not admit to a single theory. Sure, it uses signs for communication. One working definition is: a structured system of communication, consisting of utterances that can be spoken, signed, or written, formed via compositional rules (or guidelines) agreed upon by users of the language.

The study of human language is called linguistics.

We studied words and sounds earlier, so now it’s time for syntax, semantics, and pragmatics.

Grammar

Words and word-like entities are combined (arranged) according to rules to form larger units like phrases and sentences. The rules constitute a grammar. Where do the rules come from? How do we know if an utterance is well-formed?

Rules or Guidelines?

In programming languages, rules are explicit and prescriptive. In spoken natural language, we tend to treat them as guidelines, but in writing, they are often more rigidly followed.

A grammar will tell you where all the components of an utterance should go—based on certain attributes and context, but it won’t tell you what the utterance means. Classic examples are:

Kinds of Grammars

There are many different approaches to capturing our idea of a grammar. Here are five of many.

Constituency Grammar

Constituency grammar emphasizes the grouping of words into constituents or phrases, which function as single units within a hierarchical structure. The structure is typically represented as a tree where nodes correspond to constituents, highlighting the nested and recursive nature of language. Constituency grammars are also called phrase-structure grammars, and these are the generative grammars we saw in our notes on language theory.

Here is a rough constituency grammar for a very tiny subset of English sentences. It does not capture all the complexities of the full language. It omits many legal forms, ignores exceptions, and completely ignores morphology:

$\begin{array}{lcl} \textsf{S} & \rightarrow & \textsf{NP} \; \textsf{VP} \; (\textsf{CONJ} \; \textsf{S} )*\\ \textsf{NP} & \rightarrow & (\textsf{PN} \mid \textsf{DET} \; \textsf{ADJ}^* \; \textsf{NOUN}) (\textsf{RP} \; \textsf{VP})? \\ \textsf{VP} & \rightarrow & \textsf{ADV}^* \; (\textsf{IV}\;\textsf{PP}? \mid \textsf{TV} \; \textsf{NP} \mid \textsf{DV} \; \textsf{NP} \; \textsf{PP} \mid \textsf{SV} \; \textsf{S}) \\ \textsf{PP} & \rightarrow & \textsf{PREP} \; \textsf{NP} \\ \textsf{PN} & \rightarrow & grace \mid maika \mid alan \mid she \\ \textsf{DET} & \rightarrow & a \mid the \mid his \mid her \\ \textsf{NOUN} & \rightarrow & doctor \mid dog \mid rat \mid girl \mid toy \mid fox \\ \textsf{RP} & \rightarrow & who \mid that \\ \textsf{ADJ} & \rightarrow & blue \mid heavy \mid quick \mid new \mid lazy \\ \textsf{PREP} & \rightarrow & in \mid to \mid above \mid around \mid through \mid over \mid on \\ \textsf{IV} & \rightarrow & fell \mid jumped \mid swam \\ \textsf{TV} & \rightarrow & liked \mid knew \mid hit \mid missed \\ \textsf{DV} & \rightarrow & gave \mid threw \mid handed \\ \textsf{SV} & \rightarrow & dreamed \mid believed \mid thought \mid knew \\ \textsf{ADV} & \rightarrow & quickly \mid slowly \mid silently \mid well \\ \textsf{CONJ} & \rightarrow & and \mid or \mid but \\ \end{array}$

Given a constituency grammar, you can show how phrases and sentences are structured hierarchically. Here is a classic example:

fox-constituency.png

How do you tell whether some sequence of words actually forms a constituent, as opposed to being an arbitrary run of words? We run constituency tests:

Constituency ambiguity shows up whenever a sentence has more than one legal parse tree. The textbook example is prepositional-phrase attachment: “I saw the scientist with the telescope.”

Exercise: Draw both parse trees for “I saw the scientist with the telescope.” using the constituency grammar provided above. Draw both trees for “I shot an elephant in my pajamas.”

Transformational Grammar

Transformational grammar, developed by Noam Chomsky, starts from the phrase-structure tree shown above, then adds a further claim: a sentence has (at least) two levels of structure—a deep structure capturing its core grammatical relations, and an observable surface structure derived from it by transformations, rules that rearrange, insert, or delete material. “The dog chased the cat” and “The cat was chased by the dog” get different surface structures but, on this view, share essentially the same deep structure.

Continue your study at Wikipedia.

Dependency Grammar

Dependency grammar focuses on the relationships between words in a sentence, where one word (the head) governs the others (dependents). The structure is represented as a tree with directed edges from heads to dependents, emphasizing the syntactic functions and hierarchical organization of words. Unlike constituency or transformational grammar, there are no phrasal nodes at all—every node in the tree is a word, and the tree has exactly as many nodes as the sentence has words.

The idea goes back to the French linguist Lucien Tesnière, whose posthumous Éléments de syntaxe structurale (1959) argued that the verb, not the sentence-splitting subject/predicate distinction of school grammar, is the true organizing center of a clause: every other word depends, directly or indirectly, on it.

Here’s a dependency parse of The dog chased the cat. Each arc points from a head to its dependent, and is labeled with the grammatical relation between them (using the tag set from the Universal Dependencies project, a widely-used cross-linguistic annotation standard):

root nsubj det obj det The dog chased the cat

This says: chased is the root of the sentence (its main predicate); dog is its subject (nsubj); cat is its object (obj); and each noun has its own determiner (det). No node stands for “noun phrase” or “verb phrase” at all—those groupings, if you want them, are just implicit in which words are reachable from which heads.

Compare this to the constituency tree above. There, an intermediate NP node grouped the cat together as a unit before it ever attached to the verb. Here, there’s no such intermediate node—chased is directly the head of cat, with the det relation doing the work that constituency structure did with nesting. The two aren’t just cosmetic variants of each other; they encode different hypotheses about what the “real” units of syntax are.

Because a dependency tree tracks grammatical relations rather than linear groupings, it tends to survive word-order changes better than a constituency tree does. This matters a lot for languages with much freer word order than English—Latin, Russian, Czech, Japanese, Warlpiri—where a constituency tree for “subject–verb–object” order would have to be redrawn for every permutation, but the dependency relations (who did what to whom) stay exactly the same. This is part of why Tesnière’s framework caught on outside of English-centered linguistics.

A dependency tree is called projective if you can draw all of its arcs above the sentence without any of them crossing, as in the example above. Projectivity mostly (but not perfectly) tracks configurational rigidity: free word-order languages, and constructions like extraction (“What did you say she bought __?”), can force non-projective, crossing-arc trees. Parsing algorithms differ in whether they can handle these directly:

Dependency grammar is the dominant representation in modern NLP tooling: spaCy, Stanford CoreNLP/Stanza, and the Universal Dependencies treebanks (100+ languages, all annotated with the same relation labels) all use it, partly because it’s a compact, uniform target that plays nicely with statistical and neural parsers.

Exercise: Go to spaCy’s displaCy demo and parse a sentence of your choosing. Find the root. Can you construct an English sentence whose dependency parse is non-projective?

Operator Grammar

Operator grammar is a theory that focuses on the combinatory properties of operators (typically verbs) and their arguments. It emphasizes the rules governing how operators can combine with other elements to form grammatical sentences, often highlighting the constraints and regularities in argument structure.

It was developed by Zellig Harris—a major figure in American structural linguistics, and, notably, Noam Chomsky’s teacher at the University of Pennsylvania—mostly in a series of dense late-career works: A Grammar of English on Mathematical Principles (1982), Language and Information (1988), and A Theory of Language and Information (1991). Where Chomsky went on to posit an innate, autonomous syntax largely independent of meaning and usage, Harris went the other direction, trying to show that the entire structure of a language—syntax included—can be derived from just three empirically observable ingredients: which words require which other words, how likely particular combinations are, and how frequent combinations get compacted in use.

Operators and Arguments

An operator is a word that requires one or more other words, called its arguments, to be present for the sentence to be complete—in effect, exactly the subcategorization or valency of the word, but treated as the single foundational relation of the whole grammar. Words that require nothing further (roughly, ordinary nouns) form the base argument class, N. Every operator is then classified by the sequence of argument classes it demands:

ClassMeaningExampleSentence
ONoperator + 1 N-argumentstumbleJohn stumbles.
ONNoperator + 2 N-argumentswearJohn wears boots.
ONNNoperator + 3 N-argumentsputJohn put the boots in the closet.
ONOoperator + 1 N-argument, 1 O-argument (a whole clause)sayJohn said it rained.
ONNOoperator + 2 N-arguments, 1 O-argumentaskJohn asked Mary to leave.
OOOoperator + 2 O-arguments (two whole clauses)becauseIt rained because it was cold.

What makes this genuinely different from ordinary subcategorization frames is that an operator’s O-arguments aren’t noun phrases at all—they’re entire sentences, themselves headed by their own operator. So say in John said it rained takes John (an N) and the whole clause it rained (an O, itself built from the operator rained and its own N argument it) as one nested unit:

said (class ONO) N John O rained (class ON) N it

This recursive nesting of operators inside the argument slots of other operators is what lets operator grammar build sentences of arbitrary depth without ever leaving the operator-argument relation—no separate phrase-structure rules like $S \rightarrow NP\;VP$ are needed at all. (It’s also, in effect, the same insight modern computational semantics reaches for with predicate-argument structure, thematic/semantic roles, and resources like PropBank—Harris got there first, from a very different direction.)

Likelihood and Reduction

Dependency alone only says an operator’s argument slots must be filled with something of the right class; it doesn’t say which filler is natural. That’s the job of likelihood: given an operator, some fillers for its argument are simply more probable, in ordinary usage, than others. John wears hats is more likely than John wears snow, which is in turn more likely (if still odd) than John wears vacation. The set of arguments with above-average likelihood for a given operator is called that operator’s coherent selection, and Harris observed that words similar in meaning tend to have similar coherent selections—which is itself a distributional definition of what a word means, decades before “a word is known by the company it keeps” became a slogan for word embeddings and distributional semantics.

Once a combination is highly likely, speakers tend to compress it—a process Harris called reduction. Crucially, reductions are treated as near-paraphrases: they remove no information, they just make an already-predictable sentence more compact. A few kinds, with the Wikipedia article’s own examples:

Notice the shape of the whole system: dependency builds the raw predicate-argument skeleton, likelihood is what picks out well-formed, meaningful combinations from merely well-typed ones, and reduction compacts the predictable parts for efficient communication. Nothing else is required.

Why Call It a “Theory of Human Language”?

Most of what we’ve called “grammars” so far—constituency, transformational, dependency—are really representations: notations for describing the structure of a sentence once an independent rule system has already told you what counts as grammatical. Operator grammar is more ambitious: Harris intended it as a self-contained, self-organizing account of language as such, where grammaticality itself falls out of the three mechanisms above rather than being stipulated by an independent rule system. In his own framing, no external metalanguage—no separate inventory of phrase-structure rules, no abstract deep structure distinct from what’s actually said—is needed to state what a language is; the operator-argument relation, likelihood, and reduction are enough, and they’re all in principle measurable directly from how words actually co-occur in real text. That’s a substantive, falsifiable claim about the nature of language, not just a convenient diagramming notation—hence “theory of language,” not merely “grammar formalism.”

Harris pushed this furthest in his work on sublanguages—the specialized, highly regular registers used within a scientific field. In The Form of Information in Science (1989), he analyzed a corpus of immunology research articles and showed that, within that domain, the operator classes and the vocabulary that fills their argument slots become dramatically more constrained than in general English: authors reuse a small set of recurring operator-argument patterns (e.g., a fixed handful of verbs relating antibodies, antigens, and specific measured reactions) almost formulaically. If you’ve heard of modern information-extraction techniques that pull structured facts (who did what to whom) out of scientific abstracts, you’re seeing essentially the same idea: restrict a text to a narrow sublanguage and its operator-argument structure becomes tractable enough to extract automatically.

Operator grammar itself never became mainstream compared to its rivals here, but it left descendants: the Wikipedia article on the topic points to link grammar (Sleator and Temperley, 1991) as “a kind of operator grammar,” and the broader idea that syntax is fundamentally a network of predicate-argument requirements—rather than a separate phrase-structure layer—resurfaces constantly in computational linguistics, from valency dictionaries to semantic role labeling to the dependency parsers you saw above.

Construction grammar, up next, reaches a conclusion with a similar flavor—no independent rule system standing apart from meaning—but arrives there from the opposite direction: instead of grounding everything in distributional likelihood over a corpus, it grounds everything in the cognitive claim that form and meaning are inseparable at every grain size, from morphemes to entire argument-structure patterns.

Exercise: Classify the main operator in each sentence by its argument-class notation (e.g., ONN): (a) “The cat slept.” (b) “Maria gave Tom the keys.” (c) “She believes it will rain.” (d) “He resigned because he was tired.”
Exercise: Take the two sentences “John owns a car” and “The car is red” and show a plausible transposition that reduces them to one sentence, the way John wears boots; the boots are of leather reduced to John wears leather boots.

Construction Grammar

Construction grammar arrives at a similarly radical conclusion as operator grammar—no principled split between the rules of grammar and the meanings they express—but gets there from the opposite direction: not distributional likelihood over a corpus, but the cognitive claim that knowledge of language consists of a collection of form-meaning pairings, known as constructions. These constructions range from simple words to complex syntactic patterns, and meaning is derived from the interaction of these constructions rather than from abstract rules alone.

The radical part of the claim is this: a construction is a sign—a paired form and meaning, exactly like the Peircean symbols discussed above—and constructions exist at every grain size, with no principled boundary between “the lexicon” (memorized, irregular, word-sized) and “the grammar” (productive, regular, rule-sized). Traditional generative grammar treats these as fundamentally different systems; construction grammar treats them as the same kind of object at different points on a continuum of size and schematicity:

ConstructionExampleNote
Morpheme-ingfully substantive, atomic meaning (progressive aspect)
Wordcatfully substantive
Idiom (filled)kick the bucketfully substantive, but its meaning (“die”) isn’t the sum of its parts
Idiom (partially filled)the X-er, the Y-ere.g. “the more you practice, the better you get”
Argument structure constructionSubj V Obj1 Obj2fully schematic (the ditransitive), meaning “X causes Y to receive Z”

The evidence Goldberg and others point to is that argument-structure constructions seem to carry meaning of their own, independent of the verb slotted into them. The verb sneeze is intransitive—sneezing doesn’t normally take an object—yet:

“She sneezed the napkin off the table.”

is immediately understood to mean that her sneeze caused the napkin to move off the table. Nothing about the lexical meaning of sneeze supplies that caused-motion reading; it comes from the caused-motion construction itself (schematically, Subj V Obj Oblpath/loc ↔ “X causes Y to move Z”), which imposes its meaning on whatever verb is placed inside it. The same logic explains subtle contrasts like He baked her a cake (implies she ends up with the cake—the ditransitive construction’s “transfer” meaning) versus He baked a cake for her (merely “on her behalf”; no transfer is entailed).

construction: DITRANSITIVE FORM Subj V Obj₁ Obj₂ MEANING “X causes Y to receive Z”

This makes construction grammar naturally compatible with the usage-based acquisition story discussed below: instead of setting innate parameters, a learner could build up a construction inventory (sometimes called a constructicon) directly from experience, via general cognitive mechanisms like categorization and analogy, generalizing from concrete utterances to progressively more schematic patterns. It also gives a suggestive frame for thinking about what a language model is doing: an LLM trained purely on usage, with no rule component at all, ends up representing something a lot like an enormous, distributed constructicon—form-meaning (or at least form-distribution) associations at every grain size, learned rather than programmed in.

Exercise: For each of the following, decide roughly where on the substantive–schematic continuum it belongs, and state the construction’s form and its meaning (if any beyond the sum of its parts): by and large, un-X-able (as in undoable, unreadable), the X-er the Y-er, and the resultative X V Y Z-adjective (as in “She wiped the table clean” / “He hammered the metal flat”).
CLASSWORK
Construct another sentence, like the sneeze example, where an intransitive verb gets pressed into a transitive or caused-motion reading purely because of the construction it’s placed in. What does the verb normally mean on its own, and what does the construction add?

Parsing

Technically, the grammar composes utterances into hierarchical or networked structures, even though they tend to be written or spoken as temporal one-dimensional sequences.

Exercise: Show a constituency parse tree of a simple sentence vs. a center-embedded one using a tool like the Berkeley Neural Parser. Why does the center-embedded version overload working memory even though it’s grammatical?

Recursion

Recursion is the property of language that allows rules to be applied repeatedly, embedding structures within structures. This is what enables sentences to be infinitely long and complex, even with a finite set of rules and vocabulary.

Here’s an example. For the constituency grammar near the beginning of this page, we can make the derivation:

S ⇒ NP VP
  ⇒ DET NOUN RP VP VP
  ⇒ the NOUN RP VP VP
  ⇒ the dog RP VP VP
  ⇒ the dog that VP VP
  ⇒ the dog that SV S VP
  ⇒ the dog that thought S VP
  ⇒ the dog that thought NP VP VP
  ⇒ the dog that thought PN VP VP
  ⇒ the dog that thought grace VP VP
  ⇒ the dog that thought grace TV NP VP
  ⇒ the dog that thought grace hit NP VP
  ⇒ the dog that thought grace hit PN VP
  ⇒ the dog that thought grace hit alan VP
  ⇒ the dog that thought grace hit alan DV NP PP
  ⇒ the dog that thought grace hit alan threw the new blue toy to the fast rat
    .
    .
    .
  ⇒ the dog that thought grace hit alan threw the new blue toy to the fast rat

Recursion can allow sentences to become arbitrarily long and complex:

S ⇒ NP VP
  ⇒ NP SV S
  ⇒ NP SV NP VP
  ⇒ NP SV NP SV S
  ⇒ NP SV NP SV NP VP
  ⇒ NP SV NP SV NP SV S
    .
    .
    .
  ⇒ she dreamed she dreamed she dreamed . . . she dreamed the dog swam

inception.png

Language Acquisition

Grammar has to come from somewhere. A child ends up with a rich, productive system—not a memorized list of sentences—after hearing only a finite, messy, error-filled sample of a language. Where does the rest come from? This is the question that split twentieth-century linguistics down the middle.

Chomsky’s answer was that it couldn’t come from the input alone. His Poverty of the Stimulus argument holds that children productively use grammatical rules and constraints they’ve never seen exemplified, and are essentially never corrected on the subtle cases that would let them learn those constraints by trial and error. If the input under-determines the grammar, something has to fill the gap—so he proposed an innate Language Acquisition Device (LAD), a language-specific piece of mental machinery shared by the whole species, embodying a Universal Grammar (UG): a fixed set of universal principles, with a small number of open parameters that get set one way or another by whatever language a child happens to be exposed to (this specific version of the theory is usually called Principles and Parameters). Pinker’s The Language Instinct (1994) is the most famous popularization of a related idea: that grammar is a Darwinian biological instinct, shaped by natural selection specifically for communication, the way echolocation is shaped in bats. Chomsky and Pinker disagree sharply on the mechanism (an abstract, largely unselected cognitive module vs. an adaptation actively shaped by selection), but both are committed to some domain-specific innate machinery dedicated to grammar.

That premise has been under sustained attack for a couple of decades now, on several independent fronts. Usage-based linguists—Tomasello and Christiansen prominently among them—argue that general-purpose learning mechanisms (statistical pattern extraction, analogy, categorization) plus rich social interaction are sufficient, with no dedicated grammar module required; a child doesn’t need innate syntax if they’re instead innately very good at noticing patterns and very motivated to communicate. This is, not coincidentally, exactly the acquisition story that fits construction grammar from earlier on this page: if grammar just is a big inventory of form-meaning pairings at every grain size, there’s no separate “rules module” that would need to be innate in the first place—just the general cognitive capacity to extract and generalize patterns from constructions you’ve actually heard. Cross-linguistic typology has piled on more evidence: as languages outside the small set Chomsky originally generalized from got carefully documented, the “universals” UG predicted turned out to have far more exceptions than expected, casting doubt on there being a fixed, universal syntactic template underneath every human language at all.

Exercise: Read Tomasello’s Constructing a Language: A Usage-Based Theory of Language Acquisition. What are the main arguments against the need for an innate LAD? Do you find them convincing?

Some of the most striking evidence for the usage-based side comes from cases where a full grammar appears with no adult model to imitate at all. Pidgins are rudimentary contact languages, with no native speakers and minimal grammar, that spring up when adult speakers of mutually unintelligible languages need to communicate. Creoles are what happens when children are then raised hearing a pidgin as their primary input: within a single generation, the children spontaneously develop a full, grammatically regular language out of the pidgin’s inconsistent scraps—productive tense/aspect marking, consistent word order, subordinate clauses, none of which were reliably present in what they heard. The most dramatic documented case is Nicaraguan Sign Language: deaf children in 1980s Nicaragua, thrown together for the first time in newly-formed schools with no shared sign language and no fluent adult signers to imitate, collectively invented one—complete with regular grammar—within about a generation. If grammar has to be triggered by exposure to already-grammatical input, cases like this are hard to explain; if children can construct grammar out of unstructured or minimal input given the right social conditions, they’re exactly what you’d expect.

Exercise: Given everything above, which of the five grammars from the “Kinds of Grammars” section fits most naturally with an innate-UG story, and which fits most naturally with a usage-based story? Defend your answer for at least two of the five.

The Emergence of Syntax

Acquisition asks how one child, in one lifetime, ends up with a grammar. This section asks a different question: how does a whole population end up sharing one, generation after generation, with nobody ever sitting down and designing it? The leading answer treats language itself as something that evolves culturally—shaped, over repeated transmission from speaker to learner, by nothing more than what turns out to be learnable.

The Iterated Learning Model

The Iterated Learning Model (ILM), developed by Kirby, Smith, and colleagues, formalizes this as a chain: a speaker produces utterances for a range of meanings; a learner sees only a small sample of those utterance-meaning pairs (never the whole language—this is the crucial bottleneck); the learner infers a grammar from that sample and becomes the next speaker; repeat, indefinitely.

Speaker produces utterances for many meanings Bottleneck only a small sample gets transmitted Learner infers a grammar from the sample learner becomes the next generation’s speaker

Why should this loop, repeated with no communicative goal and no innate grammar built in, produce anything structured at all? Because of what survives the bottleneck. A holistic language, where every meaning gets an unrelated, unanalyzable label, can’t be reconstructed from a small sample—if a learner never happened to see the label for a particular meaning, they have no way to guess it, since nothing about the other labels tells you anything about it. A compositional language, where meanings are built from meaningful parts combined by general rules, survives the bottleneck beautifully: a learner who has seen only some of the utterances can still generalize the rule and produce correct utterances for meanings they’ve never encountered. Across repeated transmission, languages that happen to be easier to learn from partial data get reproduced more faithfully than languages that aren’t—so compositional structure accumulates over generations purely because it’s more transmissible, with no communicative pressure and no innate syntax required to explain why. Kirby, Cornish, and Smith ran exactly this as a human iterated-learning experiment: an artificial language, initially unstructured, passed through a chain of participants (each trained only on a subset of the previous participant’s output) became progressively more compositional and more learnable over successive “generations,” with no participant ever intending to create structure.

Exercise: Read the Kenny Smith paper on the Iterated Learning framework. Why must a bottleneck specifically be involved? What would happen in this model if learners instead got to see the entire language produced by the previous generation?

Related and Competing Accounts

ILM isn’t the only bottleneck-based story making the rounds. A few other threads worth knowing about:

CLASSWORK
Discuss the fact that LLMs acquire language from stimulus alone, with no innate LAD, and they do remarkably well. LLMs learn grammar implicitly from the data, without explicit instruction. They can generate grammatically correct sentences, but they don’t have an explicit representation of grammatical rules like humans do. How is this possible? Are transformers just big enough to brute-force what evolution gave us for free? Or do we have something else? How does this relate to the neural iterated-learning results above—do LLMs experience anything like a transmission bottleneck during training?

Here’s a broader set of readings on cultural evolution and the emergence of linguistic structure, for anyone who wants to go deeper:

So, Whither Universal Grammar?

Put Acquisition and Transmission together and UG’s original motivation looks a lot shakier than it did in the 1960s. The poverty-of-the-stimulus premise assumed structure had to come from somewhere innate because it couldn’t plausibly come from learning or from the input—but usage-based acquisition research, typological diversity, the spontaneous emergence of creoles and Nicaraguan Sign Language, the Iterated Learning Model’s demonstration that compositional structure emerges from transmission bottlenecks alone, and the (surprising, to many) success of LLMs learning grammar from raw text with no built-in syntax module, all chip away at that premise from different directions. None of this is a knockout blow—Chomsky, Pinker, and a real (if shrinking) contingent of nativist linguists still defend some version of innate, domain-specific machinery for language—but the burden of proof has shifted. It’s no longer obvious that you need innateness to explain how grammar gets into a human head, or how it stays coherent across a whole population over time.

Grammars for Constructed Languages

TODO

Recall Practice

Here are some questions useful for your spaced repetition learning. Many of the answers are not found on this page. Some will have popped up in lecture. Others will require you to do your own research.

  1. What is Chomsky’s poverty of the stimulus argument?
    Children acquire grammatical rules they have never seen exemplified in their input, which suggests that at least some linguistic knowledge is innate (the Language Acquisition Device). The stimulus is too impoverished for pure learning to account for the result.
  2. What does the emergence of Nicaraguan Sign Language demonstrate?
    That grammar can emerge spontaneously from social conditions alone — deaf children in 1980s Nicaragua, without an adult sign language model, invented a full grammatical sign language within one generation.
  3. What is the difference between the strong and weak versions of the Sapir-Whorf hypothesis? Which is supported empirically?
    Strong (linguistic determinism): language determines thought — we cannot think what we cannot say. Now largely discredited. Weak (linguistic relativity): language influences habitual thought. This is supported empirically, e.g., color term differences affecting discrimination speed.
  4. What are the three types of signs identified by Charles Sanders Peirce?
    Icons (resemble what they represent), Indices (causally connected to their referent), Symbols (arbitrary, conventional).

Summary

We’ve covered:

  • TODO
  • TODO