LMU ☀️ CMSI 3801
THEORIES OF COMPUTER AND HUMAN LANGUAGES
Practice Problems

Here are some long-form exercises for you to work on to improve understanding and retention of course content. Short-form recall questions, designed to be part of your spaced-repetition learning regimen, can be found at the bottom of each page of course notes.

Short Answer Questions

Here are some questions that might be found on an in-class paper-and-pencil exam.

  1. In your own words, write one sentence for each of the four major theories of computation, conveying its central question and its areas of concern. Write as if your job depended on clarity, accuracy, and solid English writing skills. If you look to an AI assistant for help, do not just copy what the bot says. The scope of the four theories are kind of fuzzy, so stick with the definitions you’ve heard in class rather than the bot’s training set.
  2. Usually, a Turing machine just rewrites its input into an output. But sometimes, all we want to compute is the answer to a YES-NO question. In this case, how does the Turing Machine announce its answer?
    It says YES by entering an accept state with no outgoing transitions for the current symbol, and answers NO by entering into a non-accept state with no outgoing transition for the current symbol. It is possible for the machine to loop forever on a given input, in which case it is said to give no answer.
  3. What is the difference between deciding and recognizing?
    A machine decides a language if it always halts with a YES or NO answer. A machine recognizes a language if it always correctly answers YES for strings in the language, but it may or may not halt with a NO for non-members.
  4. Arrange R, FINITE, RE, LR, CS, CF, and REG in subset order.
  5. What is the language class BPP?
    The set of languages that can be decided a probabilistic TM in polynomial time such that the probability of its answer being correct is $\geq$ 2/3.
  6. Arrange NP, EXPSPACE, EXPTIME, PSPACE in subset order.

Problems

Here are some problems that require some thinking, maybe a fair amount of research, and some actual work. They may involve writing little scripts, or making sketches. They aren’t exactly short-answer problems.

Some of the problems may refer to languages you have never heard of! If so, you can try solving the same problem with a language you are familiar with, or, better, look up the basics of the unfamiliar language so that you can take your best shot at the problem.

Formal Languages

The first three problems are courtesy of Phil Dorin.

  1. Let $L = \{ w \in \{0,1\}^* \mid w = w^R \}.$
    1. Is $\varepsilon \in L$?
    2. Is $101 \in L$?
    3. Is $101 \in L^2$?
    4. Is $1010 \in L^2$?
    5. Is $01101101110 \in L^*$?
  2. Let $L_1 = \{ w \in \{0,1\}^* \mid w \textrm{ has an even number of 0s and an odd number of 1s} \}$ and $L_2 = \{ w \in \{0,1\}^* \mid w = w^R \}$.
    1. Is $\varepsilon \in L_1L_2$?
    2. Is $10010110 \in L_1L_2$?
    3. Is $0010110101111 \in (L_1L_2)^*$?
    4. Is $L_1 \subseteq L_1L_2$?
    5. Is $L_2 \subseteq L_1L_2$?
    6. Is $L_1$ countably infinite? If so, prove via an appropriate bijection; if not, prove via a proof to the contrary.
    7. Is ${L_1}^*$ countably infinite? Prove or disprove.
  3. Let $L$ be the language denoted by the regular expression $0^*1 + 11(1 + 010)^*10$.
    1. Is $\varepsilon \in L$?
    2. Is $01 \in L$?
    3. Is $0001 \in L$?
    4. Is $0111 \in L$?
    5. Is $10 \in L$?
    6. Is $110100101111 \in L$?
    7. Is $1101001011110 \in L$?
    8. Is $111011101110 \in L^*$?
    9. Is $\varepsilon \in L^*$?
    10. Is $L$ countable?
  4. Given $L_1 = \{0, 011, 10\}$ and $L_2 = \{10, 1\}$. What are:
    1. $L_1 \cup L_2$
    2. $L_1 \cap L_2$
    3. $L_1L_2$
    4. ${L_2}^*$

Generative Grammars

  1. Give grammars for the following languages, all over $\{ 0, 1 \}$:
    1. Strings of length $\geq 2$
    2. Odd binary numerals
    3. Even binary numerals
    4. Binary numerals divisible by 3
    5. Binary numerals divisible by 4
    6. Signed binary numerals that are negative (in 2's complement form)
  2. Give grammars for the following languages, all over $\{ a, b \}$:
    1. Strings of length $\geq 2$
    2. Strings containing only $a$'s, except that the first character could be a $b$
    3. Strings containing at least 5 $a$'s
    4. Strings containing at least 5 consecutive $a$'s
    5. Strings not containing two consecutive $b$'s
    6. Strings whose 8th symbol from the right is a $b$
    7. Strings having twice as many $a$'s as $b$'s
    8. Strings having 3 times as many $a$'s as $b$'s
    9. Palindromes of even length
    10. Palindromes of odd length
    11. Palindromes of any length $(ww^R)$
    12. Strings whose first and last haves are the same ($ww$)
    13. Strings with an odd number of $a$'s and an even number of $b$'s
    14. $a^nb^n$
  3. Give grammars for the following languages, where the alphabet is the smallest one that makes sense for the language description:
    1. $\{ a^ib^jc^i \mid j = 2i \}$
    2. $\{ a^ib^jc^i \mid j \leq i \}$
    3. $\{ a^ib^jc^i \mid j \geq i \}$
    4. $\{ a^ib^jc^id^k \mid i,j,k \geq 1 \wedge k \textrm{ is a multiple of 3} \}$
    5. $\{ a^ib^jc^k \mid i \neq j \vee j \neq k \}$
    6. $\{ a^nb^nc^n \mid n \geq 0 \}$
    7. $\{ a^n \mid n \textrm{ is a power of 2} \}$
    8. $\{ a^n \mid n \textrm{ is prime} \}$
    9. $\{ a^n \mid n \textrm{ is not prime} \}$
    10. $\{ w \in \{a,b,c\} \mid \#_a(w) = \#_b(w) = \#_c(w) \}$
      We need the variables here because left hand sides of rules must always have at least one variable.
      s   = (x y z)*    -- repeat xyz 0 or more times
      x y = y x         -- switch them up all possible ways
      x z = z x
      y x = x y
      y z = z y
      z x = x z
      z y = y z
      x   = "a"         -- erase the variables
      y   = "b"
      z   = "c"
      
    11. $\{ a^ib^jc^id^j \mid i,j \geq 0 \}$
      s     = (l r)?        -- start with left part and right part
      l     = "a" l? x       -- generate a's on the left, counting them with x's
      r     = "b" r "d" | y  -- generate equal nums of b's and d's leaving one y in the middle
      x "b" = "b" x          -- move the x's to the right, in order to generate c's
      x y   = y "c"          -- when the x hits the y, make a c for it
      y     = ε              -- erase the y to finish it off
      
    12. $\{ a^ib^jc^k \mid 1 \leq i \leq j \leq k \}$
      s     = "a" x? "b" z "c"  -- initial set up
      x     = "a" x "b" y       -- if you generate an a, you must do b and c also
            | x? "b"? y         -- generate bc or c (keeping things increasing)
      y "b" = "b" y             -- move y's to the right
      y z   = z "c"             -- when y hits the z, make a c
      z     = ε                 -- when the z is no longer needed, drop it
      
  4. Give grammars for the following languages:
    1. The empty language
    2. $\{ 0^i1^j2^k \mid i=j \vee j=k \}$
    3. $\{ w \in \{0,1\}^* \mid w \textrm{ does not contain the substring 000} \}$
    4. $\{ w \in \{a,b\}^* \mid w \textrm{ has twice as many $a$'s as $b$'s} \}$
    5. $\{ a^nb^na^nb^n \mid n \geq 0 \}$
  5. Here’s another look at the grammar for floating-point numerals (using single-letter variables for compactness):

    $\begin{array}{l} n \longrightarrow d^+ \; f? \; e? \\ f \longrightarrow \texttt{"."} \; d^+ \\ e \longrightarrow (\texttt{"E"} | \texttt{"e"})\; (\texttt{"+"} | \texttt{"–"})? \; d^+ \\ d \longrightarrow \texttt{"0"} .. \texttt{"9"} \\ \end{array}$

    Give the $(V, \Sigma, R, S)$-definition of this grammar. (Note this means you will have to desugar the rules with |, ?, and +.)
  6. Give grammars for the languages:
    1. $\{ a^nb^nc^n\mid n \geq 0 \}$
    2. $\{ a^ib^jc^k \mid i=j \mathrm{\;or\;} j=k \}$
    3. $\{ ww \mid w \in \{a,b\}* \}$
  7. The following is a failed attempt to write a grammar for the language $L = \{w \in \{a,b\}* \,\mid\, w \mathrm{\;has\;exactly\;twice\;as\;many\;} a\mathrm{s\;as\;}b\mathrm{s}\}$:

    $S → aab \mid aba \mid baa \mid aaSb \mid abSa \mid baSa \mid aSab \mid aSba \mid bSaa \mid SS$

    1. Prove that $aaabbbbaaaaa$ is not in the language generated by this grammar.
    2. Give a correct context free grammar for $L$ (and don’t forget, that the empty string belongs, too).
    There’s a little backstory to this problem. I was given this problem while a student in UCLA in early 1988. The TA gave the incorrect answer above. I showed the problem and the TA’s solution to Phil Dorin, who didn’t think the solution was right and worked on and off for 10–15 years to prove it wrong. Finally he wrote the following message to his teacher, Sheila Greibach:

    Among the reasons that I have wanted to write is that, many years ago, my colleague, Ray Toal, whom you know, passed along a set of problem solutions that he received while studying at UCLA. They were for the 181 course—his instructor at the time was a fellow named Gabriel Robins, which probably tells you how long ago this was!—and they contained an error that I had always meant to report to you. (It’s been so long now that it has probably been corrected, but I’ll sleep a lot better once I’ve sent this off.) Specifically, the problem was to give a cfg that generated the set of all strings over alphabet $\{a,b\}$ with exactly twice as many $a$s as $b$s, which, I believe, was also a problem in an earlier edition of Hopcroft and Ullman. In any event, he gave the following solution, which he attributed to Lui:

    S → SS
    S → aaSb
    S → abSa
    S → baSa
    S → aSab
    S → aSba
    S → bSaa
    S → aab
    S → aba
    S → baa
    

    Now, I am going feel awfully much like an idiot if I am wrong about this, but... how does this grammar produce the string $aaabbbbaaaaa$ (that is, three $a$s, followed by four $b$s, followed by five $a$s)? I have managed to prove to myself that it simply can NOT produce this string, and I wonder if I should trouble you to look at it and let me know. (Technically, the grammar is also missing a rule for producing the empty string, which is also in the language, but that’s another matter.)

    I do believe that a correct grammar is:

    S → [empty string]
    S → SS
    S → aSaSb
    S → aSbSa
    S → bSaSa
    

    I’ve also worked the problem from the other direction: I constructed a npda, converted it to a cfg, and simplified it (by removing useless symbols, etc.)—but the resulting grammar doesn’t look anything like the above ones, so this didn’t provide much new insight.

    Prof. Greibach gave a nice reply, and as part of it managed to state almost nonchalantly the “obvious” proof (at least to her—in a single sentence!) of non membership of $aaabbbbaaaaa$:

    It does indeed fail on the example you gave since the first rule applied could not be any of those starting S → a... or S → b... and S → SS cannot be used because the example is not the concatenation of 2 words in the language.

Turing Machines

  1. Give a Turing Machine for multiplying a binary number by 8.
  2. Give a Turing Machine for floor-dividing a binary number by 4.
  3. Give a Turing Machine for negating a signed binary number (i.e., producing its two’s complement).
  4. Give a Turing Machine for incrementing a binary number.
  5. Give a Turing machine that produces the string "1" if its input consists of all zeros, or the string "0" otherwise.
  6. Give a Turing machine that determines whether a signed binary number is negative, i.e., that recognizes $\{ w \in \{0,1\}^* \mid w \textrm{ is a negative signed binary number} \}$.
  7. Give a Turing machine that determines whether a binary number is divisible by 5, i.e., that recognizes $\{ w \in \{0,1\}^* \mid w \textrm{ mod } 5 = 0 \}$.
  8. Give a Turing Machine ($\Sigma = \{ A \ldots Z \}$) that erases its entire input and writes the message HELLO.
  9. Give a Turing Machine ($\Sigma = \{ a,b,c \}$) that appends its input to itself. For example, if your input was $abbca$ then the output would be $abbcaabbca$.
  10. (Submitted by Amanda Marques) Give a Turing machine ($\Sigma = \{ 0, 1 \}$) that determines if its input is a palindrome, i.e., that recognizes $\{ w \in \{0,1\}^* \mid w = w^R \}$.
  11. (Submitted by Amanda Marques) Give a Turing Machine ($\Sigma = \{ a,b,c \}$) that appends the reversal of its input to itself, thereby generating a palindrome. For example, if your input was $abbca$ then the output would be $abbcaacbba$.
  12. Give a Turing machine ($\Sigma = \{ 1 \}$) that determines whether its input is exactly 8 symbols long, i.e., that recognizes $\{ w \in \{1\}^* \mid |w| = 8 \}$.
  13. Give a Turing machine ($\Sigma = \{ 1 \}$) that determines whether its input is exactly 88 symbols long, i.e., that recognizes $\{ w \in \{1\}^* \mid |w| = 88 \}$.
  14. Give a Turing machine ($\Sigma = \{ 1 \}$) that determines whether the length of its input is a power of 2.
  15. (Submitted by Amanda Marques) Give a Turing machine ($\Sigma = \{ 0, 1 \}$) that determines if its input does not contain the substring 000.
  16. Give a Turing machine ($\Sigma = \{ a, b \}$) that determines if its input has the same number of occurrences of $a$'s as $b$'s, i.e., that recognizes $\{ w \in \{a,b\}^* \mid \#_a(w) = \#_b(w) \}$.
  17. Give a Turing machine that recognizes $\{ a^nb^n \mid n \geq 0 \}$.
  18. Give a Turing machine that recognizes $\{ a^nb^nc^n \mid n \geq 1 \}$.
  19. Give a Turing machine ($\Sigma = \{ 0, 1 \}$) that determines if its input contains at least three zeros (not necessarily contiguous).
  20. Give a Turing machine ($\Sigma = \{ a, b \}$) that determines if its input contains an even number of $b$'s.
    EVEN,a,a,R,EVEN
    EVEN,b,b,R,ODD
    ODD,a,a,R,ODD
    ODD,b,b,R,EVEN
    EVEN,#,#,L,ACCEPT
    
  21. Give a Turing machine that determines for two strings, whether the first is longer than the second, given the following set up. The input alphabet is $\Sigma = \{ a, b, • \}$ and the input will be in the form $w•x$ for strings $w$ and $x$. Your TM should recognize $\{ w•x \mid w,x \in \{ 0, 1 \} \wedge |w| > |x| \}$.
  22. Give a Turing machine that determines for two strings, whether the first is a substring of the second, given the following set up. The input alphabet is $\Sigma = \{ a, b, • \}$ and the input will be in the form $w•x$ for strings $w$ and $x$. Your TM should recognize $\{ w•x \mid w,x \in \{ 0, 1 \} \wedge w \textrm{ is a substring of } x \}$.
  23. Give a Turing machine that computes the sum of two unary numbers, given the following set up. The input alphabet is $\Sigma = \{ 1, • \}$ and the input will be in the form $w•x$ for strings $w$ and $x$. Your TM should output $1^{i+j}$ when it sees the input $1^i•1^j$.
  24. Give Turing Machines that recognize the following languages. If any of the languages below are Type-3, you may (and are encouraged to) give a FA in lieu of a TM recognizer, if the FA is simpler.
    1. $\{w \in \{a,b\}* \mid w \textrm{ ends with } abb\}$
    2. $\{ w \in \{a,b\}^* \mid \#_a(w) = \#_b(w) \}$ (same number of $a$'s as $b$'s)
    3. $\{w \in \{a,b\}* \mid w \textrm{ alternates } a\textrm{'s and } b\textrm{'s} \}$
    4. $\{ a^nb^na^nb^n \mid n \geq 0 \}$
  25. Give Turing Machines that compute the following functions, where the input and output are binary numerals.
    1. $\lambda n. 2n + 2$
    2. one's complement
    3. The function described in Python as lambda n: str(n)[1:-1]

Register Machines

  1. For the JavaScript/Python expression 5 * 3 - 1 ** 3,
    1. Show a 3AC program to evaluate this expression, leaving the result in $r_0$
    2. Show a 0AC (stack machine) program to evaluate this expression, leaving the result on the top of the stack.
  2. Give stack machine code for x = y * (2 + z).
    load y
    load 2
    load z
    add
    mul
    store x
    

Language Classification

  1. Characterize each of the following languages as either (a) regular, (b) context-free but not regular, (c) recursive but not context-free, (d) recursively enumerable but not recursive, or (e) not even recursively enumerable.
    1. $\{ a^ib^jc^k \mid i > j > k \}$
    2. $\{ a^ib^jc^k \mid i > j \wedge k \leq i-j \}$
    3. $\{ \langle M\rangle\cdot w \mid M \textrm{ accepts } w\}$
    4. $\{ G \mid G \textrm{ is context-free} \wedge L(G)=\varnothing \}$
    5. $\{ a,b \}^*\{b\}^+$
    6. $\{ \langle M\rangle \mid M \textrm{ does not halt }\}$
    7. $\{ w \mid w \textrm{ is a decimal numeral divisible by 7} \}$
    8. $\{ www \mid w \textrm{ is a string over the Unicode alphabet} \}$

Computability

  1. The reachability problem is to determine for a given instruction, whether or not it might be executed for some run of the program. To optimize a program for space, we need to solve the reachability problem and remove all unreachable instructions. Show that this is impossible by reducing the halting problem to the reachability problem.
  2. Show the language $\{ \langle M_1\rangle \langle M_2\rangle \mid L(M_1) = L(M_2) \}$ is undecidable, using a rigorous reduction argument.

Complexity

  1. ...
  2. ...

Language Acquisition and Transmission

  1. ...
  2. ...

Language and Thought

  1. ...
  2. ...

Large Language Models

  1. ...
  2. ...

Programming Problems

While this course does not focus too much on programming, keeping your programming skills sharp is always a good thing. Take the opportunity to learn new things as you do.

  1. In the notes on Theories of Computer Science we saw how to express the odd-number test in Lambda Calculus notation, Lisp, Python, JavaScript, Java, Ruby, Clojure, Swift, and Rust. Show how, in each of these notations or languages, to express a function to cube a number. (Research may be required, as some of these languages may be new to you.)