1 Mealy Machines
1.1 True or False?
For each of the following functions, decide whether they can be realized by a Mealy Machine. In positive cases, provide the Mealy Machine, in negative cases, provide a proof that it cannot be realized.
What are the extensions of Mealy Machines for which the above functions are computable? Among the above functions, which ones are continuous for the regular topology?
1.2 Arithmetic Circuits
The goal of this exercise is to prove that operations on binary numbers are possible. To that end we have to provide an encoding of tuples numbers, which we do as follows: a tuple \((n_1, \dots, n_k) \in \mathbb{N}^k\) is represented on the alphabet \(\{ 0, 1 \}^k\) by writing the numbers in binary, and padding them with zeros so that the length matches. There are four variants of this encoding, obtained by deciding whether to pad on the left or the right, and whether to write numbers with the most significant bit on the left or the right.
- For each of the four possible encodings, decide whether the map \((+) \colon \mathbb{N}^2 \to \mathbb{N}\) can be represented using a Mealy Machine.
- For each of the four possible encodings, decide whether the map \((/3) \colon \mathbb{N}\to \mathbb{N}\) can be represented using a Mealy Machine.
- Write a Mealy Machine that computes \((n,4n)\) in binary.
- Deduce a Mealy Machine that computes \(5n\) in binary by using the wreath product construction and the construction of the addition.
1.3 Bonus: Presburger Arithmetic
Prove that Presburger Arithmetic is decidable.
Hint: Encoding of numbers and formulas
Encode a formula \(\varphi(\vec{x})\) as a regular language of \(\mathbb{N}^X\), where numbers are encoded in binary with the most significant bit is on the left, and the padding is on the right.
Hint: Presburger Operators
Start by proving that each of these operations are computed by Mealy Machines.
- The equality operator \((=) \colon \mathbb{N}^3 \to \{ 0,1 \}\).
- The addition operator \((+) \colon \mathbb{N}^2 \to \mathbb{N}\).
- The existential quantifier \((\exists x) \colon \mathbb{N}^{x \vec{y}} \to \mathbb{N}^{\vec{y}}\).
1.4 Flip Flop Machines
Prove that every flip-flop machine can be obtained by composing binary flip-flop machines. What is the number of intermediate machines that are needed?
Hint: Encode states
Given a state \(q \in Q\), compute using a binary flip-flop machine the sequence of approximated states \(\{ q, \neg q \}\).
Hint: For the upper bound
Use a binary encoding to obtain a logarithmic number of intermediate machines.
Hint: For the lower bound
What can you say about a machine that shifts its input by one position?
1.5 Regularity of Mealy Machines
The goal of this exercise is to understand the relationship between Mealy Machines and regular languages. Let \(f \colon \Sigma^* \to \Gamma^*\) be a function computed by a Mealy Machine.
- Prove that the image of \(\Sigma^*\) through \(f\) is a regular language.
- Prove that the pre-image of \(\Gamma^*\) through \(f\) is a regular language.
- Let \(L\) be a regular language, prove that \(f\left(L\right)\) and \({f}^{-1}\left(L\right)\) are regular languages, i.e., that \(f\) is open and continuous for the regular topology.
- Is every open and continuous map computable by a Mealy Machine?
- A function \(f \colon \Sigma^* \to \Gamma^*\) is Lipschitz for the prefix distance. What is the value of the Lipschitz constant?
- The graph of a function \(f \colon X \to Y\) is the subset \(\mathsf{graph}(f) \subseteq X \times Y\) defined by \(\left\{ (x,y) \in X \times Y \mid f(x) = y \right\}\). Can you provide a necessary and sufficient condition on the graph of \(f\) for it to be representable using a Mealy Machine?
1.6 Decidability Properties of Mealy Machines
In this exercise, the goal is to understand what is decidable about Mealy Machines. For each of the following questions, prove (or disprove) that it is decidable, and in case of decidability, provide a precise complexity class.
- Can we decide if two Mealy Machines compute the same function?
- Can we decide if a Mealy Machine is surjective?
- Can we decide if \(f(w) \sqsubseteq g(w)\) for all \(w \in \Sigma^*\)?
- Can we decide if a Mealy Machine is injective?
- Can we decide if there exists \(w \in \Sigma^*\) such that \(f(w) = g(w)\)?
- Can we decide if a Turing machine computes a function that can be computed by a Mealy Machine?
Hint: Deciding Injectivity
Consider the set \(\left\{ (u,v) \in \Sigma^* \times \Sigma^* \mid f(u) = f(v) \right\}\), and show that it is a regular language.
1.7 Variations on Mealy Machines
Describe the relationship between the expressiveness of the following variations of Mealy Machines:
- Mealy Machines
- Mealy Machines with lookaheads.
- Sequential functions.
- Mealy Machines with lookaheads with an ambiguous transition relation, but such that every run produces the same output.
- Mealy Machines with lookaheads with an ambiguous transition relation, where the semantic is undefined if there are multiple runs producing different outputs.
- Mealy Machines with transitions labelled by regular expressions.
1.8 Efficient String Matching
The goal of this homework is to study the problem of string matching. That is, given a pattern \(m \in \Sigma^*\) and a text \(t \in \Sigma^*\), one wants to produce a text \(m(t) \in (\Sigma \uplus \bar{\Sigma})^*\) where occurrences of \(m\) are overlined. To avoid ambiguity, we will overline non-overlapping occurrences of the pattern, starting from the left of the text \(t\).
- Is the function that underlines the starts of the matches computable by a Mealy Machine? By a sequential function? By a Mealy Machine with lookaheads?
- Same question with underlining the ends of the matches.
- Same question for the function \(m\).
- Conclude by providing an efficient algorithm to perform string matching. What is the (time/space) complexity in \(\left| m \right|_{}\)? What is the (time/space) complexity in \(\left| t \right|_{}\)?
2 Homework
2.1 Continuous Functions
Prove that there exists uncountably many continuous functions from \(\Sigma^*\) to \(\Gamma^*\) for the regular topology. It is true for continuous and prefix preserving functions?
Hint: The alphabet does not matter
Consider the set of all functions from \(\mathbb{N}\) to \(\mathbb{N}\).
Hint: Sufficient conditions for continuity
Show that the following conditions are sufficient for a function \(f \colon \mathbb{N}\to \mathbb{N}\) to be continuous:
- \(\forall n \in \mathbb{N}, f(n)\) is a factorial,
- \(\liminf_{n \to \infty} f(n) = \infty\).
Solution: Complete Solution
We will follow the hints given, namely that the alphabet does not matter and that it is quite easy to be continuous. Indeed, consider \(f \colon \mathbb{N}\to \mathbb{N}\) such that \(f(n)\) is a factorial number, and \(\liminf_{n \to \infty} f(n) = \infty\). Let \(L \in \{ 1 \}^* \simeq \mathbb{N}\) be a regular language, i.e., a language recognized by some finite monoid \(M\), with a morphism \(\mu \colon \mathbb{N}\to M\) and an accepting part \(P \subseteq M\). Our goal is to prove that \({f}^{-1}\left(L\right)\) is a regular language too.
Let \(n \in \mathbb{N}\) such that \(f(n) \in L\), by definition it means that \(\mu(f(n)) \in P \subseteq M\). Recall that for all \(n \in \mathbb{N}\), there exists \(m \in \mathbb{N}\) such that \(f(n) = m!\), i.e., \(f(n) = 1^{m!}\), as a consequence, \(\mu(f(n)) = \mu(1)^{m!}\).
By standard results on finite monoids, there exists an idempotent \(e\) in \(M\) such that for all \(m \geq |M|\), \(\mu(1)^{m!} = e\). Let us sketch the proof for completeness purposes. This is obtained by remarking that the semigroup generated by \(\mu(1)\) is finite, hence contains an idempotent \(e\), obtained for some power \(\mu(1)^k\) with \(k \leq |M|\). Unicity is obtained by noticing that if \(e_1 = \mu(1)^{k_1}\) and \(e_2 = \mu(1)^{k_2}\), then \(e_1 = e_1^{k_2} = \mu(1)^{k_1 \times k_2} = e_2^{k_1} = e_2\).
Now, using the fact that \(\liminf_{n \to \infty} f(n) = \infty\), there exists \(n_0 \in \mathbb{N}\) such that for all \(n \geq n_0\), \(f(n) \geq |M|!\). As a consequence, for all \(n \geq n_0\), \(\mu(f(n)) = e\). If \(e \in P\), this proves that \({f}^{-1}\left(L\right) = \left\{n \mid n \geq n_0\right\} \cup \left\{ n < n_0 \mid f(n) \in L\right\}\), the latter is a regular language as the union of two regular languages. Otherwise, \(e \not\in P\), and we conclude that \({f}^{-1}\left(L\right) = \left\{n < n_0 \mid f(n) \in L\right\}\), which is also a regular language.
To conclude, let us remark that there are uncountably many functions satisfying the above conditions. This can be proven by considering the following injection of \(\mathbb{N}^\mathbb{N}\) (which is well known to be uncountable) into such functions as follows: to a sequence \((p_n)_{n \in \mathbb{N}}\) of numbers, one can associate the function \(f \colon n \mapsto (\sum_{i=0}^n p_i)!\).
Remark that in the above injection, all functions are prefix preserving, hence the answer to the second question is also positive. This has to be compared with the Theorem charaterizing sequential functions.
2.2 Stability properties of Sequential Functions
We say that a function preserves prefixes if for all \(u,v \in \Sigma^*\), \(u \mathrel{\sqsubseteq_{\mathsf{prefix}}}v\) implies \(f(u) \mathrel{\sqsubseteq_{\mathsf{prefix}}}f(v)\). Prove that the following propositions are equivalent for a function \(f \colon \Sigma^* \to \Gamma^*\):
- \(f\) is sequential.
- \(f\) is continuous for the regular topology, Lipschitz for the prefix distance, and preserves prefixes.
2.2.1 Solution of the easy implication
Let \(f\) be a sequential function, computed by a sequential transducer \(T = (Q_T, \delta_T, q_0^T, \lambda_T)\). Our first objective is to prove that \(f\) is continuous. To that end, let \(L\) be a regular language recognized by a finite monoid \(M\), a morphism \(\mu \colon \Gamma^* \to M\), and an accepting part \(P \subseteq M\). We want to prove that \({f}^{-1}\left(L\right)\) is a regular language.
To that end, let us define \(N = (Q_T \to Q_T) \times (Q_T \to M)\), with the multiplication \((\delta, \lambda) \cdot (\delta', \lambda') :=(\delta \circ \delta', q \mapsto \lambda(q) \cdot \lambda'(q)))\). This is a finite monoid, where the pair \((\mathsf{id}, \mathsf{const}_1)\) is the identity element. Let us define \(\varphi(a) :=(\delta_T(a, \cdot), \lambda_T(a, \cdot))\) for all \(a \in \Sigma\). It defines a morphism from \(\Sigma^*\) to \(N\). Now, let us consider \(S :=\left\{ (\delta, \lambda) \in N \mid \lambda(\delta(q_0)) \in P \right\}\). It is an easy check that \(\varphi^{-1}(S) = {f}^{-1}\left(L\right)\), which proves that the latter is a regular language.
Let us now prove that \(f\) is Lipschitz for the prefix distance. Let us consider \(K :=\max \left\{ \left| \lambda_T(a,q) \right|_{} \mid a \in \Sigma, q \in Q_T\right\}\). It is an easy check that \(f\) is Lipschitz with constant \(K\).
Finally, let us prove that \(f\) preserves prefixes. Let \(u,v \in \Sigma^*\) be such that \(u \mathrel{\sqsubseteq_{\mathsf{prefix}}}v\). Because \((Q_T, q_0^T, \delta_T)\) is a deterministic automaton, \(f(v) = f(u) \cdot \lambda_T(\delta(q_0^T, u), u^{-1}v)\).
2.2.2 Solution of the hard implication
3 Cheat Sheet
3.1 Machines
3.1.1 Mealy Machine
Let \(\Sigma\) and \(\Gamma\) be two alphabets. A Mealy Machine \(\mathcal{M}\) is a tuple \((q_0, Q, \delta, \lambda)\) such that
- \(Q\) is a finite set of states.
- \(q_0 \in Q\) is the initial state.
- \(\delta \colon Q \times \Sigma \to Q\) is a transition function.
- \(\lambda \colon Q \times \Sigma \to \Gamma\) is an output function.
The semantics of a Mealy Machine is given by the following inductive equations: \[ \mathcal{M}(w) :=\mathcal{M}(q_0, w) \quad \mathcal{M}(q,\varepsilon) :=\varepsilon \quad \mathcal{M}(q,au) :=\lambda(q,a) \mathrel{\cdot}\mathcal{M}(\delta(q,a), u) \]
3.1.2 Flip-Flop Machine
A flip-flop machine is a Mealy Machine such that for all letters \(a \in \Sigma\), either \(\delta(\cdot,a)\) is the identity function, or it is a constant function. It is a binary flip-flop machine when \(Q = \{ 0,1 \}\).
3.1.3 Mealy Machine With Lookahead
Let \(\Sigma\) and \(\Gamma\) be two alphabets. A Mealy Machine with Lookahead \(\mathcal{M}\) is a tuple \((q_0, Q, \delta, \lambda)\) such that
- \(Q\) is a finite set of states.
- \(q_0 \in Q\) is the initial state.
- \(\delta \subseteq Q \times \Sigma \times Q\) is a transition relation.
- \(\lambda \colon Q \times \Sigma \times Q \to \Gamma\) is an output function.
In addition to this syntactic definition, we furthermore assume that for each \(w \in \Sigma^*\), there exists at most one path in the automaton \((q_0, Q, \delta)\) starting from \(q_0\) and reading \(w\).
The semantics of the Mealy Machine is given by considering potential runs of the machine. Because of the absence of ambiguity, it defines a partial map \(\mathcal{M} \colon \Sigma^* \rightharpoonup\Gamma^*\).
3.1.4 Sequential Functions
Let \(\Sigma\) and \(\Gamma\) be two alphabets. A sequential transducer \(A\) is a tuple \((q_0, Q, \delta, \lambda)\) such that
- \(Q\) is a finite set of states.
- \(q_0 \in Q\) is the initial state.
- \(\delta \colon Q \times \Sigma \rightharpoonup Q\) is a partial transition function.
- \(\lambda \colon Q \times \Sigma \to \Gamma^*\) is an output function.
The semantics is defined as for Mealy Machines.
Warning: this is sometimes called pure sequential functions.
3.2 Maths
3.2.1 Presburger Arithmetic
Formulas of the Presburger Arithmetic are built from the following grammar: \[ \begin{aligned} \varphi &:=\top \mid \bot \mid \varphi \land \varphi \mid \varphi \lor \varphi \mid \lnot \varphi \mid \exists x. \varphi \mid x = y + z \end{aligned} \]
Given a valuation \(\nu \colon \vec{x} \to \mathbb{N}\), we define the semantics of \(\varphi\) inductively as follows:
\[ \begin{aligned} \nu \models \top &\iff \text{true} \\ \nu \models \bot &\iff \text{false} \\ \nu \models \varphi \land \psi &\iff \nu \models \varphi \text{ and } \nu \models \psi \\ \nu \models \varphi \lor \psi &\iff \nu \models \varphi \text{ or } \nu \models \psi \\ \nu \models \lnot \varphi &\iff \text{ not } (\nu \models \varphi) \\ \nu \models \exists x. \varphi &\iff \text{ there exists } n \in \mathbb{N}\text{ s.t. } \nu[x \mapsto n] \models \varphi \\ \nu \models x = y + z &\iff \nu(x) = \nu(y) + \nu(z) \end{aligned} \]
3.2.2 Topology and Continuous functions
Let \(X\) be a set. A topology over \(X\) is a subset \(\tau\) of \(\mathop{\mathcal{P}}(X)\) closed under finite intersections and arbitrary unions. In a topological space \((X, \tau)\), the subsets in \(\tau\) are called open subsets, and their complement are called closed subsets.
A function \(f \colon (X, \tau) \to (Y,\theta)\) is continuous whenever for all open subset \(U \in \theta\), its pre-image \({f}^{-1}\left(U\right)\) is an open subset of \(\tau\). Equivalently, it is continuous if the pre-image of closed subsets are closed subsets.
3.2.3 Lipschitz functions
A function \(f \colon (X,d_X) \to (Y, d_Y)\) is Lipschitz if there exists a constant \(K \geq 0\) such that for all \(x_1,x_2 \in X^2\), \(d_Y(f(x_1), f(x_2)) \leq K d_X(x_1, x_2)\).
3.2.4 Prefix Distance
Let \(\Sigma^*\) be a finite alphabet. The prefix distance between two words \(u,v\) is \(\left| u \right|_{} + \left| v \right|_{} - 2 \left| w \right|_{}\) where \(w\) is the longest common prefix of \(u\) and \(v\).
3.2.5 Regular Topology
Let \(\Sigma\) be a finite alphabet. We equip \(\Sigma^*\) with a metric distance as follows: to a pair of words \(u,w\), we associate the minimal size \(s(u,w)\) of a deterministic automaton that separates \(u\) from \(w\). The distance between two words \(u\) and \(w\), is defined as \(d(u,w) :=2^{-s(u,w)}\). The regular topology is the topology defined by this metric on \(\Sigma^*\).
Equivalently, the regular topology is the coarsest topology containing the regular languages as closed subsets.