We develop new results about a sieve methodology for estimation of minimal state spaces and probability laws in the class of stationary categorical processes. We first consider finite categorical spaces. By using a sieve approximation with variable length Markov chains of increasing order, we carry out asymptotically correct estimates by an adapted version of the Context Algorithm (see Rissanen (1983)). It thereby yields a nice graphical tree representation for the potentially infinite dimensional minimal state space of the data generating process. This procedure is also consistent for increasing size countable categorical spaces. Finally, we show similar results for real-valued general stationary processes by using a quantization procedure based on the distribution function.