Fastest Tool

In this paper we describe the functional features and the architecture of a tool implementing the Stocks-Carrington framework (TTF) for model based testing (MBT). The resulting prototype, called Fastest, makes it easy to generate test cases from Z specifications. We not only apply the ideas of the referred framework but we also use a technique based on finite models to find test cases, which has proved to increase the level of automation during the whole testing process. The paper also discusses problems and challenges that have appeared during the development of the tool, and introduces real case studies and an analysis of the results obtained so far.

Fastest is an automatic implementation of Phil Stocks and David Carrington's Test Template Framework (TTF), a model-based testing (MBT) framework for the Z formal notation. In this paper we present a new feature of Fastest that helps TTF users to eliminate inconsistent test classes automatically. The method is very simple and practical, and makes use of the peculiarities of the TTF. Perhaps its most interesting features are extensibility and ease of use, since it does not assume previous knowledge on theorem proving. Also we compare the solution with a first attempt using the Z/EVES proof assistant and with the HOL-Z environment. At the end, we show the results of an empirical assessment based on applying Fastest to four real-world, industrial-strength case studies and to six toy examples.

Model-based testing (MBT) generates test cases by analysing a formal model of the system under test (SUT). In many MBT methods, these test cases are too abstract to be executed. Therefore, an executable representation of them is necessary to test the SUT. So far, the MBT community has focused on methods that automate the generation of test cases, but less has been done in making them executable. In this paper we propose a language to specify rules that can be automatically applied to produce an executable representation of test cases generated by the Test Template Framework (TTF), a MBT method for the Z notation.

Critical software most often requires an independent validation
and verification (IVV). IVV is usually performed by domain experts, who are
not familiar with specific, many times formal, development technologies.
In addition, model-based testing (MBT) is a
promising testing technique for the verification of critical software. Test
cases generated by MBT tools are logical descriptions. The problem is, then,
to provide natural language (NL) descriptions of these test cases, making
them accessible to domain experts.
In this paper, we present ongoing research
aimed at finding a suitable method for generating NL descriptions from test
cases in a formal specification language. A first prototype has been developed and
applied to a real-world project in the aerospace sector.

We have applied Fastest, an implementation of the Test Template Framework, to five real case studies of aerospace software. This involved the formalization in the Z notation of nontrivial parts of each system. One of these models, for instance, formalizes a significant portion of the ECSS-E-70-41A aerospace standard. The models were then fed into Fastest, which automatically generated detailed functional abstract test cases. Since these test cases are independent of any implementation, they can be used to test any of them. Furthermore, we were able to semi-automatically translate them into English so they can be used by domain experts performing independent validation and verification activities.

Fastest is a model-based testing tool for the Z notation providing an almost automatic implementation of the Test Template Framework. The core of this document is an example showing how to use Fastest to automatically derive abstract test cases from a Z specification.

Este proyecto trata sobre la automatización y extensión del Test Template Framework (TTF). El TTF es un método de testing basado en modelos (MBT) especialmente orientado a testing de unidad a partir de especificaciones Z. Aunque el TTF es un método sólido y fue ampliamente estudiado desde su primera publicación, la comunidad de MBT fue perdiendo interés enél. Nosotros creemos que esto se debió, al menos en parte, a la falta o dificultad aparente en dotarlo de un apoyo herramental. De hecho, algunos han sugerido que la generación de casos de prueba abstractos siguiendo el TTF es una actividad manual que requiere que los usuarios manipulen predicados complejos. La intención de este proyecto es mostrar que estas conclusiones son al menos dudosas, implementando una herramienta, llamada Fastest. Fastest no solo es capaz de producir automáticamente casos de prueba abstractos sino que además podría cubrir las necesidades de la comunidad Z en relación a herramientas de MBT.