Jaca - a software fault injection tool

2017

The increasing complexity of software systems challenges the assurance of the likewise increasing dependability demands. Software fault injection is a widely accepted means of assessing dependability, but far less accessible and integrated into engineering practices than unit or integration testing. To address this issue, we present a dataset of existing fault injection tools in a programmatically evaluable model. Our Fault Injection ADvisor (FIAD) suggests applicable fault injectors mainly by analyzing definitions from Infrastructure as Code (IaC) solutions. Perspectively, FIAD can yield findings on how to classify fault injectors and is extensible in a way that it can additionally suggest workloads or run fault injectors. Keywords–fault injection; infrastructure as code; testing; service-oriented systems; distributed systems.

… , 2006. ICOCI'06. …, 2006

In line with market demands and the need for technological innovations, designing and implementing software and hardware components for computing systems is growing in complexity. In order to cope with such complexity whilst meeting market needs, engineers often rely on design integration with commercial-of-the-shelf-components (COTs). In the case where lives and fortunes are at stake, there is a need to ensure dependability of COTs in terms of their robustness before they can be adopted in such an environment. However, ...

2009

Dependability is a fundamental property of computer systems operating in critical environment. The measurement of dependability (and thus the assessment of the solutions applied to improve dependability) typically relies on controlled fault injection experiments that are able to reveal the behavior of the system in case of faults (to test error handling and fault tolerance) or extreme input conditions (to assess robustness of system components). In our paper we present an Eclipse-based fault injection framework that provides a model-based approach and a graphical user interface to specify both the fault injection experiments and the run-time monitoring of the results. It automatically implements the modifications that are required for fault injection and monitoring using the Javassist technology, this way it supports the dependability assessment and robustness testing of software components written in Java.

2006

Abstract In a network consisting of several thousands computers, the occurrence of faults is unavoidable. Being able to test the behaviour of a distributed program in an environment where we can control the faults (such as the crash of a process) is an important feature that matters in the deployment of reliable programs. In this paper, we investigate the possibility of injecting software faults in distributed Java applications. Our scheme is by extending the FAIL-FCI software.

2003

This paper describes the ATIFS, a testing toolset which supports the activities of black-box tests for reactive systems, especially communication systems. In ATIFS, two types of testing are carried out: conformance testing and software fault injection. These testing types allow one to answer such questions about the system under test as: "does the system perform what is specified?", as well as "for how long does the system perform what is specified?" and "how does the system behave in the presence of faults in its environment?". This toolset was conceived and implemented aiming at providing a user with facilities for the activities of test case derivation, test execution and test result analysis. The general requirements that guided the ATIFS development, its architecture and an overview of the already implemented tools are focused on in this paper. The main tools were successfully used in the conformance tests of a real space application: telemetry rec...

Lecture Notes in Computer Science, 2010

Software model checkers are being used mostly to discover specic types of errors in the code, since exhaustive verication of com- plex programs is not possible due to state explosion. Moreover, typical model checkers cannot be directly applied to isolated components such as libraries or individual classes. A common solution is to create an ab- stract environment for a component to be checked. When no constraints on component's usage are dened by its developers, a natural choice is to use a universal environment that performs all possible sequences of calls of component's methods in several concurrently-running threads. However, model checking of components with a universal environment is prone to state explosion. In this paper we present a method that allows to discover at least some concurrency errors in component's code in reasonable time. The key ideas of our method are (i) use of an abstract environment that performs a random sequence of method calls in each thread, and (ii) restarts of the error detection process according to a specic strategy. We have implemented the method in the context of Java components and the Java PathFinder model checker. We have performed experiments on non- trivial Java components to show that our approach is viable.

Arxiv preprint cs/0101002, 2001

Correctness constraints provide a foundation for automated debugging within object-oriented systems. This paper discusses a new approach to incorporating correctness constraints into Java development environments. Our approach uses the Object Constraint Language ("OCL") as a specification language and the Java Debug Interface ("JDI") as a verification API. OCL provides a standard language for expressing object-oriented constraints that can integrate with Unified Modeling Language ("UML") software models. JDI provides a standard Java API capable of supporting type-safe and side effect free runtime constraint evaluation. The resulting correctness constraint mechanism: (1) entails no programming language modifications; (2) requires neither access nor changes to existing source code; and (3) works with standard off-the-shelf Java virtual machines ("VMs"). A prototype correctness constraint auditor is presented to demonstrate the utility of this mechanism for purposes of automated debugging. BoundedStack::push(Object obj):Object Pre: size() < capacity() Post: size() = [email protected]()+1 (where "=" denotes equality) v.lastElement() = obj result = v.lastElement() BoundedStack::pop():Object Pre: not empty() (where "not" denotes logical negation) Post: result = [email protected]() size() = [email protected]()-1 BoundedStack::peek():Object Pre: not empty() Post: result = v.lastElement() v = v@pre BoundedStack::empty():boolean Post: result = (v.size() = 0) BoundedStack::size(): int Post: result = v.size() BoundedStack::capacity(): int Post: result = v.capacity()

Dependability Engineering, 2018

Software fault injection (SFI) is an acknowledged method for assessing the dependability of software systems. After reviewing the state-of-the-art of SFI, we address the challenge of integrating it deeper into software development practice. We present a well-defined development methodology incorporating SFI-fault injection driven development (FIDD)-which begins by systematically constructing a dependability and failure cause model, from which relevant injection techniques, points, and campaigns are derived. We discuss possibilities and challenges for the end-to-end automation of such campaigns. The suggested approach can substantially improve the accessibility of dependability assessment in everyday software engineering practice.

Failure is always an option; in large-scale data management systems , it is practically a certainty. Fault-tolerant protocols and components are notoriously difficult to implement and debug. Worse still, choosing existing fault-tolerance mechanisms and integrating them correctly into complex systems remains an art form, and programmers have few tools to assist them. We propose a novel approach for discovering bugs in fault-tolerant data management systems: lineage-driven fault injection. A lineage-driven fault injector reasons backwards from correct system outcomes to determine whether failures in the execution could have prevented the outcome. We present MOLLY, a prototype of lineage-driven fault injection that exploits a novel combination of data lin-eage techniques from the database literature and state-of-the-art satisfiability testing. If fault-tolerance bugs exist for a particular configuration, MOLLY finds them rapidly, in many cases using an order of magnitude fewer executions than random fault injection. Otherwise, MOLLY certifies that the code is bug-free for that configuration .

Fault injection is a pivotal technique in dependability benchmarking. Unfortunately, existing general-purpose fault injection tools either inject faults in predetermined memory locations or resort to random injection, approaches that generally result in poor fault coverage and controllability guarantees. This makes it difficult to reproduce or compare experiments across different systems or workloads. This paper presents EDFI, a new tool for dependable generalpurpose fault injection experiments. EDFI combines static and dynamic program instrumentation to perform execution-driven fault injection, a technique which allows realistic software faults to be injected in a controlled way as the target system executes. Our instrumentation strategy guarantees a predetermined faultload distribution during the entirety of the experiment, independently of the particular system or workload considered. Our evaluation confirms that EDFI significantly improves the precision and controllability of prior tools, at the cost of only modest memory and performance overhead during fault-free execution.