Speaker: Rachel Tzoref-Brill
 
Title: Differencing and Visualization Techniques for Combinatorial Models 

Abstract: 

Combinatorial test design (CTD) is an effective test design technique, considered to be a testing best practice. CTD provides automatic test plan generation,  but it requires a manual definition of the test space in the form of a combinatorial model, consisting of parameters, their respective values, and constraints on value combinations. As the system under test evolves, e.g., due to iterative development processes and bug fixing, so does the test space, and thus, in the context of CTD, evolution translates into frequent manual model definition updates. Manually reasoning about the differences between versions of real-world models following such updates is infeasible due to their complexity and size. Moreover, representing the differences is challenging. 

In this talk we will describe two different technologies for better comprehension of combinatorial models and their evolution. 
First, we propose a syntactic and semantic differencing technique for combinatorial models. We define a canonical representation for differences between two models, and suggest a scalable algorithm for automatically computing it. We further use our differencing technique to analyze the evolution of 42 real-world industrial models. As part of the analysis we identify6 change patterns that occur in combinatorial models evolution. The analysis provides evidence for the potential usefulness of our differencing approach. 

In the second part of the talk, we will describe and demonstrate three techniques for visualizing combinatorial models and the test plans derived from them. The three techniques are based on graphs, tables, and treemaps, and are used to visualize different aspects of the models and test plans, such as the relationships between the parameters and the constraints, the relationships between the tests in the derived test plan, the degree of uniqueness of each test, the degree of legality of each parameter combination, and its degree of coverage in a derived test plan. All visualization techniques were implemented in IBM Functional Coverage Unified Solution (IBM FOCUS), an industrial-strength CTD tool.

Joint work with Aya Chayat, Shiri Ladelsky, and Shahar Maoz.