Testing with Holodeck

Holodeck's powerful features and versatile design allows you to test your application for fragility in a variety of ways, including:

• Investigative Testing
• Stress Testing
• Code Coverage Testing
• Ad-hoc/Exploratory Testing
• Fault-injection Testing
• Application Analysis and Debugging
• Environment Simulation
• Network and File Corruption

INVESTIGATIVE TESTING

Starve your application of resources and get faster answers to the fundamental questions:

• What are your application's dependencies?   Files, Registry keys, DLLs, Memory, Disk Space, Network Bandwidth?  What happens if you take any of these away?
• What system API calls does your application make? What data is being passed; What happens if an api fails? Is a hard to track bug being caused by an environmental failure? Are there performance issues?
• What are the changes, from build to build?

STRESS TESTING

By turning on Holodeck in the background while running a stress test suite, you can dramatically improve the bug-finding ability of your tests. Holodeck will inject faults, modify dependencies,  and corrupt resources as your stress test runs.  If a bug is found, Holodeck will record the activity in a way that allows the bug to be reproduced.   For stress testing, Holodeck expects a stress test harness to launch it, telling it the stress options and Application Under Test on the command line.

CODE COVERAGE TESTING

Increase your code coverage through iterative fault injection.  Give Holodeck a set of tests and it will observe application behavior and then automatically apply faults, limits, and tests to hit every error code path available.  For code coverage testing, Holodeck expects to be launched by a test harness as well. If there is a crash in the Application Under Test, the harness is responsible to do the right thing (launch debugger, record crash, etc)

DIRECTED AD-HOC TESTING

Use the Holodeck UI to manually set limits, faults and run tests that you think will break your application.

• Set common faults such as Common disk failures, Common memory failures, Common network failures, which are hard to use without Holodeck
• Fail individual API calls.  Can be targeted to specific usage by matching parameter values.  Discover what happens if you fail a LoadLibrary or CreateFile call, or if you fail a memory allocation
• Use limits to deprive your application of resources.  Memory limit. Disk limit. Network limit
• Use corruption to test file and network packet parsing.  Corrupt randomly, with find and replace, or with regular expressions

RUN-TIME FAULT INJECTION/HOSTILE ENVIRONMENT TESTING

Holodeck can be used in a lab environment where you can observe and document the behavior of your application in depth as you scout out the most vulnerable areas.  Take your new found knowledge of your application and define tests that will fire only under the circumstances you dictate, corrupt the files and network packets that your application is dependent upon, trace data as it makes its way through the system, and then define the most powerful buffer overrun tests your application has ever had to endure.   You can also use Holodeck's generated code templates in your development platform to easily inject your own code into your application's process space, intercepting any api call you choose.

APPLICATION ANALYSIS AND DEBUGGING

• Track and manipulate registry, file, memory, network, dll, and exe dependencies
• Track data usage across public interfaces
• Track environmental factors in difficult to reproduce bugs

ENVIRONMENT SIMULATION

Simulate hostile and unpredictable environments, and common environmental failures.

NETWORK AND FILE CORRUPTION