3rd Generation Frameworks : Keyword-driven

Third Generation frameworks are the most defined frameworks, and should be considered when the automation scope is relatively high, and when at least one of the automation resources has strong technical and logical skills, and a proficiency in the automation tool being used to develop the framework. The two frameworks that fit into this generation include Keyword-driven and Model-based frameworks.

Model-based PDF Print E-mail

Often called an “Intelligent framework”, Model-based frameworks go beyond creating automated tests that are executed by the tool. These frameworks are typically “given” information about the application, and the framework “creates” and executes tests in a semi-intelligent manner. Test automators describe the features of an application, typically through state models which depict the basic actions that may be performed on the application, as well as the broad expected reactions. Armed with this information the framework dynamically implements tests on the application.


Model-based Pros

* Increased coverage overtime – With a minimal amount of scripting, a lot of the application may be tested. The coverage may not necessarily be extremely high per execution, but given the random nature of the testing, the application test coverage will increase over time.
* Increased application exploration – Test automation is typically created to perform a specific set of test sequences with each test execution. Model-based test automation frameworks contain a certain degree of artificial intelligence, which allows it to perform an almost exploratory type of testing, in a way that is similar to how a manual tester might explore the application.
* Increased potential for defects discovery – Test automation is normally not meant to uncover a lot of new defects. Test automation is normally used to ensure existing functionality still works. The exploratory nature of Model-based frameworks, however, increase the chances of new defects being uncovered, given the fact that new ground is covered with each automated test run.


Model-based Challenges (Con)

* Requires higher degree of application knowledge – In order to maintain Model-based frameworks, a higher degree of application knowledge is required. This is directly related to the fact that there is less maintenance of automated tests (which is largely a technical activity), and more maintenance of application models that define application behavior.
* Required technical expertise – The technical skills required to create and maintain Model-based frameworks is relatively high. There are numerous dependencies and relationships that must be understood and maintained, as well as advanced tool components and structures.
* Increased management support – Management support is probably the most challenging with Model-based frameworks. Mainly because it is a lot simpler to calculate and communicate ROI with other frameworks. Model-based ROI calculations will more than likely be based largely on risks. Risk ROI calculations alone are often very difficult to convey to management as a justification for the time and resources necessary for creating and maintaining the structures, documentation, and personnel (both technical and non-technical).

Keyword-driven PDF Print E-mail

Often called “Table-driven”, this framework tends to be more application-independent than other frameworks. This framework processes automated tests that are developed in data tables with a vocabulary of Keywords (aka. 'Action' words) that are independent of the automated test tool used to execute them.


Screen Object Action Value Recovery Comment
LoginScreen Username Input “John”
LoginScreen Password Input “JPass”
LoginScreen Login Click
Welcome Verify_Screen Abort_Test



The keywords are associated with application-specific and application-independent functions and scripts that interpret the keyword data tables along with its application-specific data parameters. The automated scripts function as Drivers to execute the interpreted statements in the application under test.


1 Open Keyword File
2 Execute Data rows to end of file (EOF)
3 If == Input Then
4 Call Input function with Screen, Object and Value parameters
5 If == Click Then
6 Call Click function with Screen and Object parameters
7 If == Verify_Screen Then
8 Execute Verify_Screen function with Screen parameter
9 Implement Exception Handling Routine Based on Pass/Fail Status
10 End Loop


Keyword Driven frameworks rely heavily on Functional Decomposition and Data Driven framework concepts. The sample driver script above opens the Keyword file, then reads each of the rows in the file. In each row, it reads the Action statement, then calls the appropriate function. The arguments of the function are based on other fields in that row.


Keyword Driven Pros

* Increased Reusability – Reusability is even further increased with a Keyword Driven framework, because most of the functions are created in such a way to not only be reusable for multiple tests within a single application, but also for tests across multiple applications. Redundancy may therefore be decreased across all applications that an organization may be responsible for automating.
* Earlier script development – This framework increases the ability for automation to begin before the application is delivered. Using information gathered from Requirements or other documentation, keyword data tables can be created that mirror corresponding manual test procedures.
* The script is easier to read – Keyword data tables are often easier to read than regular test scripts, because the keyword data tables mirror manual test procedures. The keywords are typically verb-like phrases that make reading a keyword data table similar to reading a collection of sentences, which is easier than reading code statements that don’t mirror natural language.
* Increased standardization – The increase of reusable framework components is followed by increased standardization. The added advantage that the Keyword Framework has over Functional Decomposition, however, is that standards are by default imposed through the implementation of a Keyword framework. Provided that the framework components are created with appropriate standards, they will be invoked in the keyword data tables with every use of the framework’s keywords. Standardization helps with script maintenance, because it decreases guess work involved in figuring out what a script does and how best to fix it.
* Error handling easier to introduce – Patch work error handling solutions on a script by script basis are difficult to introduce and maintain. With reusable Keyword framework components, error handling can be introduced that reaches multiple scripts across multiple applications. This will ultimately improve the effectiveness of unattended execution of the test scripts.
* Decreases technical nature of application automation – Working with keyword data tables for everyday application automation is a lot less technical than working with code statements. Therefore individuals that are not as technical can be brought onto the team to help create automated tests.
* Greater traceability to manual test cases – Given the fact that keyword data tables so closely resemble a manual test procedure, it becomes simpler to trace actions in automated tests to actions in manual tests. In addition, there will be greater reuse of manual test procedures.


Keyword Driven Challenges (Cons)

* Required technical expertise – While the technical nature of creating automated tests within the framework is decreased, the technical skills required to create and maintain the framework itself is greatly increased from that required for Functional Decomposition frameworks. There are numerous dependencies and relationships that must be understood and maintained, as well as advanced tool components and structures.
* Intuition is less useful – In order to effectively implement a Keyword framework, reliance on intuition must be reduced, while reliance on standards must be increased. While some standards are automatically imposed, with this type of framework, many standards are not, so there’s an ongoing effort to ensure resources are aware of standards, understand them, and are able to effectively implement them. Increased documentation will probably be required to identify framework features, particularly documentation that chronicles the keywords that exist as part of the framework that may be used.
* Increased management support – Management support is a challenge for any automation effort, but it is particularly difficult with Keyword Driven frameworks. There must be strong management support, however, for the time and resources necessary for creating and maintaining the structures, documentation, and personnel (both technical and non-technical).
* Restrictive for technical staff – For technically adept resources that are tasked with day-to-day automation of a software application, Keyword frameworks may be overly restrictive. They may be perceived as an entity that “ties their hands” into automating in a “standard” way at the expense of automating in the most efficient way for a particular application, or particular feature within an application. Keyword applications typically require increased “public relations” work to sale the approach to both resources and management.