Diagnostics and Testing

Introduction

When implementing code it is important for the admin and developers to have easy methods to execute core features. This is implemented as standard and can be added to any extension.

Implementation

The diagnostics will appear on the manager

1618259696471

This is implemented by simply adding the following JSP to the extension

wex/jsp/utils/diags.jsp

As the following versions that need to replace the JSP the replacement JSP will not load as this tomcat server will not reload without a server the page can have a sequence 1, 2, 3 ... The latest number will take president.

wex/jsp/utils/diags1.jsp

This technique is used to replace JSP in other implementations and to avoid repetition of code an include is used 

<%@ include file="diags.jsp"%>

Notes

The diagnostics extension can be implemented in any form but a template with the example extension provides some CSS and JQuery libraries and templates

Example

The Diagnostic Example implements a simple diagnostics test

Testing

When developing an extension repeatable testing is essential, especially to verify functionality both for verification and validation. It is essential to differentiate these testing practices

Verification - Are we building the product right ?

Validation - Are we building the right product ?

We primarily focus on verification here as as that is required for regression and upgrade testing, whereas validation tends to be important for customer acceptance and sign off.

There are some very important elements to a good testing strategy

Extension Testing

Extensions are tested in the same way that all Windchill customization code should be tested, however due the nature of extensions it does raise some interesting questions. First let's discuss why we test? Clearly our overall objective is create flawless software. However anyone who as actually thought about this realizes that this is impossible, we need to have a minimum goal of fit for purpose and simply aspire to perfection but plan for a reasonable compromise. The question from a business perspective is where is the compromise.

Imperfect is not bad

Let's looks at the consequence of delivering imperfect, e.g flawed software. Sounds uncomfortable to us professionals but if we accept perfectis not achievable we must accept this and manage it. At this point we know our minimal goal is fit for purpose and in traditional models of software delivery we will not deliver software at this point in the development and testing lifecycle. Why? Well we often only have a single window of delivery, often known as a Go-Live and if we delivered flawed software, we can have unhappy users and also need to wait, in many cases months, until we can resolve even the smallest issue that caused the flaw. The tendency therefore is to over test and try to achieve flawlessness by testing (as we have seen an impossible task) and we extended both the delivery times and the length of time users have to wait, for what is sometimes, vital features.

High Iteration Delivery

Extensions allow all code to be hot loaded therefore a system shutdown is not required. This means that the level of acceptable can be lower than in traditional software, minor flaws can be corrected rapidly and if a major flaw is found the entire extension can be undeployed.

The only exception to this is if the extension causes system instability or data corruption, which should not occur as minimum testing should always be done

Other strategies can also be implemented, by using a simple configuration switch we can enable the extension to be visible only to a limited set of test users and confirm the validity of the extension before enabling it more generally. This is know as feature toggling and is used by many large enterprises.

Some clients with cluster configuration has a node that is configure to be a test node, and only accessible by a small number of test users. In this case we can install the extension only on this node, and deploy it on other nodes in the infrastructure one the new extension version has been validated.

The practice of limiting participation to a release is called a canary release.

Windchill Specific Testing

Windchill also raises some difficult problems when it comes to testing. It is a multi faceted system, meaning access to data may involve a combination of the following

Therefore creating a repeatable test environment is often a significantly greater problem that creating the running the tests, which is why automation although useful, tends not to have the same priority as in other forms of software testing.

Test Data

Test data can be add using the Load Extension Testing Automation. This can load both data and configuration e.g. Access control rules, and most importantly does it in a repeatable way and in many cases can be compatible with newer versions of Windchill.

Automation

Unit Testing

The extension framework fully support all forms of unit testing and with the Load Extension this can be automated to the point of loading the data automatically to run the tests. However in many case the ROI for unit testing is low, often extensions have limited use of complex standalone algorithms that benefit from unit testing. Code will often read or create Windchill data, making the repeatable unit test complicated to manage.

UI Testing

Frameworks like selenium can be useful for making repeatable test cases, and again are full compatible with the extension framework. However as extensions by their nature have a fast iteration and deployment lifecycle, the discipline and cost required to maintain these scripts can be prohibitive and return limited ROI.