Automated Testing in 4D

I’m back to 4D after 3 years doing other projects. And I am really missing automated testing (unit tests, functional tests). Automated tests make refactoring with confidence possible, they help catch regression errors, and they document the expected behavior of the code.

What is the state of the art in 4D automated testing?

Currently as I know it there is no formal 4D testing libraries. Like PHPUnit, Jasmine, Mocha etc. There was a basic Testing library from back in the V11 days that used to exists but the developer has long since abandoned it. If one could locate it again it may be a good place to restart a 4D testing framework.

Assertions are available and can be used to build unit tests.

https://doc.4d.com/4Dv18/4D/18/ASSERT.301-4504608.en.html

Personally I just build a unit test for each function I want to test. I just end up naming the function TEST_ and then set a script to run them. The difficulty is that much of the legacy 4D code is not written in such a way as to make using testing work well. With the new ORDA functions and code I am hoping to see a resurgance in the ability to do unit testing. But that would require likely a library to allow such tests to be run.

Another tool is

https://ch-de.4d.com/ajtoolsunittest

But I didn’t use it yet.

2 Likes

Indeed, for anyone with a legacy 4D app, support for Automated Testing is the single most important feature currently missing from 4D. Without it, major refactoring is not a viable option. I’ve managed development for two legacy applications where refactoring was sorely needed, but was not possible because of the massive cost of manual QA testing. As Eric alluded, new apps can be written in ways which support unit testing, and if you design your UI appropriately, you can simulate interacting with the UI via code, but that needs to be designed in from the beginning.

I read of 4D developers who periodically “re-write” their apps and I’ve often wondered what they do to ensure sufficient regression test coverage.

4D Professional Services offers consulting services where they can provide automated testing for your 4D app, but it’s not something that can be managed by a developer.

Maybe it won’t matter as our 4D apps move to web UI and leave 4D UI behind as there are many test automation tools for web apps? I haven’t watched the Summit videos yet, but maybe that’s the ultimate goal of 4D?

Tom Benedict

Why is it not something that can be managed by a developer? I’m curious about what tools they are using that does not include 4D.

I hope that is not the ultimate goal of 4D. 4D’s biggest advantage is cross platform native Mac and Windows UI. There are a lot of better options for web only.

1 Like

Hi,

At the 2013 summit (Paris), I did a presentation on Refactoring, I introduced the necessity of unit test (from books by Martin Fowler) and I presented a component to do unit testing.

The component was based/inspired from a tool written by Rob Laveaux. I have it available in v15.

People often confuse unit test and automated testing… it is not the same.

In unit test, the developer write the test before coding a module (or before refactoring). Unit test are best to test a function. In unit tests you defined the input and declare the expected output. The good thing about thinking in test driven development way is that it makes you think of test from the ground up. For instance, if you write a function to calculate the age, it will be difficult to test if you don’t provide the “today” date (as an optional parameter). I often get frustrated when a technical function cannot be tested without having to go through many screens and a specific set of data.

It is more complicated to add unit test when the code has not been written in the “test driven” approach.
To be honest, I use unit tests on few occasions (it is a gut feeling). The most work I have done in unit test is around a ociLib component (using 4D for oci). it takes time to write but it improves quality and it improves your confidence when you want to refactor some code to improve/optimize code, change data model (and helps finding out regression caused by a change in the environment, i.e. new 4D release, new OS, etc…).

The recommandation for unit test is break down your code in testable modules and functions. When you have some UI, choose a design approach where the logic can be tested independently from the UI. It is easy to test a function, it is complicated to test a UI.
Objets make it easier to write testable code (you provide all the input as a single objet and you check the result).

To test the UI using automated tests, you need a tool and you need to “teach” your tool the test scenario. And that can break if you move a button… I can imagine how difficult and expensive this is.

Then, the part where I struggled, it testing a function which relies on complex data sets (many tables, many records, etc…). Manipulating (i.e… creating a specific dataset) to set the unit test is hard work.

It comes down to code quality and the amount of money the client is prepared to pay for its product.

Finally, it is easy to do a self test function to run at the start of your application where you call sensitive functions with expected results using the ASSERT command. And apply the “fail fast, fail early” and stop the application if something does not behave like expected.

This is an example :

C_LONGINT($vl_epoch)
SET ASSERT ENABLED(True)
ASSERT(CURL_dateStringToEpoch ("Sun, 06 Nov 1994 08:49:37 GMT";->$vl_epoch))
ASSERT($vl_epoch=784111777)
ASSERT(CURL_dateStringToEpoch ("Sunday, 06-Nov-94 08:49:37 GMT";->$vl_epoch))
ASSERT($vl_epoch=784111777)
ASSERT(CURL_dateStringToEpoch ("Sun Nov  6 08:49:37 1994";->$vl_epoch))
ASSERT($vl_epoch=784111777)
ASSERT(CURL_dateStringToEpoch ("06 Nov 1994 08:49:37 GMT";->$vl_epoch))
ASSERT($vl_epoch=784111777)
ASSERT(CURL_dateStringToEpoch ("06-Nov-94 08:49:37 GMT";->$vl_epoch))
ASSERT($vl_epoch=784111777)
ASSERT(CURL_dateStringToEpoch ("Nov  6 08:49:37 1994";->$vl_epoch))
ASSERT($vl_epoch=784111777)
ASSERT(CURL_dateStringToEpoch ("06 Nov 1994 08:49:37";->$vl_epoch))
ASSERT($vl_epoch=784111777)
ASSERT(CURL_dateStringToEpoch ("06-Nov-94 08:49:37";->$vl_epoch))
ASSERT($vl_epoch=784111777)
ASSERT(CURL_dateStringToEpoch ("1994 Nov 6 08:49:37";->$vl_epoch))
ASSERT($vl_epoch=784111777)
ASSERT(CURL_dateStringToEpoch ("GMT 08:49:37 06-Nov-94 Sunday";->$vl_epoch))
ASSERT($vl_epoch=784111777)
ASSERT(CURL_dateStringToEpoch ("94 6 Nov 08:49:37";->$vl_epoch))
ASSERT($vl_epoch=784111777)
ASSERT(CURL_dateStringToEpoch ("1994 Nov 6";->$vl_epoch))
ASSERT($vl_epoch=784080000)
ASSERT(CURL_dateStringToEpoch ("06-Nov-94";->$vl_epoch))
ASSERT($vl_epoch=784080000)
ASSERT(CURL_dateStringToEpoch ("Sun Nov 6 94";->$vl_epoch))
ASSERT($vl_epoch=784080000)
ASSERT(CURL_dateStringToEpoch ("1994.Nov.6";->$vl_epoch))
ASSERT($vl_epoch=784080000)
ASSERT(CURL_dateStringToEpoch ("Sun/Nov/6/94/GMT";->$vl_epoch))
ASSERT($vl_epoch=784080000)
ASSERT(CURL_dateStringToEpoch ("Sun, 06 Nov 1994 08:49:37 CET";->$vl_epoch))
ASSERT($vl_epoch=784108177)
ASSERT(CURL_dateStringToEpoch ("06 Nov 1994 08:49:37 EST";->$vl_epoch))
ASSERT($vl_epoch=784129777)
ASSERT(CURL_dateStringToEpoch ("Sun, 12 Sep 2004 15:05:58 -0700";->$vl_epoch))
ASSERT($vl_epoch=1095026758)
ASSERT(CURL_dateStringToEpoch ("Sat, 11 Sep 2004 21:32:11 +0200";->$vl_epoch))
ASSERT($vl_epoch=1094931131)
ASSERT(CURL_dateStringToEpoch ("20040912 15:05:58 -0700";->$vl_epoch))
ASSERT($vl_epoch=1095026758)
ASSERT(CURL_dateStringToEpoch ("20040911 +0200";->$vl_epoch))
ASSERT($vl_epoch=1094853600)
2 Likes

Hello Hunter,

We have a create AJ_Tools_UnitTest which is a component developed with 4D v17 R5. Its purpose is to give 4D developers a way to do unit test and partial integration tests within 4D.

Here the link to our Github repository AJAR Project - GitHub

3 Likes

Yes Bruno, well explained! But just a little precision, Unit Test is different from TDD (Test Driven Development). You can do UT without TDD! UT is the application of tests you wrote during TDD before writing code, or test you wrote once code written :slight_smile:
TDD is a very good approach, but it has a cost. And that’s why it has a value!

Thinking TDD means you think on User cases before writing a line of code.
Then the code is easy to write, as you know where are the edges and exceptions.

Yes, I used in the past a component to do your own UT. It works well, but the only thing is that you overload 4D by writing an UT method for a method. If you want to check every method, then every method will be in two, and will be included in the final compiled product!

I used one in the past, which is UnitTester. Works well. But after some time I gave up taking that time to use UT. But I still do TDD on paper before writing !

There is ajar product too I haven’t used it yet.

4D Professional Services has a proprietary component which works with scripts created with AutoIT, a freeware scripting language designed for automating Windows GUI testing.

I misspoke when I said this could not be managed by a developer. There is no reason a developer could not engineer a similar solution.

Tom Benedict

This in particular is where I’m struggling. Testing functions that don’t test the database is possible with and without a framework. But testing code that uses the database, basically everything important, is hard. In other languages/databases I’m able to create test data in code and have the database built and destroyed automatically with each test or set of tests. This way I can run the same tests over and over without screwing up the data and making the tests unreliable.

Is there a way to do that in 4D? Create a database, populate it, use it, and destroy it all in code from within 4D?

1 Like

As I said, I’ve been gone from 4D for a while and I can’t remember: Can I use a v17 R5 component in a 17.3 project? Or will I need to wait until we get into v18?

No you can’t. Sorry.

V17R5 is the minimal version you can use.

Thanks for the info. I’ll add to the list of things to look at as we migrate to v18.

1 Like

In v18, as I understand it, there’s no reason that you can’t generate a complete database via a script. Prior to v18 you could generate a schema via a SQL script, populate resulting tables via 4D or SQL scripts and create methods from text files. Forms were harder to generate. I’ve never done this on a large scale, but in v13 I once used this concept to build a simple database which illustrated a bug I was reporting to 4D Tech Support.

Another thing I’ve done, in a similar vein, is to create configuration scripts (methods) which set up all the data needed to test a feature. That script could behave differently depending on context (Dev, Test, Staging, Production) and it could run automatically at startup, if needed. It worked like a cheap melding of TDD (Test Driven Design) and CM (Configuration Management). Scripts take a little bit of extra time to create and test, but that effort pays off very quickly compared to manual configuration via UI.

So, the answer to your question is “Yes!”

Tom Benedict

1 Like

Using the already mentioned https://github.com/markschaake/UnitTester4D each test case is run in the context of a transaction. A test case consists of a setup phase where the transaction starts, the test phase itself with various tests and a tear-down phase at the end. In the tear-down phase the transaction is cancelled.
The component offers a convenient method to create new data records. Yes it could be bunch of work to write such tests, but writing tests is not for free, you save the time afterwards as already mentioned in this topic.

If you’re interested in automated UI testing, Squish from FrogLogic works well with 4D and lets you create scripts by example or by programming (Python). Eggplant is another that lets you do the same kinds of things.

Squish is an important part of testing every release we do - we can quickly catch regressions and errors that a user might face, as well as collect timings from version to version to make sure we’re not slowing down, etc…

3 Likes

Thanks for the tool suggestions. I’ll take a look at them.