Integration Test and Functional Test in plone testing explained


As a plone developer, I have to do writing lots of unittests for our projects, so I had concerned on plone testing documentation from plone’s official site. Usually I learnt from there about two types of testing, `Integration Testing` and `Functional Testing`. Also knew about Testing Layer and how layer fixture is used while making above kind of tests, in other words which layer fixture is appropriate for which testing, i.e for Functional Testing should be used Functional Testing Layer fixture.

Integration Testing Fixture is generally creating a transaction for each test and just abort the transaction after test being done. Single database(demo storage) is whole test layer lifecycle,  there is not possible to do any persistent activity (transaction commit) in side tests code. Beside Functional Testing Fixture provides database (demo storage) for each test and removed during test teardown (making a stack of database for each test and pop up during tear down process), full transaction lifecycle is happened here, even you could do manually commit and/or use of sub-transaction as well.

By naming (Integration Testing, Functional Testing ) and characteristics  of both, I got confused! my thinking was integration test means collection of components (could be some functions from a module or coming from other modules or api methods from different packages) whether working together perfectly, so here we need persistent features (transaction commit) as we might need to add fixture data and should be available outside of session. For functional test, i thought that this about single component (for example: test single function or method of class), so here we don’t need transaction features, but I was completely wrong (my thought was so stupid)

To get rid out of my confusion, I have planned to dig down into online resources. Very first  I am trying understand that what are meaning of Integration test and Functional test , so ask google and found out from stackoverflow what I was looking for! about Functional testing:

Functional testing is when you test the system against the functional requirements of the product. Product/Project management usually writes these up and QA formalizes the process of what a user should see and experience, and what the end result of those processes should be. Depending on the product this can be automated or not.

Now i have better about those terms and why need persistent feature for Functional Testing

When and where should use each testing type

Integration Testing Fixture

This is relatively faster, you should use it as much as possible, until your test needs (commit) persistent feature. Ideal for testing of methods/functions, where you want to test how those are working together.

Functional Testing Fixture

Primary use case is, when you need to test/monitor after commit reaction of each function/component and make sure data goes to database successfully. If you have SqlAlchemy’s transaction integration, this functional testing fixture is mandatory for you, as some exception/error is thrown by SqlAlchemy during data going to be committed. i.e Integrity violation. Another required case is, when your test will need to use test browser, there you must need completion of transaction lifecycle to avail saved data during assertion. Test browser to be working,  you have to use ZServer Layer Fixture (from that is making test server for you.

Naturally Functional testing is slower compared to Integration testing , even it will be getting more slower if you attach ZServer fixture. In my previous experiences, i made functional test fixture with integrating zserver fixture and use this fixture for all of tests (i didn’t care if persistent(transaction commit) is required or not, test browser is needed or not!. ), one of cause, there was my lack of knowledge as result small test took so much time. Now i am learning and have some opinion from me! You should make two functional testing fixture, one without zserver fixture and one with zserver. So if any test needs test browser you will go for with zserver fixture. Bellows are example

from plone.testing import z2
from import FunctionalTesting
from import IntegrationTesting

MY_FIXTURE = MyFixtureLayer()
MY_INTEGRATION_TESTING = IntegrationTesting(
MY_FUNCTIONAL_TESTING = FunctionalTesting(
 bases=(MY_FIXTURE, ),


Testing prototypes in DSDM

DSDM prototype serves two different roles:

  • it is a partial build of the system that will be delivered
  • it is a technique for gathering information to clarify functional or non-functional requirements.
Different test criteria apply depending on the role the prototype is serving:
  • To the extent that the prototype represents part of the functionality of the final system, a test pass/fail can be based on whether or not the test result from the prototype matches an expected result based on the known  requirements.
  • To the extent that the prototype is intended to clarify requirements, the criterion for test success or failure is whether or not the prototype generates the expected information. This is exploratory. The expected
    information cannot be predefined in the same way that an expected result can be. The test of the prototype is being used to expand the knowledge and understanding of the prototype builder. The test is successful if this
    is achieved in the area under investigation.

A single prototype may serve in both roles and therefore should be tested against both criteria.In the Functional Model and Design and Build Iteration phases, prototyping generally iterates up to three times in line with the timeboxing process.  The prototypes that are produced can fall within any one of four categories. Prototypes of all categories can be tested against a mixture of “expected result” and “expected information” criteria.

This iterative lifecycle is summarised in the table below, which shows the categories of prototype and the type of testing criteria that will tend to apply at each iteration.

Iteration within Timebox
Prototype Category Investigation Refinement Consolidation
Business EI EI/ER ER
Usability EI EI/ER ER
Performance EI EI/ER ER
Capability EI EI/ER ER
Testing criteria for the categories of prototype (EI = Expected Information, ER = Expected Result)