Getting started

Prepare environment

Make sure that you have the following software installed and configured on your development environment:

• PHP version supported by the Magento instance under test
• Composer 1.3 or later
• Java 1.8 or later
• Selenium Server Standalone 3.1 or later and ChromeDriver 2.33 or later or other webdriver in the same directory

Install Magento

Use instructions below to install Magento.

Step 1. Clone the magento2 source code repository

1
git clone https://github.com/magento/magento2.git

or

1
git clone git@github.com:magento/magento2.git

Step 2. Install dependencies

Checkout the Magento version that you are going to test.

1
cd magento2/

1
git checkout 2.3-develop

Install the Magento application.

1
composer install

Prepare Magento

Configure the following settings in Magento as described below.

WYSIWYG settings

A Selenium web driver cannot enter data to fields with WYSIWYG.

To disable the WYSIWYG and enable the web driver to process these fields as simple text areas:

1. Log in to the Magento Admin as an administrator.
2. Navigate to Stores > Settings > Configuration > General > Content Management.
3. In the WYSIWYG Options section set the Enable WYSIWYG Editor option to Disabled Completely.
4. Click Save Config.

or via command line:

1
bin/magento config:set cms/wysiwyg/enabled disabled

Clean the cache after changing the configuration values:

1
bin/magento cache:clean config full_page

Security settings

To enable the Admin Account Sharing setting, to avoid unpredictable logout during a testing session, and disable the Add Secret Key in URLs setting, to open pages using direct URLs:

1. Navigate to Stores > Settings > Configuration > Advanced > Admin > Security.
2. Set Admin Account Sharing to Yes.
3. Set Add Secret Key to URLs to No.
4. Click Save Config.

or via command line:

1
bin/magento config:set admin/security/admin_account_sharing 1

1
bin/magento config:set admin/security/use_form_key 0

Clean the cache after changing the configuration values:

1
bin/magento cache:clean config full_page

Webserver configuration

The MFTF does not support executing CLI commands if your web server points to <MAGE_ROOT_DIR>/pub directory as recommended in the Installation Guide. For the MFTF to execute the CLI commands, the web server must point to the Magento root directory.

Nginx settings

If the Nginx Web server is used on your development environment, then Use Web Server Rewrites setting in Stores > Settings > Configuration > General > Web > Search Engine Optimization must be set to Yes.

To be able to run Magento command line commands in tests, add the following location block to the Nginx configuration file in the Magento root directory:

1
2
3
4
5
6
7
8
9
location ~* ^/dev/tests/acceptance/utils($|/) {
  root $MAGE_ROOT;
  location ~ ^/dev/tests/acceptance/utils/command.php {
      fastcgi_pass   fastcgi_backend;
      fastcgi_index  index.php;
      fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
      include        fastcgi_params;
  }
}

Set up an embedded MFTF

This is the default setup of the MFTF that you would need to cover your Magento project with functional tests. It installs the framework using an existing Composer dependency such as magento/magento2-functional-testing-framework. If you want to set up the MFTF as a standalone tool, refer to Set up a standalone MFTF.

Install the MFTF.

1
composer install

Step 1. Build the project

In the Magento project root, run:

1
vendor/bin/mftf build:project

If you use PhpStorm, generate a URN catalog:

1
vendor/bin/mftf generate:urn-catalog .idea/

If the file does not exist, add the --force option to create it:

1
vendor/bin/mftf generate:urn-catalog --force .idea/

See generate:urn-catalog for more details.

Step 2. Edit environmental settings

In the magento2/dev/tests/acceptance/ directory, edit the .env file to match your system.

1
vim dev/tests/acceptance/.env

Specify the following parameters, which are required to launch tests:

• MAGENTO_BASE_URL must contain a domain name of the Magento instance that will be tested. Example: MAGENTO_BASE_URL=http://magento.test

• MAGENTO_BACKEND_NAME must contain the relative path for the Admin area. Example: MAGENTO_BACKEND_NAME=admin

• MAGENTO_ADMIN_USERNAME must contain the username required for authorization in the Admin area. Example: MAGENTO_ADMIN_USERNAME=admin

• MAGENTO_ADMIN_PASSWORD must contain the user password required for authorization in the Admin area. Example: MAGENTO_ADMIN_PASSWORD=123123q

Learn more about environmental settings in Configuration.

Step 3. Enable the Magento CLI commands

In the Magento project root, run the following command to enable the MFTF to send Magento CLI commands to your Magento instance.

1
cp dev/tests/acceptance/.htaccess.sample dev/tests/acceptance/.htaccess

Step 4. Generate and run tests

To run tests, you need a running Selenium server and mftf commands.

Run the Selenium server

Run the Selenium server in terminal. For example, the following commands run the Selenium server for Google Chrome:

1
cd <path_to_directory_with_selenium_server_and_webdriver>/

1
java -Dwebdriver.chrome.driver=chromedriver -jar selenium-server-standalone-3.14.0.jar

Generate and run all tests

1
vendor/bin/mftf generate:tests

1
cd dev/tests/acceptance

1
vendor/bin/codecept run functional -c dev/tests/acceptance/codeception.yml

See more commands in codecept.

Run a simple test

To clean up the previously generated tests, and then generate and run a single test AdminLoginTest, run:

1
vendor/bin/mftf run:test AdminLoginTest --remove

See more commands in mftf.

Step 5. Generate reports

During testing, the MFTF generates test reports in CLI. You can generate visual representations of the report data using Allure Framework. To view the reports in GUI:

• Install Allure
• Run the tool to serve the artifacts in dev/tests/acceptance/tests/_output/allure-results/:

1
allure serve dev/tests/acceptance/tests/_output/allure-results/

Learn more about Allure in the official documentation.

Set up a standalone MFTF

The MFTF is a root level Magento dependency, but it is also available for use as a standalone application. You may want to use a standalone application when you develop for or contribute to MFTF, which facilitates debugging and tracking changes. These guidelines demonstrate how to set up and run Magento acceptance functional tests using standalone MFTF.

Prerequisites

This installation requires a local instance of the Magento application. The MFTF uses the tests from Magento modules as well as the app/autoload.php file.

Step 1. Clone the MFTF repository

If you develop or contribute to the MFTF, it makes sense to clone your fork of the MFTF repository. For contribution guidelines, refer to the Contribution Guidelines for the Magento Functional Testing Framework.

Step 2. Install the MFTF

1
cd magento2-functional-testing-framework

1
composer install

Step 3. Build the project

1
bin/mftf build:project

Step 4. Edit environment settings

In the dev/.env file, define the basic configuration and MAGENTO_BP parameters.

Step 5. Enable the Magento CLI commands

Copy the etc/config/command.php file into your Magento installation at <magento root directory>/dev/tests/acceptance/utils/. Create the utils/ directory, if you didn’t find it.

Step 6. Remove the MFTF package dependency in Magento

The MFTF uses the Magento app/autoload.php file to read Magento modules. The MFTF dependency in Magento supersedes the standalone registered namespaces unless it is removed at a Composer level.

1
composer remove magento/magento2-functional-testing-framework --dev -d <path to the Magento root directory>

Step 7. Run a simple test

Generate and run a single test that will check your logging to the Magento Admin functionality:

1
bin/mftf run:test AdminLoginTest

You can find the generated test at dev/tests/functional/tests/MFTF/_generated/default/.

Step 8. Generate Allure reports

The standalone MFTF generates Allure reports at dev/tests/_output/allure-results/. Run the Allure server pointing to this directory:

1
allure serve dev/tests/_output/allure-results/