Giter VIP home page Giter VIP logo

aem-test-samples's Introduction

AEM Test Samples

This is a collection of test modules that can be run to validate an AEM cloud-based deployment. Tests are written according to Best practices.

Modules

  • smoke - generic smoke tests

How to use

Clone the repository and use maven for running each of the test modules.

The build also produces a jar-with-dependencies that can be run as a self-contained test module (using java directly or a small maven pom with failsafe configured).

Run the tests against localhost

mvn clean verify -Ptest-all

Run the test against your AEM Cloud Service author and publish tiers

The eaas-local profile has been added for convenience to allow to run the test locally against an AEM Cloud Service. The same test client configuration is used when the test module is executed in the Cloud Service

mvn -Peaas-local clean verify \
-Dcloud.author.url=<your-aem-author-url> \
-Dcloud.author.user=admin \
-Dcloud.author.password=<your-admin-password> \
-Dcloud.publish.url=<your-aem-publish-url \
-Dcloud.publish.user=admin \
-Dcloud.publish.password=<your-admin-password> \

Requirements

User

The test modules require the admin user or an admin-like user with enough privileges to create content, new users, groups and replicate content.

Replication

The tests also verify author-publish replication therefore for them to work correctly replication needs be be configured correctly.

Note that ReplicationIT will create and delete a randomized page in path like: /content/test-site/testpage_632460d4-361c-4b9b-9eef-d2446f79ec9c

Sling properties

The eaas-local profile facilitates the definition of sling properties expected by the aem-testing-clients (and the underlying Sling Testing Clients) in a convenient way.

The system properties are as follows:

  • sling.it.instances - should be set to 2
  • sling.it.instance.url.1 - should be set to the author URL, for example, http://localhost:4502
  • sling.it.instance.runmode.1 - should be set to author
  • sling.it.instance.adminUser.1 - should be set to the author admin user, e.g. admin
  • sling.it.instance.adminPassword.1 - should be set to the author admin password
  • sling.it.instance.url.2 - should be set to the author URL, for example, http://localhost:4503
  • sling.it.instance.runmode.2 - should be set to publish
  • sling.it.instance.adminUser.2 - should be set to the publish admin user, for example, admin
  • sling.it.instance.adminPassword.2 - should be set to the publish admin password
  • sling.it.configure.default.replication.agents - should be set to false

UI Tests

Custom UI testing is an optional feature that enables you to create and automatically run UI tests for your applications. The UI Testing section of the documentation provides in-depth information of their structure and usage.

  • /ui-cypress provides a sample Custom UI test module driven by Cypress.
  • /ui-selenium-webdriver provides a sample Custom UI test module driven by Selenium WebDriver.
  • The aem-project-archetype can be used as reference for Custom UI test modules driven by Selenium + WebdriverIO

Notable Examples

aem-test-samples's People

Contributors

absarasw avatar adobre90 avatar akrainiouk avatar amit-jain avatar andreituicu avatar andresbott avatar balasoiuroxana avatar bisswanger avatar bogdananton avatar chibulcu avatar chibulcuteanu avatar cristianbrande avatar cschneider avatar dulvac avatar etugarev avatar francescomari avatar igmarcu avatar jasghar avatar kronnox avatar marcpf avatar mtarantino avatar nortoneo avatar pat-lego avatar raducotescu avatar rbartl-pv avatar renovate[bot] avatar stefangrimm avatar stefanseifert avatar tyge68 avatar volteanu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aem-test-samples's Issues

ui-selenium-webdriver: Executing Docker fails with "Error response from daemon: error while creating mount source path '/Users/konradwindszus/git/adobe/aem-test-samples/ui-selenium-webdriver/target/reports': chown /Users/konradwindszus/git/adobe/aem-test-samples/ui-selenium-webdriver/target/reports: permission denied" when executing `mvn clean`

When you execute mvn clean verify -Pui-tests-docker-execution -DAEM_AUTHOR_URL=https://author.my-deployment.com (notice the additional clean phase) it fails with

[INFO] --- exec:1.6.0:exec (docker-compose-up) @ com.adobe.cq.cloud.testing.ui.java.ui.tests ---
 Container ui-selenium-webdriver-selenium-chrome-1  Created
 Container ui-selenium-webdriver-tests-1  Created
Attaching to selenium-chrome-1, tests-1
Gracefully stopping... (press Ctrl+C again to force)
Error response from daemon: error while creating mount source path '/Users/konradwindszus/git/adobe/aem-test-samples/ui-selenium-webdriver/target/reports': chown /Users/konradwindszus/git/adobe/aem-test-samples/ui-selenium-webdriver/target/reports: permission denied
[ERROR] Command execution failed.
org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)

This is due to the fact that in this case the parent directory target has been deleted. You should always ensure that the target is there even if no docker build has happened in that directory.

I have been trying to use Selenium-Webdriver in my project for 2 weeks without any luck.

Expected Behaviour

Documentation written in a way everyone could follow and understand it.

Actual Behaviour

I've been trying to follow the documentation to integrate Selenium Webdriver into my project generated with a Maven archetype on a AEMaaCS environment and I can say that it's hardly impossible to follow the documentation and make it works.

Reproduce Scenario (including but not limited to)

Steps to Reproduce

1- Create a new AEM local instance using AEMaaCS latest SDK from Software Distribution
2- Generate your sourcecode using Maven archetype 41
3- Trying to modify the ui.test module on your project and use the Selenium-Webdriver from aem-test-samples repo FOLLOWING the README and make it works.

Platform and Version

Sample Code that illustrates the problem

Logs taken while reproducing problem

Support replication for preview in ReplicationIT

Currently ReplicationIT does not support testing replication(activate/deactivate) on a preview agent. The replication actions sould be tested on preview agent also if the agent is available.

ui-selenium-webdriver: The packaging plugin for this project did not assign a main file to the project but it has attachments. Change packaging to 'pom'

When executing the first step from https://github.com/adobe/aem-test-samples/blob/aem-cloud/ui-selenium-webdriver/README.md#all-in-one with Maven 3.9.6 you run into the following error:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-install-plugin:3.1.1:install (default-install) on project com.adobe.cq.cloud.testing.ui.java.ui.tests: The packaging plugin for this project did not assign a main file to the project but it has attachments. Change packaging to 'pom'. -> [Help 1]

This is a new error in introduced with maven-install-plugin 3.0 (https://issues.apache.org/jira/browse/MINSTALL-118). The fix is to convert the packaging to pom.

In addition it is probably never useful to deploy anything for this project to a Maven repository therefore it would be useful to disable it with

<build>
        <pluginManagement>
            <plugins>
                <plugin>
                    <artifactId>maven-deploy-plugin</artifactId>
                    <configuration>
                        <skip>true</skip>
                    </configuration>
                </plugin>
                <plugin>
                    <artifactId>maven-install-plugin</artifactId>
                    <configuration>
                        <skip>true</skip>
                    </configuration>
                </plugin>
.....

ReplicationIT should have more meaningful error messages

In cases that the tests of ReplicationIT time out, the error message should be more descriptive. So instead of breaking becuause of a timeout it should hint to check the replication queues.
An additional potential root cause could be that the path is not allowed to get accessed by the dispatcher configuration.

Error while upgrading cypress version to 13.13.3

Team - I am trying to upgrade thecypress version from 12.17.0 to 13.13.3. This is based on the latest version available in docker-hub.

Expected Behaviour

The docker image should build fine without any error.

Actual Behaviour

The docker image is not getting built and is giving an error as follows:

ESC[91mW: Target Packages (main/binary-amd64/Packages) is configured multiple times in /etc/apt/sources.list.d/microsoft-edge-stable.list:1 and /etc/apt/sources.list.d/microsoft-edge.list:3
W: Target Packages (main/binary-all/Packages) is configured multiple times in /etc/apt/sources.list.d/microsoft-edge-stable.list:1 and /etc/apt/sources.list.d/microsoft-edge.list:3
E: Package 'python' has no installation candidate
ESC[0mThe command '/bin/sh -c apt -qqy update     && apt -qqy --no-install-recommends install     python     build-essential' returned a non-zero code: 100
2024/08/19 09:55:38 Container failed during run: build. No retries remaining.
failed to run step ID: build: exit status 100

Reproduce Scenario (including but not limited to)

We are on AEMaaCS and the error is coming in the cloud pipeline, during the build-images step, for the ui.tests module.

Steps to Reproduce

Run the docker build, using the Dockerfile with cypress version set to 13.13.3

Platform and Version

AEMaaCS SDK version: 2024.7.17258.20240726T172406Z

Sample Code that illustrates the problem

https://github.com/adobe/aem-test-samples/blob/aem-cloud/ui-cypress/Dockerfile

Logs taken while reproducing problem

Shared above

SearchIT should be disabled as it is only ever either a flake, or a false negative

SearchIT performs what appears to be a pretty basic search, but the CQClient.searchInPages() method that it calls relies on the long-deprecated com.day.cq.wcm.core.impl.servlets.FindReplaceServlet, which constructs an XPath query that is not optimized to leverage any product index definitions (no nodetype predicate, leading wildcard for the jcr:like expression, etc.).

It turns out that during the Cloud Manager Product Tests phase, successful execution of this test actually depends on the query engine choosing traversal, because an nt:base index definition scoped to the entire repository takes too long to update for the product test page creation to be reflected in the index before the test timeout.

This issue represents a frequent blocker for deploying product upgrades to some customers who have deployed an nt:base index definition, and it should simply be disabled as it does not truly demonstrate that supported product behavior is working.

assets.cy.js: @completeupload invocation does not complete if invoked with cy.wait construct

Expected Behaviour

assets.cy.js >> @completeupload should complete successfully, when the image path and previous steps are correct

Actual Behaviour

assets.cy.js >> @completeupload times-out while the status of asset upload is 'Processing'

Steps to Reproduce

npx cypress run --headed --no-exit --browser chrome >> reports timeout for @completeupload step

Enhancement Proposal

Instead of cy.wait, we can rather invoke the more deterministic cy.waitUntil - as follows:

    // wait for the /content/dam.completeUpload.json POST to complete before polling for the asset
    cy.waitUntil(
        () => ['@completeupload'], {
          errorMsg: `asset ${imagePath} should be uploaded`,
          timeout: 5000,
          interval: 1000
        }
    );

Please let me know if this looks ok - will accordingly raise a PR.

Add product IT to ensure Headless Admin App loading

Expected Behaviour

We have experienced on multiple occasions that custom project code has overridden CORS/CSRF/Sling Filter configs that conflicts with the respective OOTB AEM configurations. Duet to which following APIs request gets blocked:

POST/libs/dam/cfm/headless/endpoints/content/graphql.json
POST/libs/dam/cfm/headless/endpoints/content/graphql.A.json

For e.g. WKND Site has overridden these configs. Created an issue#301

As part of this issue, we should add the Product Functional tests to ensure:
A. the backend headless APIs (GraphQL) should not break due to custom configurations.
B. Headless Admin App is getting loaded with a repo pointing to the AEM CS author environment through IMS user

NOTE: The backend headless APIs (GraphQL) is supported under specific FT. Ensure the feature is enabled before running the tests.

Actual Behaviour

Reproduce Scenario (including but not limited to)

Steps to Reproduce

Platform and Version

Sample Code that illustrates the problem

Logs taken while reproducing problem

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

  • Update dependency org.apache.maven.plugins:maven-install-plugin to v3.1.3
  • Update dependency org.awaitility:awaitility to v4.2.2
  • Update dependency org.apache.maven.plugins:maven-jar-plugin to v3.4.2
  • Update dependency org.apache.maven.plugins:maven-javadoc-plugin to v3.10.0
  • Update dependency org.apache.maven.plugins:maven-resources-plugin to v3.3.1
  • Update dependency org.apache.maven.plugins:maven-surefire-plugin to v3.5.0
  • Update dependency org.apache.maven.plugins:maven-surefire-report-plugin to v3.5.0
  • Update dependency org.codehaus.mojo:exec-maven-plugin to v3.4.1
  • Update dependency org.seleniumhq.selenium:selenium-java to v4.24.0
  • Update maven Docker tag to v3.9.9
  • Update cypress/included Docker tag to v13
  • Update dependency cypress to v13
  • Update dependency cypress-fail-on-console-error to v5
  • Update dependency cypress-terminal-report to v6
  • Update dependency org.slf4j:slf4j-simple to v2
  • ๐Ÿ” Create all rate-limited PRs at once ๐Ÿ”

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

docker-compose
ui-selenium-webdriver/docker-compose-chrome.yaml
ui-selenium-webdriver/docker-compose-firefox.yaml
dockerfile
ui-cypress/Dockerfile
  • cypress/included 12.17.0
ui-selenium-webdriver/Dockerfile
  • maven 3.8.6-eclipse-temurin-11
maven
cf-smoke/pom.xml
  • org.apache.maven.plugins:maven-assembly-plugin 3.7.1
  • org.apache.maven.plugins:maven-compiler-plugin 3.13.0
  • org.apache.maven.plugins:maven-javadoc-plugin 3.7.0
  • com.gkatzioura.maven.cloud:azure-storage-wagon 2.3
  • org.apache.maven.plugins:maven-surefire-plugin 3.3.1
  • org.apache.maven.plugins:maven-failsafe-plugin 3.3.1
  • org.apache.maven.plugins:maven-surefire-plugin 3.3.1
  • org.apache.maven.plugins:maven-failsafe-plugin 3.3.1
  • com.adobe.cq:aem-cloud-testing-clients 1.2.6
  • org.slf4j:slf4j-simple 1.7.36
smoke/pom.xml
  • org.apache.maven.plugins:maven-assembly-plugin 3.7.1
  • org.apache.maven.plugins:maven-compiler-plugin 3.13.0
  • org.apache.maven.plugins:maven-javadoc-plugin 3.7.0
  • com.gkatzioura.maven.cloud:azure-storage-wagon 2.3
  • org.apache.maven.plugins:maven-surefire-plugin 3.3.1
  • org.apache.maven.plugins:maven-failsafe-plugin 3.3.1
  • org.apache.maven.plugins:maven-surefire-plugin 3.3.1
  • org.apache.maven.plugins:maven-failsafe-plugin 3.3.1
  • net.sourceforge.htmlunit:htmlunit 2.70.0
  • com.adobe.cq:aem-cloud-testing-clients 1.2.6
  • org.slf4j:slf4j-simple 1.7.36
ui-cypress/pom.xml
  • org.apache.maven.plugins:maven-clean-plugin 3.4.0
  • org.apache.maven.plugins:maven-resources-plugin 3.0.2
  • org.apache.maven.plugins:maven-jar-plugin 3.4.2
  • org.codehaus.mojo:exec-maven-plugin 3.3.0
  • org.codehaus.mojo:exec-maven-plugin 3.3.0
ui-selenium-webdriver/pom.xml
  • org.apache.maven.plugins:maven-clean-plugin 3.4.0
  • org.apache.maven.plugins:maven-resources-plugin 3.3.1
  • org.apache.maven.plugins:maven-jar-plugin 3.4.2
  • org.apache.maven.plugins:maven-enforcer-plugin 3.5.0
  • org.apache.maven.plugins:maven-assembly-plugin 3.7.1
  • org.apache.maven.plugins:maven-install-plugin 3.1.2
  • org.codehaus.mojo:exec-maven-plugin 3.3.0
  • org.apache.maven.plugins:maven-antrun-plugin 3.1.0
  • org.codehaus.mojo:exec-maven-plugin 3.3.0
ui-selenium-webdriver/test-module/pom.xml
  • org.apache.maven.plugins:maven-clean-plugin 3.3.2
  • org.apache.maven.plugins:maven-resources-plugin 3.3.1
  • org.apache.maven.plugins:maven-jar-plugin 3.3.0
  • org.apache.maven.plugins:maven-assembly-plugin 3.6.0
  • org.apache.maven.plugins:maven-enforcer-plugin 3.4.1
  • org.apache.maven.plugins:maven-surefire-plugin 3.2.5
  • org.apache.maven.plugins:maven-failsafe-plugin 3.2.5
  • org.apache.maven.plugins:maven-resources-plugin 3.3.1
  • org.apache.maven.plugins:maven-surefire-report-plugin 3.2.5
  • org.apache.maven.plugins:maven-surefire-plugin 3.2.5
  • org.apache.maven.plugins:maven-failsafe-plugin 3.2.5
  • org.apache.maven.plugins:maven-surefire-report-plugin 3.2.5
  • org.seleniumhq.selenium:selenium-java 4.18.1
  • junit:junit 4.13.2
  • commons-io:commons-io 2.15.1
  • com.adobe.cq:aem-cloud-testing-clients 1.2.6
wcm-smoke/pom.xml
  • org.apache.maven.plugins:maven-assembly-plugin 3.7.1
  • org.apache.maven.plugins:maven-compiler-plugin 3.13.0
  • org.apache.maven.plugins:maven-javadoc-plugin 3.7.0
  • com.gkatzioura.maven.cloud:azure-storage-wagon 2.3
  • org.apache.maven.plugins:maven-surefire-plugin 3.3.1
  • org.apache.maven.plugins:maven-failsafe-plugin 3.3.1
  • org.apache.maven.plugins:maven-surefire-plugin 3.3.1
  • org.apache.maven.plugins:maven-failsafe-plugin 3.3.1
  • net.sourceforge.htmlunit:htmlunit 2.70.0
  • com.adobe.cq:aem-cloud-testing-clients 1.2.6
  • org.slf4j:slf4j-simple 1.7.36
xf-smoke/pom.xml
  • org.apache.maven.plugins:maven-assembly-plugin 3.7.1
  • org.apache.maven.plugins:maven-compiler-plugin 3.13.0
  • org.apache.maven.plugins:maven-javadoc-plugin 3.7.0
  • com.gkatzioura.maven.cloud:azure-storage-wagon 2.3
  • org.apache.maven.plugins:maven-surefire-plugin 3.3.1
  • org.apache.maven.plugins:maven-failsafe-plugin 3.3.1
  • org.apache.maven.plugins:maven-surefire-plugin 3.3.1
  • org.apache.maven.plugins:maven-failsafe-plugin 3.3.1
  • com.adobe.cq:aem-cloud-testing-clients 1.2.6
  • org.slf4j:slf4j-simple 1.7.36
  • org.awaitility:awaitility 4.2.1
npm
ui-cypress/test-module/package.json
  • cypress 12.17.0
  • cypress-fail-on-console-error ^4.0.3
  • cypress-multi-reporters ^1.6.3
  • mocha-junit-reporter ^2.2.0
  • cypress-terminal-report ^5.3.3

  • Check this box to trigger a request for Renovate to run again on this repository

The ReplicationIT test must check status on author

The current ReplicationIT test implementation has fundamental issues

  1. It creates test pages in the customer content. This breaks a contract because the product edits content in the customer space. In some edge cases, test pages can end up published.
  2. It tests publishing infrastructure (CDN, dispatcher) that is out of scope for the Replication feature. The dispatcher and parts of the CDN are owned by the customer, so a wrong customer configuration can fail the product test.

If testing the publishing infrastructure is required, it should be done at project level because most of the infra is owned by the customer.

Regarding ReplicationIT, it should be rewritten in a way that it only uses information available on author. It's possible to do so, because the Replication framework does provide the status of individual replication actions on author. Also, the test would effectively cover more Replication feature than the current implementation.

(a) Instead of creating new pages in the customer content space, do replicate a path in the immutable content such as /libs.
(b) Instead of checking for replication status by polling the publish tier, do check for status on author either via events or via the replication queues.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.