Case Study: User Change Password Automated Test
Completing this simple automated test well is not as easy as some might think.
This is also included in the “How to in Selenium WebDriver” series.
This article will provide four approaches to One Test Automation Scenario Interview Question that Most Candidates Failed.
“Change a User’s Password”
Table of Contents:
· Be aware of Side-Effect of Test Execution
· 1. Change the password back
· 2. Starting with a newly created user
· 3. Save the password-change status into a file
· 4. Database Reset
· Review (by Zhimin)Be aware of Side-Effect of Test Execution
Unlike Unit Testing, functional testing may have side effects, that is, changing the state of the application or certain data. This test case falls into that category.
After a user changes his password successfully, we cannot use the old password to log in again. The below video (animated GIF) shows the execution of running a change password test twice, then failing the second time.
Here I will present four approaches.
1. Change the password back
Once the password has been changed (and verified), immediately change it back to the previous password.
Test Script:
# try running this case twice
it "[1] User can change password, Change it back" do
sign_in("james@client.com", "test01")
click_avatar_menu("Edit Profile")
edit_user_page = EditUserPage.new(browser)
edit_user_page.enter_password("newpass")
edit_user_page.enter_confirm_password("newpass")
edit_user_page.click_save
relogin("james@client.com", "newpass")
# reset the user's password back
click_avatar_menu("Edit Profile")
edit_user_page = EditUserPage.new(browser)
edit_user_page.enter_password("test01")
edit_user_page.enter_confirm_password("test01")
edit_user_page.click_save
try_for(3, 0.5) { expect(toast_text).to include("User profile has been updated successfully") }
endPros:
Easy to understand
Cons:
Failures on the second change (set it back) could still leave the data in an invalid state
Some gap time
Zhimin: I use the term ‘Gap Time’ to define the time that the user login was invalid. During that time, other users (or tests) unable to login.
Obviously, the shorter gap time, the better.
Discussion:
A logical question from an experienced tester would be: that is unreliable, as the second change-password steps could fail. Yes, that’s true.
I added a good degree of fault tolerance using retry in Ruby, in this case, up to 5 attempts.
# reset the user's password back
attempt_count = 1
begin
click_avatar_menu("Edit Profile")
edit_user_page = EditUserPage.new(browser)
edit_user_page.enter_password("test01")
sleep 0.1
edit_user_page.enter_confirm_password("test01")
edit_user_page.click_save
try_for(3, 0.5) { expect(toast_text).to include("User profile has been updated successfully") }
rescue => e
attempt_count += 1
retry if attempt_count < 5
endThe test script shall achieve >99% reliability now. For the rest < 1%, I would leave to the Auto-Rerun feature of a Continuous Testing Server, such as BuildWise. If someone is fixated on 100% reliability, BuildWise also has a ‘Manual Rerun’ feature.
2. Starting with a newly created user
Use a newly created temporary account as the test user, so we won’t care about changing its password afterwards.
Test Script
it "[2] User can change password, NEW USER EVERY TIME" do
# register a brand new user
visit("/sign-up")
sign_up_page = SignUpPage.new(browser)
new_email = Faker::Internet.email
puts new_email
try_for(2) { sign_up_page.enter_email(new_email) }
sign_up_page.enter_password("test02")
fail_safe{ sign_up_page.enter_captcha("wise") }
sign_up_page.click_create_account
expect(page_text).to include("Please check your email to activate your account.")
activate_url = nil
try_for(10, 5) {
open_email("Account activation") { |email_body_html|
activate_url = Nokogiri::HTML(email_body_html).at_css("a#activate-link")["href"]
}
driver.get(activate_url)
try_for(3, 0.5) { expect(toast_text).to include("Account activated, you can sign in now.") }
}
# Now new change password steps...
sign_in(new_email, "test02")
expect(page_text).to include("Find Business")
click_avatar_menu("Edit Profile")
edit_user_page = EditUserPage.new(browser)
edit_user_page.enter_password("newpass")
edit_user_page.enter_confirm_password("newpass")
edit_user_page.click_save
relogin(new_email, "newpass")
try_for(2) { expect(toast_text).to include("You have signed in successfully") }
endNote: the new user’s email and username must be unique. We can achieve this easily with Faker library.
Pros:
Guaranteed working (if user registration works)
No issues with gap time, as nobody is aware of this new temp user.
Cons:
May leave many temporary users created in the system
(if doing proper CI/CD, running a whole suite of automated tests multiple times a day, that means quite a lot)Slow
Dependent on user registration (which may fail)
Maybe incur extra costs
Zhimin: for example, one admin told me that MS Dynamic 365 accounts cost money.
Discussions:
Some would write delete the new user in after(:all) or tearDown() . This means extra test steps: longer running time and possible failures as well.
after(:all) do
// add logic to find and delete his newly created user
driver.quit
end
3. Save the password-change status into a file
Store test users’ credentials somewhere, such as a shared folder, so you can read them from the test scripts. The automated script will get the password dynamically from this special file.
After changing one user’s password successfully, an automated test script must update this special file.
Test Script
# create users.json file on a shared folder, e.g./Shared/user.json
it "[3] User can change password, read pass from external file" do
users_login = JSON.parse(File.read('/Users/Shared/user.json'))
# get current password from the JSON
password = users_login["james@client.com"]
# create a new password
new_pass = Faker::Internet.password(min_length:6, max_length:6)
# Login
sign_in("james@client.com", password)
click_avatar_menu("Edit Profile")
# Change the password
edit_user_page = EditUserPage.new(browser)
edit_user_page.enter_password(new_pass)
edit_user_page.enter_confirm_password(new_pass)
edit_user_page.click_save
try_for(2) { expect(toast_text).to include("User profile has been updated successfully.") }
# if change successful, update users.json
users_login["james@client.com"] = new_pass
relogin("james@client.com", new_pass)
try_for(2) { expect(toast_text).to include("You have signed in successfully") }
File.write('/Users/Shared/user.json', JSON.dump(users_login))
endPros:
Quick, no need to revert (second changing it back).
Short gap time
Cons:
Dependent on an external file (access, permission, … )
Extra care needs to consider when executing tests in different environments, such as in a CT server or Build Agents.
Need to check the special file before performing manual testing
The file might be modified by humans or saved in a wrong format
4. Database Reset
Invoke a utility feature that reset the data in the database. Stay tuned for my father’s “The Simpsons’ Pattern” article. For now, check out “Free Test Automation Practice Site with Database Reset”. You can try out the database reset utility now.
Test Script:
it "[4] User can change password, DB RESET" do
reset sign_in("james@client.com", "originalpass")
click_avatar_menu("Edit Profile")
edit_user_page = EditUserPage.new(browser)
edit_user_page.enter_password("newpass")
edit_user_page.enter_confirm_password("newpass")
edit_user_page.click_save relogin("james@client.com", "newpass")
try_for(2) {
expect(toast_text).to include("signed in successfully")
}
endPros:
Simple
Fast, if done properly
Reliable
Short gap time (if we call
reset) after the test execution
Cons:
Development assistance is required.
Database reset needs to be very quick.
Below is the timing for WhenWise’s quick reset.
Reset [Quick] took 0.38085 secondsReview (by Zhimin)
Writing a highly-reliable automated test takes a lot more effort than an average one. That is where a real Automated Tester excels. However, those efforts are often under-appreciated by the managers.
Out of the four approaches Courtney listed, I guess, readers are mostly interested in Approach #4. The concept behind that is ‘The Simpsons’ Data Reset Pattern. This requires developer assistance. A common question is: “Invoking a reset will affect others (tests and users)”. Yes, that is right. “Data-Reset”, if used frequently, shall be only in dedicated test servers. The screenshot shows the TestWisely app’s four dedicated test servers (for test automation)
ci1-testwisely.agilweay.netci2-testwisely.agilweay.netci3-testwisely.agilweay.netci4-testwisely.agilweay.net
Astute readers might have already figured out why there are multiple? Besides parallel execution (speed up), a data reset on ci1-testwiselywon’t affect the other three.
Data-Reset is rarely implemented in software projects, the Approach #1 and #2 are more practical to most testers. Based on the nature of your app, choose it wisely.
if the sign-up process is expensive (takes a long time), use Approach #1.
Then deal with the gap time (the test user’s password remained invalid before it is changed back). I recommend optimizing the speed for the setting-it-back steps, but don’t overdo it. Keep the test simple. We can use the auto-rerun feature in CT servers to handle that.if the sign-up process is relatively quick, use Approach #2
this way, we achieve (nearly) total independence from other tests, at the expense of the time creating extra once-off test data.
Further reading:
 | A guest post by
|