Acceptance Tests and Outcomes

1. Scenarios from Business Requirements Document

 

Although user testing should be based on the test scenarios and acceptance criteria identified in the Business Requirement Document, it has not been strictly possible to test the scenarios identified in the BRD. This is attributable to two conditions affecting this element of this project. The first is because of the way in which the project's requirements have been stated (see table below). These are very general statements that can be deemed successful (or not) through the project team undertaking a series of constituent tests. This latter point is the second condition impacting on the testing - we do not have a TEST environment for Office 365. So, the testing is made of individual, modular elements that will determine the veracity of the business requirements after these have been conducted and discussed with the project team.  The individual tests are listed in the lower table, and the judgement of the project team of the impact on the requirements will be recorded in the table here:

 

BRD Ref.

Scenario/Test

Tested by

Date Tested

Outcome

1.1

The email address must be generated using a variation of the existing rules.

 

 

 

2.1

Amend process to generate name-based aliases for UF applicants at point when account affiliation is created in IDM.

 

 

 

2.2

Historically assigned aliases must be respected to ensure uniqueness

 

 

 

2.3

A searchable email directory must be available.

 

 

 

2.4

A mechanism for feeding this data to the mail relays must be created.

 

 

 

2.5

The mail relays must be updated to accept this data

 

 

 

2.6

The student welcome email must be updated to include the name-based email address in addition to the UUN-based alias

 

 

 

2.7

UUN@sms.ed.ac.uk must continue to route mail for all students

 

 

 

2.8

For all students there must be a process manual / automated for updating name-based aliases

 

 

 

 

 

 

 

 

 

 

BRD Ref          

Notes on Test and/or Test Outcome                     

 

 

 

 

 

 

 

2. UAT Scenarios

Additional test scenarios used in testing but not sourced from the Business Requirements Document should be identified here. These tests are the constituent tests identified above:

 

 

Ref        

Test Scenario and Acceptance Criteria              

Tested By          

Date Tested          

Outcome (PASS / FAIL)             

  Integration Tests      

01

New staff account - Check VRSIDM UI shows standard staff display.

HB

 

Pass

02

New staff account simulated EDDIR entry - Check alias display and downstream notifications.

HB

 

Pass

03

New staff account aged and archived - Spec for non-student accounts unclear so for UAT purposes Check mail alias status

Result: University mail entry retained as per spec.

HB

 

Pass

04

Returning staff account - Check alias display and downstream notifications

Result: Account re-activated with original uun. University email alias re-linked. University email alias present in downstream service notifications.

HB   Pass
05

New Functional Account with assigned email address -  Check alias  display and downstream notifications

Result: Functional com033uat created. Email added. Identity and University email alias sent correctly including to Office365.  

HB   Pass
06 New Visitor Staff Account - Check alias  display and downstream notifications HB   Pass
07

New Applicant with home email address. - Check email address and downstream notifications.

Result:Account s9999975 created with Home address.  

HB   Pass
08

Archive applicant -  Check downstream notifications and email address.

Result: Account s9999975 archived. Confirmed Home email address removed.  

HB   Pass
  Revised Functionality Testing      
09

 New Applicant with Home email address. Promote to STUUG - Check email address and notifications

Result: Applicant s9999973 created with Home address jwilson3@zzz.zzz Confirmed email address jwilson3@zzz.zzz sent to downstreams Promoted to STUUG Email addresses assigned J.Wilson-40. Confirmed J.Wilson-32, J.Wilson-38 in historical table and remainder up to J.Wilson-39 in idstore_emailaddresses so J.Wilson-40 is correct Confirmed email address jwilson3@zzz.zzz, s9999973@sms.ed.ac.uk and  J.Wilson-40@sms.ed.ac.uk (primary) sent to downstreams Added future application VRS/IDM UI Personalised email display incorrect

Jira COM033-19 raised

Re-tested - Student primary affiliation logic now taking priority

HB  

Pass on re-test

 

10

New Applicant with Home email address. Promote to STUUG - Check email address and notifications.

Result: Applicant s9999970  created with home address Confirmed email address mtwain@zzz.zzz sent to downstreams Promoted to STUUG Confirmed email address assigned M.Twain@sms.ed.ac.uk as Primary Confirmed mtwain@zzz.zzz, s9999970@sms.ed.ac.uk and M.Twain@sms.ed.ac.uk (primary) sent to downstreams.

HB   Pass
11

New fast track student (no application). Alter primary and check notifications

Result: Created STUUG s9999969 . Email addresses assigned s9999969@sms.ed.ac.uk and S.Tracy@sms.ed.ac.uk (primary). Notifications s9999969@sms.ed.ac.uk (UniAlias) and S.Tracy@sms.ed.ac.uk (University). Alter primary and check notifications. Primary changed from S.Tracy@sms.ed.ac.uk to s9999969@sms.ed.ac.uk. Notifications s9999969@sms.ed.ac.uk (University) and S.Tracy@sms.ed.ac.uk (UniAlias).

HB   Pass
12

New student with name typo

Result: Created STUUG s9999968 with name scott scotty mccleugh Mail assigned mccleugh@sms.ed.ac.uk   (primary)  and s9999968@sms.ed.ac.uk Notifications mccleugh@sms.ed.ac.uk   (University), s9999968@sms.ed.ac.uk (UniAlias) Changed forename from scott to Scott Mail assigned S.mccleugh@sms.ed.ac.uk (primary),  mccleugh@sms.ed.ac.uk    and s9999968@sms.ed.ac.uk Notifications S.mccleugh@sms.ed.ac.uk (University), s9999968@sms.ed.ac.uk (UniAlias), mccleugh@sms.ed.ac.uk (UniAlias) Changed surname from mccleugh to McCleugh Mail assigned S.McCleugh-1@sms.ed.ac.uk (primary),  S.mccleugh@sms.ed.ac.uk,  mccleugh@sms.ed.ac.uk    and s9999968@sms.ed.ac.uk Notifications S.McCleugh-1@sms.ed.ac.uk (University),  S.mccleugh@sms.ed.ac.uk (UniAlias),,  mccleugh@sms.ed.ac.uk (UniAlias),     and s9999968@sms.ed.ac.uk (UniAlias),

Expected existing alias S.mccleugh@sms.ed.ac.uk to be updated. Instead new alias S.McCleugh-1@sms.ed.ac.uk was created. JIRA COM033-20 Created.

Re-test result: IDM logic no-longer creates duplicate alias, but instead ignores change. Although technically incorrect, testing confirms that this is the behaviour of the Exchange/Office365 connector, so am in agreement with Dev Team that maintaining sync with Office365 is the best action. Correction would be a  manual process.

 

 

HB   Pass on re-test.

 

 

13

New PGT Account-Multiple forenames

Result: Account s9999967 created with name Test Test Test Anderson-Anderson Email address assigned T.T.T.Anderson-Anderson@sms.ed.ac.uk

JIRA COM033-1 raised to check potential for multiple forenames. Dev Tea, confirmed that existing logic creates a maximum of 3 initials. Tested and passed.

 

HB  

Pass

 

14

NewPGR account

Result: PGR Account s9999966 created with name Kiss Kiss Bang Bang. Alias assignged K.K.Bang-Bang@sms.ed.ac.uk Simulate EDDIR email alias. Primary alias changed from K.K.Bang-Bang@sms.ed.ac.uk to KissKiss.BangBang@ed.ac.uk Partial Success - idstore_emailaddresses shows source as IDM which should be EDDIR. JIRA COM033-21 created.

Re-test result: EDDIR updates are now flagged as source = EDDIR in both process and notification queue messages.

 

 

HB   Pass on re-test
15

Applicant becoming STUUG then archived then returning

Result: S9999964 created Email R.Bilboa-1@sms.ed.ac.uk (University), s9999964@sms.ed.ac.uk (UniAlias), r.bilboa@zzz.zzz (Home) App and stu affiliations aged out and account archived. Emails retained R.Bilboa-1@sms.ed.ac.uk, s9999964@sms.ed.ac.uk Return as app with different email Application created with R.Bilboa-1@sms.ed.ac.uk (Primary), s9999964@sms.ed.ac.uk (UniAlias),  r.bilboa@yyy.yyy (Home)

Fail - legacy student email aliases assigned to Applicant.

Display issue is fixed by https://www.jira.is.ed.ac.uk/browse/COM033-19

The root issue of this JIRA is related to htps://www.jira.is.ed.ac.uk/browse/COM033-14 This is an existing IDM issue affecting applicants who are currently alumni, applicants who are returning withdrawn students and returning visitors and functional accounts which are not assigned Office 365 or Staffmail  on their return visit.

Conclusion: Exsting IDM issue out-with scope,  for this phase, but should be flagged for investigation in later phases/replacment projects.

 

 

 

 

 

HB  

Pass on re-test subject to  conclusion detailed.

16

New STUUG with non-ASCII name

Result: Created account  s9999963 Éanna João        Muñoz jiménez Resulted in a primary key  lookup violation. Traced to characters missing from table idstore_email_unicode . Fail - JIRA COM033-3 extended to cover this.

JIRA COM033-3 extended to cover this.

Re-test: Missing Unicode characters added. Duplicate non-displayable characters  rationalised. Testing confirmed that any remaining unknown characters are simply omitted from alias generation.

 

HB   Pass on re-test
17

 Multiple Student progression

Created student s9999962 Added alumnus affiliation Aged student affiliation to deleted. Removed personalised email and reverted uun-based email to University. Created new stuug affiliation Result: email S.Tracy-1@sms.ed.ac.uk assigned as University Added another stuug affiliation Result: Email unchanged as required. Added another separate stuug affiliation. Result: Email unchanged as required.

HB   Pass
18

Expiring  student alumnus - new student affiliation

Created account s9999961 Added alumnus affiliation. Expired student affiliation Removed personal email address and reverted uun-based email to primary University. Repeated student insert. Result: Personalised email address created Expired student affiliation Removed personal email address and reverted uun-based email to primary University. Altered identity datemodified to simulate an dormant alumnus account. Created a separate student affiliation . Result: Personalised email address not created.

HB   Pass
19

idstore_email_addresses table.

Random check confirms that known IDM email addresses have been paired with matching imported aliases. All existing student values for Preferred are '1'. All existing student preferred email values for field Obsolete are '0' .

HB   Pass
20

JIRA COM033-4  Build Issue - Acceptance Re-testing Subsection: "I believe sn and givennames should be treated the same because if a surname contains a single space, then I would suggest replacing it with a hyphen rather than potentially merging two words that originally had a space between them."

Tested using account s9999946 David David Davidson. Email assigned  D.David-Davidson@sms.ed.ac.uk

 

HB   Pass
21

JIRA COM033-6 - Build Issue

JIRA investigation confirmed that failed  personalised email address generation would fall back to uun@sms. Tested using s9999945 @@@@ @@@Davidson  - University alias set to s9999945@sms.ed.ac.uk

HB   Pass
  Remaining Tests      
         
  Mail Relay extract testing ITI    
  Mail directory extract testing ITI    
  Specific mail relay/directory check to ensure that non-UoE addresses which have been set incorrectly as type University do not cause extract issues. ITI    
  Mail aliases which have been matched to IDM accounts do not use the obsolete flag. Check that this does not affect Mail Directory extract behaviour. ITI    
  Mail Alias API testing AppMan   Testing de-scoped as API not deployed to Test environment due to API project delays.
  IDM Office365 connector - welcome email wording review and testing AppMan    
  End-to end test of new and old-style student visibility in Relay lists and Directory system |ITI/Service Managment  

Feasibility dependent upon whether or not ITI are able to simulate Relay/Directory update.  Alternative: Estimated risk assessment.

   

 

 

 

Ref            

Notes on Test and/or Test Outcome               

 

 

 

 

 

 

 

3. Open Issues

 

Any issues identified during UAT must be added to the Test Log. Please summarise or insert a copy of any open issues from the JIRA Test Log. It may be agreed that UAT can be signed off while some issues remain open please provide details of the UAT impact of each open issue.

 

BRD Ref           

JIRA Test Log Ref            

Issue Summary           

Impact on UAT Sign Off               

 

 

 

 

 

 

 

 

 

 

 

 

 

[Link to updated JIRA Test Log ]

 

4. Document Sign Off

[Please add other signatories where required]

 

 

Project Manager

Name

Date Signed Off

Project Sponsor

Name

Date Signed Off

Business Analyst

Name              

Date Signed Off                        

Add other signatories here