Acceptance Tests and Outcomes

UAT Scenarios from Business Requirements Document

 

 

A number of issues that have come to be known as the ‘Zero/Null issues’ have been reported by SMART users and recorded in the SMART Wiki. These issues have now been addressed and the purpose of this testing is to check that SMART now handles the different scenarios correctly. As a tester, you should select an appropriate course that you administer and add results or outcomes that will test each of the reported issues. The same course can be used to test all issues so that things are simplified by not producing several different sheets for each scenario.

 

Reported issue

Test Scenario

Pass/Fail- Comments

1.     Where the student was absent from the exam and 0 is shown, then first sittings do not appear on the Registry csv file, despite appearing on the screen.

 

 

 

·         Input an exam result of 0 for one student

·         Update that record to ‘Abs’ to show that student was absent from exam

·         Recalculate spreadsheet for course

·         Download Return to Registry spreadsheets

·         Check that ‘Absent’ student record is listed correctly

·         If a student is absent then a mark of 0 and grade of AN should be passed to EUCLID

 

Crystal: Pass – all ok.

Linda: Hi Frank, I've been able to check all of these except for 5 and they appear OK to me. If Will and Crystal are satisfied with 5, then I'm happy to sign off. Thanks again, Linda 

2.     If a student obtains a true zero in an examination, the Registry return lists ON SCREEN are correct. However, in the .csv file it becomes Abs (Abs from exam) while in the .xml file it becomes ‘AN’, ‘Abs’ with a mark of 0. As an example, a student who scored a total grade of 8 for a course and failed the exam appears with a Mark of 8 in SMART, but as Abs in the various output files.

 

 

·         Input an exam result of 0 for one student

·         Recalculate spreadsheet for course

·         Download Return to Registry spreadsheets

·         Check that result of 0 is listed correctly

·         If a student obtains a true zero, EUCLID expects a mark of 0. On import it will allocate a grade of ‘H’.

 

·         Input an exam result of 8 for one student

·         Recalculate spreadsheet for course

·         Download Return to Registry spreadsheets

·         Check that result of 8 is listed correctly

·         If a mark of ‘8’ is obtained (as the actual course mark) this should be passed to EUCLID without a grade. On import EUCLID will allocate a grade of ‘H’.

 

Chrystal: Pass - The true 0 is returned correctly as fail.

On import of the xml file, there is no grade returned for the 0. There isn’t any grade for the other marks either, except for the CA and AN, which is correct. I don’t think a grade is needed for pass and fail, including a true 0 fail. . 

David Watters:  As for your other query, this is also expected behaviour - we have been told by SACS (Registry) that SMART should not allocate a grade under these circumstances. EUCLID will do this from the mark.

Linda: Hi Frank, I've been able to check all of these except for 5 and they appear OK to me. If Will and Crystal are satisfied with 5, then I'm happy to sign off. Thanks again, Linda 

3.    A similar problem, when a student handed in no coursework and was Absent from the exam. When the school admin changed the result to 'Resit as First Attempt', the XML file shows <mark>0</mark>, <grade>AN</grade> and <smart_result>Abs</smart_result>. Both the pdf and csv files show the Mark as blank, the Result as "Abs" and the Comment as "(Abs from exam)".  Students who had handed in coursework, and were Absent from the Exam, and had Resit as First Attempt are showing correctly on both the XML and pdf files.

  • Select an exam result for one student
  • Modify that record to ‘Resit as first attempt’
  • Recalculate spreadsheet for course
  • Download Return to Registry spreadsheets
  • Check that ‘Resit…’ student record is listed correctly
  • Resit as first attempt should not have a mark or grade passed to EUCLID

 

Crystal: Pass – all ok.

Linda: Hi Frank, I've been able to check all of these except for 5 and they appear OK to me. If Will and Crystal are satisfied with 5, then I'm happy to sign off. Thanks again, Linda 

 

4.    In Results-Course Level, main structure, the "amend data" option was used to set the "Resit as first attempt" and re-calculate. Results were correctly amended in the Course Spreadsheet, Return to Registry on-line and Return to Registry CSV but on Return to Registry XML it is returned as ABS. In PHYS09018 the above was repeated on this level 9 course, and found that the XML file was correct.

 

  • Select an exam result for one student
  • Modify that record to ‘Resit as first attempt’
  • Recalculate spreadsheet for course
  • Download Return to Registry XML spreadsheet
  • Check that ‘Resit…’ student record is listed correctly
  • Resit as first attempt should not have a mark or grade passed to EUCLID

 

Crystal: Pass – all ok.

Linda: Hi Frank, I've been able to check all of these except for 5 and they appear OK to me. If Will and Crystal are satisfied with 5, then I'm happy to sign off. Thanks again, Linda 

5.    There is an issue with class only students appearing as such in EUCLID, but as ABS or WD in SMART. For examples, see student s0564555 under course CLLC11026, and students s1126885 and s1129205 in course ELCC11006.

  • Select a course on which your school has ‘Class Only’ students
  • Download Return to Registry spreadsheets
  • Check that any ‘Class Only student record is listed correctly
  • Amark/grade of 0/NO should be returned to EUCLID for class only students

 

Crystal: All information returned is as described in the test scenario.

However, we are not sure whether we need to return any result to Registry for “class only” students, or whether this is the right data to return. Can you please check with them? If this is the right data to be returned, would the EUCLID uploader accept it?

David Watters: The result for Class Only students is correct, as Will has pointed in his subsequent message. 

Will Hossack:After much searching on the Class Only issue raise in the testing, I've finally found  the table of code (it is under EUCLID FAQ and not the Registry web site), see

http://www.euclid.ed.ac.uk/Staff/FAQ/Assessment_Results.html

and yes, "NO" for NOn-assesed is the right code for "Class Only".  The reason we were uncertain was that in previous years we have had Class Only students accidentially recorded as "ABS" or worse "FAIL" if their were returned to Registry.

I've also had a little play in the background and thinks now look good

Linda: Hi Frank, I've been able to check all of these except for 5 and they appear OK to me. If Will and Crystal are satisfied with 5, then I'm happy to sign off. Thanks again, Linda 

 

 

 

 

Other UAT Scenarios

Additional test scenarios used in testing but not sourced from the Business Requirements Document should be identified here. The justification for including the scenario in the UAT must also be recorded.

 

 

Ref

Test Scenario and Acceptance Criteria

Tested By

Date Tested

Outcome (PASS / FAIL)

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Ref

Notes on Test and/or Test Outcome

 

 

 

 

 

 

Open Issues

 

Any issues identified during UAT must be added to the Test Log. Please summarise or insert a copy of any open issues from the JIRA Test Log. It may be agreed that UAT can be signed off while some issues remain open please provide details of the UAT impact of each open issue.

 

 

 

BRD Ref

JIRA Test Log Ref

Issue Summary

Impact on UAT Sign Off

 

 

 

 

 

 

 

 

 

 

 

 

 

Link to updated JIRA Test Log

Document Sign Off

Please add other signatories where required

 

 

Project Manager

Name

Date Signed Off

Project Sponsor

Name

Date Signed Off

Business Analyst

Name

Date Signed Off

Add other signatories here