Acceptance Tests and Outcomes
UWP005 UAT & Report data
V1.2 by Stratos Filalithis, 7/2/2014
Overview
This document contains conclusions and reports on UAT for project UWP005. The reports were delivered using a BOXI Universe and the respective application. BOXI Functionality was explored and reports were exported to Microsoft Excel for further examination. The outputs were created based on the 4 objectives described in the beginning of the project.
The output files can be found here:
O:\projects\WDP\secure\Polopoly Content Audit Reports (UWP005)
All files have a similar structure with similar fields. For the most important, a small description is:
Field | Description/Content |
Root | Has the value of 2.0 all reports feed data for the whole of the Polopoly Tree |
Parent Item | The parent section for this record |
Child Item | The child section/article for this record (usually a unique value, depending on the report) |
Item Name | The section/article title |
URL | The full Polopoly URL (not the same as the website URL) |
Objective 1 (Priority: 2)
To audit the structure and metadata of the CMS content in preparation for migration to the new CMS and export this audit in csv file format.
File | Contents |
O1. Breadcrumb trail (for UWP Homepage).xlsx | The full breadcrumb trail for the UWP Website Homepage. This is the full Polopoly breadcrumb, so it contains more information than the usual Website URL, but it can probably get removed by a massive find/replace operation. |
O1. Breadcrumb trail (WholeTree).xlsx | This contains the full breadcrumb for all the articles on Polopoly. Keep in mind that this means all articles, even those and the MediaBank section, so that’s why there is a whopping 147000+ records on it. |
O1. Full Polopoly Tree (UWP Website - 2.580) | All children of section 2.580 (UWP Website Section) |
O1. Full Polopoly Tree (UWP Website Dev Area Unpublished- 2.1307).xlsx | All children of section 2.1307 (UWP Website Dev Area, which is unpublished) |
O1. Full Polopoly Tree (Vet School Website - 2.2371).xlsx | All children of section 2.2371 (Vet School Website) |
O1. Full Polopoly Tree (Whole Tree).xlsx | The full Polopoly Tree. This contains all articles, including Media Bank. |
O1. Polopoly Orphaned Component Tree | All the orphaned Polopoly content, meaning content that was disconnected from their parent. |
O1. Polopoly Orphaned Component Tree (Dev Area – 2.1307) | All the orphaned content from the Dev Area.
|
UAT Result
Success criterion | Result |
Scripts which can be run by either run by CMS Managers or run at their request through a support call. The output is in the form or a csv file. | PASS The reports can be created directly from BOXI and exported to several formats, including Excel readable |
Ability to produce complete content structures from any section or subsection in the CMS and should include the UoE, Media bank and Development Zone areas of Polopoly | FAIL Only children are shown when filtering and not grandchildren. Is there a way to filter the results in a way to show the full tree structure below a specific section? Note: It will be addressed by the developers, by updating the output.
Update: UWP had full structure of tree but could not filter it. Agnes had been waiting on confirmation on how to do this from UWP which they have now given her. This will only involve removing one clause and will not hold deployment up. Agnes will implement immediately and Neil will follow up after Agnes’ change.
Retest: PASS Stratos retested the update on 11 Feb and is now a PASS |
Ability to export Polopoly structures to .csv including information such as key metadata fields, published/unpublished status and all out bound links for further analysis | PASS Reports include key metadata (incl. workflow & published state) but not all outbound links (Is this addressed by Objective 3?) |
Objective 2 (Priority: 3)
To express the relationship between users, groups, workflow and permissions on sections to allow CMS managers to compare this with the CMS structure and metadata. This will inform and simplify the recreation of workflows in the new CMS.
File | Contents |
O2. Group - User.xls | Contains all Polopoly user groups with users added to them |
O2. Group - Workflow - Permission.xlsx | Contains all user groups with workflows and permissions assigned to them |
O2. Group - Workflow - Section.xlsx | Contains all user groups with workflows and permissions assigned to them per Polopoly section |
UAT Result
Success criterion | Result |
Scripts which can be run by either run by CMS Managers or run at their request through a support call. The output is in the form or a csv file. | PASS The reports can be created directly from BOXI and exported to several formats, including Excel readable |
Ability to see what sections each group is associated with | PASS Report O2. Group – Workflow Section includes this information |
Ability to see what groups are associated with a section and it’s children | PASS Report O2. Group – Workflow Section includes this information Note: There is no visible connection between parents & children on this report. This information should come with feedback from other reports. |
Ability to export this information to .csv for further analysis
| PASS The reports can be created directly from BOXI and exported to several formats, including Excel readable |
Breadth and depth of groups workflow permissions - mandatory
| PASS O2. Group Workflow Permission report contains this information |
Ability to see what groups an individual user is associated with
| PASS O2. Group User report contains this information |
Ability to see what workflows each group is associated with
| PASS O2. Group Workflow Permission report contains this information |
Ability to export this information to .csv for further analysis | PASS The reports can be created directly from BOXI and exported to several formats, including Excel readable |
Objective 3 (Priority: 1)
To have the information on shared content references within the CMS in order to rebuild these references within the new CMS.
The output file structure fields are:
Field | Description/Content |
User Item | The Polopoly item that references another Polopoly item (similar to parent) |
User Item Component.Item Name | User Item’s Title |
User Item Media.Media | Is the user item a media article? |
Usertype | Type of reference (i.e. include article, etc.) |
Used Item | The Polopoly item that is referenced by the User Item (similar to child) |
Used Item Component.Item Name | Used Item’s title |
User Item Media.Media | Is the used item a media article? |
Output files list:
File | Contents |
O3. Content References (UWP Home - 1.86349).xlsx | What articles are referenced in article 1.86349 (UWP Website Home) |
O3. Content References (Where UWP About Us is used - 1.86380).xlsx | What articles are referencing article 1.86380 (UWP About Us) |
O3. Content References.xlsx | Full list of referencing and referenced articles |
Specific example
There are some examples under Unidesk call I110316-0389. One of them is used below to showcase the difference between the `References` tab on Polopoly and the BOXI output.
The included article id is 1.17200.
Polopoly `References` tab results | ||
Article ID | Title | Is it actually used? |
1.18790 | Overview | Yes |
1.18814 | Overview | No |
1.15168 | Overview | Yes |
1.12366 | Knowledge Exchange Case Studies | No (empty article) |
1.18791 | Overview | No |
1.19811 | Overview | No |
2.2352 | Includes Research/KE | This is the article’s parent section |
BOXI output results | ||
Article ID | Title | Is it actually used? |
1.18790 | Overview | Yes |
1.15168 | Overview | Yes |
BOXI results showcase the exact articles that reference another article (and what type of reference it is), but they not include the parent section (this can be easily retrieved, if needed, by the full content tree).
UAT Results
Success criterion | Result |
All historical inbound references appear correctly within the references tab. | PASS Comparison with Polopoly data and actual references revealed that BOXI report is true. - Check if there is a possibility to add the parent section on the results (2.2352 in the above example). |
Objective 4 (Priority: 4)
To ascertain the CMS usage at normal and peak times in order to assist capacity planning for the new CMS.
File | Contents |
O4. Daily Activity.xlsx | Numbers of activity per date. Activity can be anything from logging in to publishing an article. There is no definite action on these actions but Polopoly logs all user activity in a log file. |
O4. Daily Logins.xlsx | Numbers of logins per date. Subset of activity. |
O4. Hourly Activity.xlsx | Number of activity per date and hour. |
O4. Hourly Logins.xlsx | Numbers of logins per date and hour. |
UAT Results
Success criterion | Result |
Who is using the system at which times (peak times for usage) | PASS Output reports for this Objective contain this information collectively per date or per hour/date since 05/12/2011 (more than 2 years before). |
What common tasks are being completed in specific areas of content (range and type of task). | PASS There is no definite list of actions except for Logins (was expected). |
Remark
Daily login data seem to be a bit off, as it can be seen on the graphic below. Login activity seems to be a lot more until mid-August 2012, and then follows a period of lot less overall logins. Our understanding is that this might happens because the Polopoly TEST login details have been overwritten by the LIVE data when the last database copy was made. We need to be certain that this is the case, or else why this change in data appears.
UPDATE AT ASOR: Agnes confirmed this is the case
The spikes of the last three months seem to be consistent with Polopoly training dates (when Polopoly Test usage is probably heavier).
Conclusions
BOXI Reports contain information directly from the Polopoly Test database, so it can be described as accurate as it can be.
- BOXI Filters don’t work as flexible as we would probably want to, so it could be beneficial to export reports to an Excel format and filter data there. We need to investigate if BOXI filters can deliver more flexibility.
- The most important problem is that we cannot directly output the tree for a specific website section without a need to go over all the sections that it contains. For example, searching for section 2.580 on the Full Polopoly Tree (Objective 1) returns all 18 children of that section but not the children of its children sections, so we must go through all the subsections and get the partial trees to create the full UWP website tree.
Document Sign Off
See ASOR milestone https://www.projects.ed.ac.uk/project/uwp005/milestones/acceptance-review-all-objectives