Acceptance Tests and Outcomes

 

UWP005 UAT & Report data

 V1.2 by Stratos Filalithis, 7/2/2014

Overview

This document contains conclusions and reports on UAT for project UWP005. The reports were delivered using a BOXI Universe and the respective application. BOXI Functionality was explored and reports were exported to Microsoft Excel for further examination. The outputs were created based on the 4 objectives described in the beginning of the project.

The output files can be found here:

O:\projects\WDP\secure\Polopoly Content Audit Reports (UWP005)

All files have a similar structure with similar fields. For the most important, a small description is:

Field

Description/Content

Root

Has the value of 2.0 all reports feed data for the whole of the Polopoly Tree

Parent Item

The parent section for this record

Child Item

The child section/article for this record (usually a unique value, depending on the report)

Item Name

The section/article title

URL

The full Polopoly URL (not the same as the website URL)

 

Objective 1 (Priority: 2)

To audit the structure and metadata of the CMS content in preparation for migration to the new CMS and export this audit in csv file format.

File

Contents

O1. Breadcrumb trail (for UWP Homepage).xlsx

The full breadcrumb trail for the UWP Website Homepage. This is the full Polopoly breadcrumb, so it contains more information than the usual Website URL, but it can probably get removed by a massive find/replace operation.

O1. Breadcrumb trail (WholeTree).xlsx

This contains the full breadcrumb for all the articles on Polopoly. Keep in mind that this means all articles, even those and the MediaBank section, so that’s why there is a whopping 147000+ records on it.

O1. Full Polopoly Tree (UWP Website - 2.580)

All children of section 2.580 (UWP Website Section)

O1. Full Polopoly Tree (UWP Website Dev Area Unpublished- 2.1307).xlsx

All children of section 2.1307 (UWP Website Dev Area, which is unpublished)

O1. Full Polopoly Tree (Vet School Website - 2.2371).xlsx

All children of section 2.2371 (Vet School Website)

O1. Full Polopoly Tree (Whole Tree).xlsx

The full Polopoly Tree. This contains all articles, including Media Bank.

O1. Polopoly Orphaned Component Tree

All the orphaned Polopoly content, meaning content that was disconnected from their parent.

O1. Polopoly Orphaned Component Tree (Dev Area – 2.1307)

All the orphaned content from the Dev Area.

 

 

UAT Result  

Success criterion

Result

Scripts which can be run by either run by CMS Managers or run at their request through a support call. The output is in the form or a csv file.

PASS The reports can be created directly from BOXI and exported to several formats, including Excel readable

Ability to produce complete content structures from any section or subsection in the CMS and should include the UoE, Media bank and Development Zone areas of Polopoly

FAIL Only children are shown when filtering and not grandchildren. Is there a way to filter the results in a way to show the full tree structure below a specific section? Note: It will be addressed by the developers, by updating the output.

 

Update: UWP had full structure of tree but could not filter it. Agnes had been waiting on confirmation on how to do this from UWP which they have now given her. This will only involve removing one clause and will not hold deployment up. Agnes will implement immediately and Neil will follow up after Agnes’ change.

 

Retest: PASS

Stratos retested the update on 11 Feb and is now a PASS

Ability to export Polopoly structures to .csv including information such as key metadata fields, published/unpublished status and all out bound links for further analysis

PASS

Reports include key metadata (incl. workflow & published state) but not all outbound links (Is this addressed by Objective 3?)

 

Objective 2 (Priority: 3)

To express the relationship between users, groups, workflow and permissions on sections to allow CMS managers to compare this with the CMS structure and metadata. This will inform and simplify the recreation of workflows in the new CMS.

File

Contents

O2. Group - User.xls

Contains all Polopoly user groups with users added to them

O2. Group - Workflow - Permission.xlsx

Contains all user groups with workflows and permissions assigned to them

O2. Group - Workflow - Section.xlsx

Contains all user groups with workflows and permissions assigned to them per Polopoly section

 

UAT Result  

Success criterion

Result

Scripts which can be run by either run by CMS Managers or run at their request through a support call. The output is in the form or a csv file.

PASS The reports can be created directly from BOXI and exported to several formats, including Excel readable

Ability to see what sections each group is associated with

PASS Report O2. Group – Workflow Section includes this information

Ability to see what groups are associated with a section and it’s children

PASS Report O2. Group – Workflow Section includes this information Note: There is no visible connection between parents & children on this report. This information should come with feedback from other reports.

Ability to export this information to .csv for further analysis

 

PASS The reports can be created directly from BOXI and exported to several formats, including Excel readable

Breadth and depth of groups workflow permissions - mandatory

 

PASS O2. Group Workflow Permission report contains this information

Ability to see what groups an individual user is associated with

 

PASS O2. Group User report contains this information

Ability to see what workflows each group is associated with

 

PASS O2. Group Workflow Permission report contains this information

Ability to export this information to .csv for further analysis

PASS The reports can be created directly from BOXI and exported to several formats, including Excel readable

Objective 3 (Priority: 1)

To have the information on shared content references within the CMS in order to rebuild these references within the new CMS.

The output file structure fields are:

Field

Description/Content

User Item

The Polopoly item that references another Polopoly item (similar to parent)

User Item Component.Item Name

User Item’s Title

User Item Media.Media

Is the user item a media article?

Usertype

Type of reference (i.e. include article, etc.)

Used Item

The Polopoly item that is referenced by the User Item (similar to child)

Used Item Component.Item Name

Used Item’s title

User Item Media.Media

Is the used item a media article?

Output files list:

File

Contents

O3. Content References (UWP Home - 1.86349).xlsx

What articles are referenced in article 1.86349 (UWP Website Home)

O3. Content References (Where UWP About Us is used - 1.86380).xlsx

What articles are referencing article 1.86380 (UWP About Us)

O3. Content References.xlsx

Full list of referencing and referenced articles

Specific example

There are some examples under Unidesk call I110316-0389. One of them is used below to showcase the difference between the `References` tab on Polopoly and the BOXI output.

The included article id is 1.17200.

Polopoly `References` tab results

Article ID

Title

Is it actually used?

1.18790

Overview

Yes

1.18814

Overview

No

1.15168

Overview

Yes

1.12366

Knowledge Exchange Case Studies

No (empty article)

1.18791

Overview

No

1.19811

Overview

No

2.2352

Includes Research/KE

This is the article’s parent section

 

BOXI output results

Article ID

Title

Is it actually used?

1.18790

Overview

Yes

1.15168

Overview

Yes

BOXI results showcase the exact articles that reference another article (and what type of reference it is), but they not include the parent section (this can be easily retrieved, if needed, by the full content tree).  

UAT Results  

Success criterion

Result

All historical inbound references appear correctly within the references tab.

PASS Comparison with Polopoly data and actual references revealed that BOXI report is true. - Check if there is a possibility to add the parent section on the results (2.2352 in the above example).

 

Objective 4 (Priority: 4)

To ascertain the CMS usage at normal and peak times in order to assist capacity planning for the new CMS.

File

Contents

O4. Daily Activity.xlsx

Numbers of activity per date. Activity can be anything from logging in to publishing an article. There is no definite action on these actions but Polopoly logs all user activity in a log file.

O4. Daily Logins.xlsx

Numbers of logins per date. Subset of activity.

O4. Hourly Activity.xlsx

Number of activity per date and hour.

O4. Hourly Logins.xlsx

Numbers of logins per date and hour.

 

UAT Results  

Success criterion

Result

Who is using the system at which times (peak times for usage)

PASS Output reports for this Objective contain this information collectively per date or per hour/date since 05/12/2011 (more than 2 years before).

What common tasks are being completed in specific areas of content (range and type of task).

PASS There is no definite list of actions except for Logins (was expected).

Remark

Daily login data seem to be a bit off, as it can be seen on the graphic below. Login activity seems to be a lot more until mid-August 2012, and then follows a period of lot less overall logins. Our understanding is that this might happens because the Polopoly TEST login details have been overwritten by the LIVE data when the last database copy was made. We need to be certain that this is the case, or else why this change in data appears.

 UPDATE AT ASOR: Agnes confirmed this is the case

The spikes of the last three months seem to be consistent with Polopoly training dates (when Polopoly Test usage is probably heavier).

Conclusions

  • BOXI Reports contain information directly from the Polopoly Test database, so it can be described as accurate as it can be.

  • BOXI Filters don’t work as flexible as we would probably want to, so it could be beneficial to export reports to an Excel format and filter data there. We need to investigate if BOXI filters can deliver more flexibility.
  • The most important problem is that we cannot directly output the tree for a specific website section without a need to go over all the sections that it contains. For example, searching for section 2.580 on the Full Polopoly Tree (Objective 1) returns all 18 children of that section but not the children of its children sections, so we must go through all the subsections and get the partial trees to create the full UWP website tree.

 

 

 

 

 

 Document Sign Off

See ASOR milestone https://www.projects.ed.ac.uk/project/uwp005/milestones/acceptance-review-all-objectives