User experience maturity measurement

Select maturity model

msg as a group is very diverse in terms of professional and technical expertise, active in a wide range of industries and spread across several business divisions and subsidiaries. This also applies to the user experience professionals. Some work alone, others in teams. Some work in customer projects, others in product teams. The UX maturity assessment was therefore not only intended to find out how well we are generally positioned in the area of user experience, but also how the different areas of the Group compare with each other and where there is potential for improvement.

With these requirements in mind and after comparing various maturity models, we chose the Customer and User Centered Maturity Model, or CUXM for short. This model is freely available and currently provides a questionnaire consisting of 23 questions in eleven categories. The model was developed as part of a community project and is being further developed by this community.

Adapt maturity model

For msg as an IT service provider, however, the 23 questions and six maturity level descriptions of the CUXM model were not entirely appropriate. However, it is crucial for the quality of the results that the questionnaire and the assessment are understood within the company and take into account the respective work context. We revised the CUXM accordingly before starting the measurement.

We developed a multi-stage process to adapt the CUXM questionnaire to our specific requirements. This included an initial adaptation of the questions by two UX professionals and, building on this, a revision in a workshop with UX professionals from various units of our Group. With this first version of the questionnaire, we carried out the UX maturity level measurement in two pilot business units. This was followed by a further adjustment, a third pilot and finally the final revision of the questionnaire.

The result is three questionnaires that are largely the same in terms of content. Only the wording of the questions varies and is based on the working environment of the target group surveyed. One questionnaire is used to measure an entire unit, i.e. a department, a business unit or a subsidiary. The other two questionnaires are designed to measure UX maturity at team level. One focuses on teams working on customer projects, the other on teams working on msg products. In this way, all participants receive questions that fit their environment.

Measuring maturity

Measuring the UX maturity level for a business unit involved four steps for us. As we were in the adaptation process, we decided to complete the questionnaire together in the form of individual interviews. This offered the advantage that problems in the concept of the questionnaire would be uncovered during the interview and allowed us to ask questions for a better understanding.

Conducting interviews: After the first interviews, we learned that this approach offers even more advantages. We learned a lot about the external perspective on UX, especially when interviewing people from outside the field. These interviews can also be seen as coaching on UX. Due to the content of the questionnaire with the different categories, the interview participants get to know the whole variety of UX. The structure of individual questions with six different answer options shows the development possibilities very well. This also enabled our interview participants to learn a lot.

In the end, we found out why the UX maturity level is the way it is. Because we recorded all statements, nuances, assumptions and ignorance. By combining this with interviews, the UX maturity measurement itself became and remains more important than the result.

Follow-up online survey: We followed up the interviews with an online survey. This step is optional. It depends on the size of the area or team to be measured, as well as their geographical distribution or spread across time zones. The online survey broadens the database.

Hold a workshop: A workshop is not optional. We brought the participants of the interview step together for a two-hour meeting with a clear agenda. We start with a brief introduction to user experience in order to pick up any non-specialists and establish a general basic understanding. This is followed by a presentation of the results of the maturity level measurement. It explains the level of the measured area and the results in the individual categories. After the presentation, we enter into a discussion with three selected questions from the questionnaire. We choose questions with a wide range of answers.

The next item on the agenda is the analysis of the current situation. It sets out what currently works well and what works less well in terms of user experience in this area. Based on this, we develop a target vision and clarify where the area would like to develop in terms of user experience over the next two years and how they will achieve this. It is particularly important here to define measures that are as concrete and realistic as possible. At the end of the workshop, we formulate the next personal steps to be taken in the coming weeks.

Schedule a follow-up appointment: To review these personal next steps, a follow-up meeting can take place around two months after the workshop. At this appointment, the measures and goals from the workshop are reviewed and further tasks are assigned.

Conclusion

The measurement of the UX maturity level itself is more important than the pure key figure. The key figure does say where a department or team stands. However, the survey itself explains why a department or team is where it is. Only this combination of quantity and quality guarantees that improvement measures are possible.

Select participants: Successful UX maturity measurement requires not only an adapted questionnaire and a targeted approach, but also a suitable group of participants. The more heterogeneous, the better. We benefited in particular from the different expertise of the respondents. Instead of only considering user experience professionals, we specifically added representatives from the disciplines of requirements engineering and business analysis, development and operation, product and project management.

Involve management: Managers from higher levels, such as division heads, should definitely take part in the maturity level measurement. They are often unable to answer questions about project procedures in detail. However, they can find out personally how the assessment was arrived at. In this way, organizational and procedural framework conditions can be changed more quickly and easily.

Keep at it consistently: The measured level of maturity is not the end of the process, but the beginning. Everyday life quickly swallows up all good intentions, and measures are forgotten or come to nothing. It is therefore important to plan the process as a continuous improvement process from the outset.

Set priorities: The goal is not to reach the maximum level of the maturity model for all units and teams. Nor is it necessary. The important thing is to examine the best practices depicted in the maturity model and to consciously prioritize and focus on your own goals.

Outlook

As a next step, we plan to carry out the UX maturity assessment in other units as well as in project and product teams. In the medium term, we are aiming for regular evaluations and an exchange between the evaluated units.

We are already working successfully for our customers as user experience experts. With the maturity level measurement, the topic of user experience will continue to develop within the msg group over the next few years.