This article explains how to navigate to your Practice test insights page, which includes numerous metrics about how learners are interacting with your practice tests.
- Practice test insights are available for instructors who have published a practice test. Learn how to create a practice test.
Table of contents
- How to navigate to your Practice test insights page
- Practice test insights page features
- Overall test stats
- Performance funnel
- Version history
- Question analytics
- Flexible date range selector
- Inconsistent data error
- Question quality
How to navigate to your Practice test insights page
You can find the Practice test insights page as part of the instructor Performance features:
1. Navigate to the Instructor view (learn how).
2. Select Performance on the left-hand side.
3. Under Engagement, select Practice test insights.
Practice test insights page features
On the Practice test insights page, you can explore a wide range of data for any course with at least one practice test. Select the dropdown at the top to select any applicable course.
Then, use the date range picker to choose preset timeframes or pick your own custom date range.
If you have practice tests included in the Udemy Business content subscription, metrics for Udemy Business learners are included in the Practice Test Insights data.
1. To view performance data specific to Udemy Business learners, select the Udemy Business tab near the top of the page.
2. Select the “Edit practice test” button to update your practice test based on the data you see (learn how).
Overall test stats
The average number of attempts section shows you the average number of times learners re-take the practice test and the number of learners this metric is based on. This is calculated by dividing the total number of attempts within the time period by the total number of unique test takers.
The average time spent section shows the average time spent by learners who passed the test and the number of learners who passed.
Please note: Data is processed and available to view in real time.
Performance funnel
In the performance funnel section, you can view the number and percentage of learners who started, completed, and passed the practice test.
The data displayed in the performance funnel shows all learner attempt data for a selected date range.
Version history
The version history section helps you to keep track of your edits.
- Review the version log to see all updates you have published to practice tests, along with the dates that each version of the test was available to learners.
- Click “filter date range” to view practice test insights for the date range in which that particular version was, or is currently available to learners.
Pro tip: Be specific
When you update your practice test, add detailed notes to better understand how learners respond to different versions of the test.
If you reference a certain question, be specific about what the question was. Your learners are shown the questions in a random order. So, if you only say, "Updates on question number 5," your learners won’t be able to follow.
Remember, the words you use here are also visible to your learners, so the more specific you are, the better!
Question analytics
In the question analytics section, you can dive deeper into each question to determine where learners got stuck.
- Use the search box to quickly find questions by searching for keywords.
- Select the dropdown to filter questions by success ratio.
- For each question view the average time spent, number of learners, and success ratio. Select “show details” to see a breakdown of which answers learners chose.
- Select “edit question” to jump right into the editor so you can make updates to your practice test based on practice test insights.
Note: The data displayed in the question analytics reflects how learners responded the first time they saw a question. For learners who retake tests, only the first response will be included in the data.
Flexible date range selector
With the flexible date range selector, you can view insights for a customized period of time.
Inconsistent data error
When viewing insights across multiple test versions, the combined data may be misleading if the versions differ significantly.
Major updates to questions, such as changing answer options or correct answers, can lead to inconsistencies in question analytics. In such cases, an Inconsistent data message will be displayed.
If you see an Inconsistent data error, please use the version log to select the date range for a single version of the practice test. This approach lets you analyze how each version of the question performed separately, providing you with accurate data.
Pro tip: To improve long-term insights when making major changes to questions, here are some best practices:
- Delete and recreate questions
- Avoid editing existing questions significantly
- Maintain the position of the correct answer (For example, keep the correct answer in position 4 in both versions)
Major changes can include:
- Altering the question's core meaning
- Changing the number of answer options
- Completely replacing an answer option
Question quality for multiple choice questions
For the latest version of your practice test, we calculate a question quality rating for multiple choice questions that 50 or more learners have attempted.
The question quality rating is based on two key factors from classical test theory (CTT):
- Discrimination: Discrimination refers to how well a single question is contributing to our understanding of a learner’s performance on the practice test. A low discrimination item can signal a question that is confusing, ambiguous, has multiple correct answers, or is even inaccurate.
- Difficulty: Difficulty refers to the proportion of learners who get a question correct (the higher the number the easier the question). Questions that are very easy or very hard can signal problems with a question.
While viewing the latest version of the practice test, you'll see one of three ratings next to multiple choice questions:
✅ Good
- Question is performing as expected
- Data indicates learners have understood what's being asked
⚠️ Needs attention
- Data indicates a potential for improvement.
- Your expert judgment can validate whether or not this is true.
- Review practice test insights and decide if you’d like to make an improvement
📊 Not enough data
- Fewer than 50 learner attempts
- Rating pending more responses