Original post by Adam Olley April 29, 2020


With the current state of the world, most institutions are looking to perform an increasing number of their quizzes/tests/exams in Moodle and are concerned about the potential performance implications that holds for their Open LMS Enterprise instances.

Observed usage

The following table lists some of the larger quizzes (those with a time limit, >200 quiz takers and a relatively short time window) we’ve seen in 2020:

Time limit (hours)Time window (hours)Takers
0.751.08462
1.501.50289
1.001.50248
1.535.00201
1.251.33396
1.671.75490
1.501.50374
1.004.001184
0.921.50312
1.002.00218
0.750.83518
1.501.50210
2.002.25456
0.580.83252

 

Time limit: The length of time a user has to complete the quiz after starting an attempt

Time window: The length of time between the set quiz open and quiz close times

Takers: The number of unique users that attempted the quiz

With the exception of the last entry, none of these quizzes had any performance issues detected by our monitoring or reported by users.

The root cause of the performance issues that occurred with the final one in the list was caused by a user editing the course whilst the quiz was starting. Editing a course requires the course cache to be rebuilt - and users of a course have to wait while that happens. That, coupled with the short time window available to take the quiz meant that a lot of users were already in the course.

Recommendations

Avoid editing the course

Editing a course should be avoided completely when a large quiz is set to occur - or in fact when any large number of users are expected to be active within the course. Another example would be when an assignment is due - for some courses the busiest time is immediately prior to assignment due dates.

Editing a course is not just limited to editing activities and resources - it includes all actions that alter the course when you have the editing mode turned on. This includes altering course sections (toggling visibility, adding new ones, etc), simple renames of activities.

When a course is edited, Moodle needs to rebuild the cache associated with that course. Whilst this cache is building, any users trying to use the course need to wait. Most of the time, this rebuild is only a few seconds at most and doesn’t cause major issues. But for more complex courses, or depending on the plugins enabled, a course cache rebuild can take longer than a few seconds - and several hundred users waiting on that can use up a lot of the available PHP worker threads used to serve the site.

Larger windows

Where possible, it’s beneficial to have as large a time window as is practical for taking a quiz. For example, a 1 hour quiz that is only open for an hour guarantees that 100% of the people that want to take the quiz will be trying to do so within that 1 hour timeframe. Whereas a 1 hour quiz with a 3 hour window means the quiz takers will be more spread out as to when they start the quiz, reducing the load.

Restrict access over time

Another option available is by using the “Restrict access” capabilities to open a quiz to different groups (or groupings) in the course at different times.

Here’s an example of a quiz configured to only be available to users of Group “G1” from midday, and then to open further to users of Group “G2” from 1 PM.

 

 

Such a setup lets you forcefully stagger when the quiz is available to users. Combined with a quiz time limit, you can still ensure no one has more time in the quiz than anyone else.

Consider enabling “Skip regrades on quiz view”

This is a performance customisation developed for Enterprise (BASE-2301) in response to performance issues we observed for some courses with complicated gradebook setups and large numbers of users.

When viewing a quiz, it checks to see what grade the user achieved in the quiz to display it on the page (it does this check regardless of if the user has yet attempted the quiz). Most of the time, this is fine. However if a course has a full gradebook regrade pending - which is where Moodle runs through all the data in the courses gradebook and performs all the necessary calculations to make sure everything is up to date - then this check triggers that regrade.

For courses with complicated gradebook setups and/or large numbers of enrolled users and graded activities, this calculation can take long enough that users trying to view the quiz will see long load times until the regrade operation completes.

By enabling this setting - simply viewing a quiz will not trigger a full regrade if one is pending. The regrade operation will still occur under the usual circumstances elsewhere (staff viewing the gradebook, etc).

Several clients have this enabled already, some for several years, and this has proven to be a solid benefit to the operation of the quiz module at scale.

This can be enabled via the UI by an administrator under “/admin/settings.php?section=local_blackboard_misc” - and can be disabled at any time just as easily.