David Mudrak's blog

Since very early versions of Moodle, the Workshop activity module has been the only plugin offering advanced grading based on assessment forms (like rubrics, marking criteria etc). No surprise there was a demand for such feature in other modules too – most notably in the Assignment. The feature request MDL-795 was reported for it in September 2003. Today, that issue can be marked as resolved – advanced grading methods (including Rubric as the first implemented plugin) survived the integration review and were merged into the main development tree. In other words, they will be part of Moodle 2.2. For the user documentation of this new feature see the Advanced grading methods and Rubrics wiki pages (or, if you prefer a screencast, see a nice preview prepared by Mary Cooch linked from there).

Me and a colleague of mine – Marina Glancy from Moodle HQ – formed the development team working on this. The project started by initial discussions and research in late July and August that led to the functional specification document. The spec was published and the community was asked for early feedback on the overall design. The actual coding started in the second half of September and after two months of heavy development, we managed to produce what we consider as a production ready feature. And – which is important in the context of the current development model – we did it in time (well, almost ). It’s a nice opportunity for a short reflection of the process.

For me personally, there were several factors that helped us to finish the project successfully. Let me mention some of them in random order.

Manageable goals and reasonable milestones

The project scope is pretty huge. At the end, advanced grading methods should become natural part of grading interfaces in all activity modules that support grading (Assignments, Glossaries, Databases, …) as well as other places dealing with grading (like essay questions in the Quiz, manually created grade items in the Gradebook). It would take much longer to make such a big change in one step. As a proof of concept, we focused on (a) supporting advanced grading in the Assignment module only and  (b) implementing the only one grading method plugin – rubrics. However, the framework is supposed to be easily extended to other activities and to support other grading methods via separate plugins. On the other hand it also means that the new API fits well the single use case. We are aware of that and we expect that the API will probably have to be improved in the future.

Minimal impacts on the current Moodle code

The advanced grading framework has been designed to extend current Moodle features instead of replacing them. The very last thing in the world I want to do is to rewrite Gradebook. Really, the Gradebook in Moodle is Pandora’s box for every developer. Better not to touch it unless you have written permission (and order) to do so. If you change something, half of the community report it as a bug. If you make it optional, the second half of the community groan the UI is too difficult.

So, instead of replacing current grading interfaces, the framework allows us to “attach” complex grading forms to any part of Moodle that can be described via a combination of context, component, gradable area name and item identifier. For example, let us have a single instance of the Assignment module. It has its context (CONTEXT_MODULE instance), component (“mod_assignment”), gradable area name (“submissions”) and item id (the submission id). This pattern in nothing new in Moodleverse. We use it with success in all the new APIs – files, comments or ratings.

This design allowed us to implement advanced grading into the Assignment module without touching the module’s database tables. Grading forms are seen as tools that calculate the final grade and provide it back to the module. All the data are kept in separate database space. In fact, the activity module does not need to know whether the grade was given directly from the grades selector (the standard way in Moodle) or calculated from a complex rubric.

As a result of this approach, even though the patch touches more than 60 files and adds more than 6000 lines of new code, it removes/modifies just 50 lines of the current code. I believe it definitely made the final integration review easier.

Keep the form definitions close to the place they are used

As you know, things like scales or questions in Moodle are typically defined at higher context levels (in the course, in the course category or for the whole site). That introduces a lot of issues when it comes to modifying the scale or question; or during backup and restore. For example, when you make a backup of an activity using a scale, you need to include the scale definition. But then, when you restore it, you need to carefully decide (read: guess ) whether or not a new scale should be created from the backup. For the same reason, scales in Moodle can’t be altered once they are used by some activity.

In this project, we tried an alternative approach. The grading form definitions are associated with the gradable area they are actually used at. In other words, every single assignment has its own copy of the grading form. It is possible to easily re-use these forms other activities by cloning the form. But as long as a new copy is always created, it was pretty easy to write backup and restore support (well, as easy as writing any backup and restore code in Moodle 2.x is ) for the forms and control modifications of them.

Splitting the development workload

I must admit I would not be able to finish the project on my own. Working on it in the team was a must – and actually a new experience for me personally. All my previous big DEV projects (the new Workshop module, AMOS or restoring 1.9 backups into 2.x) were sort of one-man show. This was the first time I had to cooperate closely with another developer. Those who were involved know that this was pretty challenging for me and the coding was salted with many emotional grrr’s and wtf’s. Today, I must say I am happy I had Marina working on this project with me. It would not happen without her.

The nature of the design allowed us to split the work so I could work on the backend framework that provides general support for all grading methods (managing the gradable areas, cloning the forms, sharing them as templates etc). Marina did a great job developing the Rubric plugin itself and most notably integrated it into the Assignment module. It was not an easy task given the current state of the Assignment module’s internals. Marina has my respect and I sent her Kudo for this.

Continuous peer-review and feedback

We were working on opposite sides of the planet on this project and the time zones difference worked against us. In such situation, I found Github very helpful. It allowed us to work so that Marina sent me pull requests regularly and I merged them into the pre-integration feature branch. We usually started our day by reviewing the changes introduced by the other one.  Github allows to comment the code easily in-line and send feedback to each other. While implementing the rubric plugin, Marina suggested significant improvements of the API this way.

Also, Martin Dougiamas was involved, feeding us regularly with ideas on how the user interface and the general user experience should look and feel like. So, if you like it, praise Martin. If you don’t, blame me and Marina

Current stable version 2.0.1+

Total of 64 patches were accepted by the integration and testing teams during this week for Moodle 2.0 stable branch (which still lives on Git ‘master’). This is a new record in terms of number of pull requests. Also, with only 5 pull requests rejected, this is the best accepted/rejected ratio achieved so far. It also means I am no longer able to provide an overview of all accepted changes so I will focus only on those I helped to test or review.

Moodle can now be installed on PostgreSQL server with ‘standard_conforming_strings’ option enabled. This option is on by default since PostgreSQL 9.1 and Bruce Momjian’s blog post explains why (MDL-26351). ♦ It has been known for a while that MSSQL and Oracle databases do not like SELECT DISTINCT if the list of returned columns contains some TEXT ones. This is tricky especially in a case like SELECT DISTINCT t.*, if the table t contains some TEXT fields. As most Moodle developers use PostgreSQL or MySQL as a part of their environment, some affected queries still survive in Moodle code. Eloy Lafuente found and fixed couple of them, using a nice inner join with subselect solution (MDL-26371). ♦ Mark Nielsen spotted a bug in the implementation of print_collapsible_region() function. Glenn Ansley took this as an opportunity for his first contribution to Moodle, created a branch at his moodle.git clone at github.com and prepared a patch there (MDL-26131). ♦ Andrew Davis fixed a really nasty bug in blogging subsystem that was causing accident removal of all recent records in the table ‘post’. This table holds not only blog posts but also user notes too, for example (MDL-26010). A regression of a recent change in the upgrade code of SCORM module was fixed by Eloy Lafuente (note for myself: next time I test SCORM upgrade code, I shall have some SCORM module instance actually created at the site… MDL-26361) ♦ And finally, images embedded into forum posts that are sent by email to the forum subscribers are now correctly displayed in the email clients. Given that the client has access to the forum either via the current user’s session (in case of web email clients) or the course grants access for anonymous hosts (MDL-25944).

Previous stable version 1.9.10+

All 9 submitted patches were accepted and they landed on MOODLE_19_STABLE branch. ♦ Petr Škoda fixed a problem with redirecting to a login page on https protocol and a bug mediaplugin filter. ♦ Eloy Lafuente found some queries using SELECT DISTINCT from TEXT fields in 1.9 version, too. ♦ Dan Marsden fixed a bug in a pop-up form in SCORM module. ♦ Aparup Banerjee fixed the number of glossary entries being displayed in the recent activity block. ♦ Tim Hunt fixed a bug in quiz that used to display overall feedback based on the rounded grade instead of the real value.

Quotes of the week

“We will ALWAYS have blockers and critical issues, they never end. If we tried to solve them all, we’d never release anything.”
– Martin Dougiamas has a pragmatic view on release policy

“Any code you write yourself should follow all the coding guidelines perfectly. However, when looking at old code, you should be tolerant of what you find.”
– Tim Hunt in a discussion on a policy of naming database tables in Moodle

“Never trust user input.”
– Jonny Barnes at an excellent page summarizing what should a developer know before building a public web site

Is it there yet?

If you are watching some issues in the tracker, you probably know those emails informing you about the change of the issue status. If you receive an email that your favourite issue has been resolved or closed, it is good to understand what exactly it means.

As you probably know, the current Moodle development workflow is based on so called fork and pull model. All developers submit their patches into separate branches in their own public forks (clones) of the official Moodle git repository. Then they ask the integration team for pulling the changes from these forks into the official repository. Although most Moodle contributors seem to use github.com as a place to publish their repositories, we do not use pull requests feature offered by Github. Instead, a PULL issue in Moodle tracker must be created for each submitted branch (patch). In the created PULL issue, the contributor describes the submitted patch and provides information for the integration team necessary for including the patch.

The relevant MDL issue being fixed by the submitted patch is marked as resolved immediately after the PULL request is created. The contributor is supposed to link the PULL issue with the MDL issue and resolve the MDL issue. So resolved means there is a patch available for this issue, but the patch is not part of Moodle package yet.

Every week on Monday, the integration team goes through the list of submitted PULLs and reviews the patches. If the patch is accepted, it is merged into integration.git repository and the status of the PULL issue is changed to “ready for testing”. When all submitted patches are reviewed (that is they are either accepted or rejected), the testers come and test all changes. If the test passes, the tester closes the MDL issue (alternatively, the release manager closes all MDL issues with their tests passed at the end of testing). So closed MDL means the patch has been accepted by both integration team and testing team. But the patch is still not part of the official Moodle package.

You must wait until all integrated patches are tested and the patches are pushed from integration.git repository into the official moodle.git repository. At the same time, ZIP and TGZ packages are regenerated and can be downloaded. This typically happens on Wednesday afternoon (European time).

As an example, let us say I work on some bug. On Thursday morning, I commit a fix into my repository at github.com. Then I go to the tracker, create a new PULL issue and resolve the linked MDL issue. On Monday, a member of the integration team reviews my patch and if I am lucky enough, they accept it. On Tuesday, a member of testing team follows the steps I provided in the PULL requests and hopefully confirms that the patch really fixes the reported issue. The linked MDL issue is closed then. Therefore the next weekly build generated on Wednesday will contain my fix.

Latest stable version 1.9.8+

There are 10 commits into the stable branch from the last week. Half of them, committed by Dan Marsden, deal with various SCORM issues (MDL-22168, MDL-22046, MDL-21568, MDL-18202, MDL-21444). Tim Hunt fixed three Quiz issues (MDL-22241, MDL-20586, MDL-22257). Thanks to Gordon Bateson, Hotpot module now uses HTML OBJECT tag instead of IFRAME, due to XHTML 1.0 strict compatibility (MDL-17702).

Unstable development version 2.0dev

Total of 294 commits found their way into the future version branch last week. Several various subsystems and major patches were included – for example the new My home page, Moodle Hubs support and many others. Petr Å koda’s recent patch, that reimplements the whole enrolment subsystem and changes some of the key concepts of roles and their assignments, is still being reviewed in a separate branch. Martin Dougiamas is going to announce the release of Moodle 2.0 – Preview 1 version, just working on the release notes yet.

Quotes of the week

“Users will remember for almost forever a bad/broken release; but will quickly forget that a stable/fine/working release was a year late.”
– Greg Mushial summarizes his experience from 40+ years involved in software development

“Beta version software is likely to be useful for internal demonstrations and previews to select customers. Some developers refer to this stage as a preview, a prototype, a technical preview (TP) or as an early access.”
– Wikipedia, Software release life cycle

Help tooltips in Moodle 2.0

Among many other (and definitely more significant and more important) improvements in Moodle 2.0, there is a new way how the help tooltips are handled. From the users’ point of view, help is still available via those question mark icons. Instead of opening a pop-up window in the browser, Moodle 2.0 uses YUI3 overlay to display a “bubble” with the help text. The behaviour of the tooltip had been a subject of a discussion (MDL-22067) and at the end, it was decided to follow the functionality found in Mahara or Github.

From the developer’s point of view, help texts are not stored in separate files any more. In the dark past, help files used to play a role of inline help, documentation, manuals or even tutorials on using Moodle. Nowadays, with Moodle Documentation portal being actively growing thanks the community contributions, we are trying to revert helps to what they are supposed to be – useful hints on various user interface elements, their meaning and usage. Therefore, the help files are being migrated into proper Moodle strings in 2.0.

For every help, two strings must be defined. One for the label of the thing the help is describing. In the forms, this is the label of the input element in almost all cases. The second string is the help text itself. The identifier of the second string must be the same as the identifier of the first one, just with “_help” suffix, for example:

$string['subject'] = 'Subject';
$string['subject_help'] = 'Message subject will be used as a title at the message view page.';

Let us say we want to add such help in a Moodle form, for ‘msgsubject’ input field. You would probably have a code like:

$mform->addElement('text', 'msgsubject', get_string('subject', 'mymodule'));
$mform->addHelpButton('msgsubject', 'subject', 'mymodule');

That would use the string ‘subject’ defined in your ‘mymodule’ language file as a title of the help and the string ‘subject_help’ defined in the same file as the help text. To get the same result in Moodle 1.9, you would have the help defined in a HTML file, let us say help/messagesubject.html a the code in the form would look like:

$mform->setHelpButton('msgsubject', array('messagesubject', get_string('subject', 'mymodule'), 'mymodule');

To display a standalone help icon anywhere at your page, use help_icon() method of the core renderer instance:

echo $OUTPUT->help_icon('subject', 'mymodule');

Help strings can use Markdown syntax for basic HTML formatting. We use that just for producing bullet list from the lines starting with asterisk.

To give the translators something to start with instead of rewriting all help strings from scratch, we are copying the contents of the translated help files into new help strings wherever it is possible.

Post scriptum

I can’t think of anything to write here today