David Mudrak's blog

Latest stable version 1.9.7+

There were 12 commits into MOODLE_19_STABLE last week. Penny Leach added support for local events and running event queue in cron (MDL-17442). Eloy Lafuente committed a calendar patch prepared by Daniel Neis (MDL-17510) and another one by Darko Miletic from UVCMS, our Moodle Partner. Helen Foster fixed two help files, reported by Bente Olsen (MDL-21053, MDL-21054). Petr “frankenstein” Skoda fixed several input validation bugs, credit goes to Sascha Herzog for spotting them.

Unstable development version 2.0dev

There were 28 commits into the main development branch. Besides merges from the stable branch, various work in the Gradebook, Lesson module, themes, YUI, formslib, Quiz module and other areas of the code has been done.

Quotes of the week

“I never liked the frankenstyle plugin-naming scheme (coursereport_outline) … it started as an ugly hack to get around our own limitations in capability names and then took off like a virus all over the code.”
– Martin Dougiamas

“Arrrgh, the $PAGE magic is driving me crazy!”
– Petr Skoda

New algorithm of permission evaluation

On Sunday, Petr Skoda committed first parts of his recent work on the new permission evaluation, roles and enrolments changes for Moodle 2.0. The patch is quite big, affects 26 core files, inserts 1765 new lines and removes 2075 current lines of code. Compared with the current approach, the new way of calculating the effective user rights is significantly modified. In vast majority of cases, if role permissions and their overrides are used in a sane way, the new algorithm is backwards compatible, though.

The concept of roles, capabilities, permissions and their overriding at various context levels is one of the most powerful aspects of Moodle. Capabilities allow to tweak the behaviour of activity modules to fit the current needs. However, it is not easy (or sometimes even possible) to explain how the whole machinery actually works. Although Tim Hunt has prepared an useful tool to visualise how the permission is evaluated in the given context for the given user, it may be a tricky job to make sure the module works as expected in all thinkable situations. To be fair, it must be said that usually these tricky situations are very theoretical ones. Anyway, in Moodle 2.0, every time a permission check is done – for example by has_capability() call – the following steps are proceed.

• Firstly, a list of all roles that have something to do with the given permission or its override is populated.
• Then each of these roles is evaluated separately to look up for the roles that gives the user the permission in the given context.
• If the user is assigned any of the roles that gives him/her the evaluated permission, he/she is able to perform the controlled action.

This new way is expected to fix several known problems where a teacher was not actually able to guess or influence the resulting permission. From the usability perspective, it allows to design teacher-friendly and intuitive user interface to override role permissions. From a developer’s point of view, it allows to integrate capability check into your own SQL statement which generally increases the performance. See MDL-21710 and MDL-21711 for more details of this.

Post scriptum

$ git traffic –help
`git traffic’ is aliased to `log –after=”last monday 00:00″ –until=”last sunday 23:59″ –format=”%Creset%h by %an %Cgreen%s”‘

Latest stable version 1.9.7+

There were 13 commits into the stable branch last week. Dan Marsden fixed a bug in importing SCORM 2004 packages (MDL-21584). Tim Hunt fixed the random order of responses shown in their history (MDL-21701). Dan Poltawski committed a set of patches fixing various bugs here and there, including gradebook overview report and hidden role assignments during metacourse sync (MDL-20683, MDL-20993, MDL-21192, MDL-21284, MDL-21424, MDL-21593, MDL-20626). Helen Foster fixed the explanation of the course maximum upload size (MDL-21546). Finally, Eloy Lafuente fixed the wrong time display in logs exported into Excel format (MDL-14934).

Unstable development version 2.0dev

There are 44 commits into the main development branch (future Moodle 2.0) from the last week. Some bigger projects (like Eloy’s new backup and restore subsystem or Tim’s new question engine) are cooked in separate git branches and are waiting for their Merging Day to come yet. Petr Škoda started to work on enrolments and capabilities evaluation improvements (MDL-21655). Guys in Perth continue their work on webservices, HTML output rendering and other parts of the new code.

Quotes of the week

“There is really no pressing reason to choose a particular weekly [build] and label it 1.9.8 and spending time on all the release process that entails.”
– Tim Hunt

“Help files are tooltips, not the manual.”
– David Mudrak

Hello, can you speak Moodle?

Almost a year ago, our translation coordinator Koen Roggemans started a discussion on how to improve the translation process in Moodle. The thing evolved into quite a big project that tries to deal with the most important issues immediately and to open the way for sorting out the others in the future. The research, proposals and progress are documented at Development:Languages. When Moodle 2.0 beta is released, there will be a central portal prepared. This central portal is part of new AMOS system to store and organise the strings and their translations. Its main job is to take care of string versions tracking and branching, so Moodle translators do not need to deal with strings that are not used any more, they can be notified when new strings are added or even changed in the English source etc.

Interesting and important discussion started as a response to the proposal to replace current help files (stored in separate HTML files) with their significantly shortened versions (maintained as normal strings). Some (of us, have to say) consider help file as a short tooltip that should shortly explain the usage of the given page element – in several sentences, one or two paragraphs of text at most. Anything else, like detailed explanation of the feature, step by step tutorials etc, should be moved into our wiki docs. There are objections against this, though. Some of our Moodle Partners pointed out that Internet connectivity may be issue for some institutions running local Moodle servers and moving the information from help files (which are available locally from the server) to the remote wiki server would actually make them not-available.

Post scriptum

The snow is slowly melting and the spring unstoppably comes. :wq

(1) make coffee – unfortunately this does not work as Linux command yet on my machine. I should find some time to implement http://tldp.org/HOWTO/Coffee.html

(2) Start-up Vimperator and triage my email inbox, using a variant of GTD method.

(3) Connect to Moodle developers and Moodle commits Jabber rooms

(4) $ cd public_html/moodle-head

(5) $ git fetch

Download the recent commits from our git mirror of the main CVS repository.

(6) $ gitk master..origin

This gives me an overview on what happened in the code since the last time I merged.

(7) $ git checkout master && git pull

Fast-forward my local master branch so it contains the recent commits.

(8) $ git checkout fix/somebug-MDL-xxxxx

I love branching. I do every development (even a simple bug fix) on a separate branch.

(9) $git cvsexportcommit -c -p -u -v <hash>

Once the fix is committed on my branch, I export it to CVS. Thanks to my set-up everything happens really quickly. If the fix is done on a stable branch I have to move the merged-status tag manually yet in the CVS checkout, which is a bit annoying. Once we move to git (note not “If we moved to git” :-p) thing will become even easier.

Latest stable version 1.9.7+

During the last week, 23 patches were committed into MOODLE_19_STABLE (commits by Moodle Robot excluded). Many of them sent by Penny Leach are fixing various MNet issues.

Quote of the week

“I commited junk from another file as part of commit. Sorry everyone. I’m to use emacs for the weekend as punishment.”
– Dan Poltawski

Output rendering in Moodle 2.0

Outputting HTML in Moodle 2.0 was a big project started by Tim Hunt last year, continued by Nicolas Connault, Sam Hemelryk, Petr Škoda and others. After some refactoring, it seems to be stable now. This is an example of basic usage in your module:

// in mod/mymod/view.php
$output = $PAGE->get_renderer('mod_mymod'); // (1)
$welcome = new mymod_greeting(); // (2)
$welcome->set_message(get_string('helloworld', 'mymod')); // (3)
echo $output->header(); // (4)
echo $output->render($welcome); // (5)

HTML code to be sent to the browser is generated by $output which is an instance of a rendering class. In (1) we get the renderer for our module. This is instance of class mod_mymod_renderer which must be defined in mod/mymod/renderer.php. It knows how to display our module specific page elements.

In (2), we create an object that represents an element to be displayed. It may be a forum post, submitted work by student or anything that your module is dealing with. In our case, the object $welcome represents a welcome message to be displayed. The rule is that the class mymod_greeting must implement the interface renderable. This interface itself is empty but implementing it is a sign for Moodle core that the instances of this class can be passed to the render() method later.

In (3), we set a property of the renderable widget. In this simple case, it would make sense to pass the localized welcome message as a parameter of the $welcome constructor. I intentionally used the explicit way to underline that $welcome is an object and that additional properties may be set or methods to prepare the object can be called.

In (4), we ask our renderer to display HTML page header, navigation trail and other standard components. Our class mod_mymod_renderer does not do this job itself but passes the call to the underlying core renderer automatically.

Finally in (5), we display our welcome message. The method render() accepts a renderable object and calls our own method render_greeting() defined in our renderer. This method accepts the only parameter which must be instance of mymod_greeting.

For the detailed description of the whole system and nice UML diagrams of class dependencies see the documentation page.

About

I love Linux kernel traffic and kernel trap newsletters and enjoy reading its Czech translation. So I decided I could try and write a short blog post every Monday on what happened in Moodle development world last week. Take this as an attempt. Maybe it survives, maybe not. This is the first post in the series. The number 07/2010 means the last week was the 7th week in the year 2010 according to my phone calendar (should be the ISO week numbering scheme).

A new book called Mahara 1.2 E-Portfolios: Beginner’s Guide has been released by Packt Publishing recently.

This book is your step-by-step guide to building an impressive professional e-portfolio using Mahara. It covers the key features of Mahara that will help you set up your customized digital portfolio and display the artefacts in your preferred way allowing contribution from selected users only.

I am currently waiting for a copy of the book and will write a review soon into this blog. Sample chapter is available for free in PDF format.

1. Show your real face – Stephan Rinke and Michael  Tighe were right in their excellent workshop “In Moodle We Trust“. Your picture is important in an on-line environment. Recognizing the human face is one of the first things we learn as babies and it remains being important through all our life. Showing your real face express your honesty to the others and the fact you do not hide anything. It establishes the first contact with other people and it really helps if you know who is the guy giving the talk.
2. Tell where you are, what time it is and what the weather is – Helen Foster started her excellent keynote presentation with a very kind introduction of herself. If people come to your session at different time zones, from different countries and in different seasons, it is nice they can imagine you in a real-world context. Are you sitting in the office? Or giving a talk from the bed? Is there your family around? Is it snowing there or did you just come from surfing at a sunny beach?
3. Invest time into preparing the slides or demonstration – these kind people come to your session and they are going to spend an hour just listening to you – paying their attention to what you say. Give them clear sign that their presence is important to you and that you were preparing for it. Tim Hunt had a great idea using questions in the new Quiz module as a way to show his slides. It was clear he was carefully thinking about what to say and how to explain it in advance.
4. Bullet lists in presentation slides sucks – presentation is not a book. One picture, even a simple sketch drawn by your hand, may express more than dozens of the words.

But we had to celebrate and be glad, because this brother of yours was dead and is alive again; he was lost and is found.

– (Luke 15:31-32, NIV)

Yesterday, new workshop module has been committed into Moodle CVS HEAD and became a part of standard Moodle 2.0 distribution again. It’s been a long way to get into this stage but the real journey just starts. There are still some features to tune up and some known bugs to fix. The module seems to work pretty well, though. I had some real testing data from workshop 1.9 courses available (many thanks to those who could provide them) and from my point of view, the new UI gives clearer overview and more control over the activity.  However, my opinion does not matter – our good Moodle users will have to say what works for them and what does not.

Most of the legacy workshop features remained. There are two most significant differences. Firstly, new workshop puts two grade items into the course gradebook – a grade for submission (it est a grade that students get for their own work) and a grade for assessment (a grade that estimates the quality of the peer-assessment). Legacy versions of workshop automatically summed up these two grades. Nowadays, Moodle gradebook allows more way of aggregations. Teachers can set up their gradebooks to mimic the old behaviour, of course.

The second significant breakage of backward compatibility is in how the assessments of example submissions are evaluated. Example submissions are provided by teacher and students are expected to train the assessment process on them. Teacher can, for example, put an example of a really good work and a really poor one. In previous workshop version, grade for the assessment of example submissions was immediately calculated so the student got a feedback like “if you assessed this submission in this way, you would receive this grade for assessment”. In new workshop, evaluation of students’ assessment is more dynamic process. Teacher can use different methods of the grade calculation and change the input parameters of the calculation “on the fly”. Therefore, workshop module itself is not able to calculate the grade for assessment without teacher assistance. Instead, teachers just provide so called reference assessments of example submissions (where they express the quality of the submission) and such benchmark is then displayed to students. So students can compare their assessment with the teacher’s one but no grade can be calculated.

I hope the other improvements – namely the possibility to control the submission allocation for peer-assessment, manual switching of workshop phases, overall UI rework to make information intuitive and easy to understand – will compensate the lack of these dropped features. And who knows – maybe in some future version, we will find a way how to introduce them again in the new framework.

While the test site where you can play with the new workshop module is up and running, I am working on migration procedures to get current 1.9 workshop data into the new framework. I am following similar approach as Petr used for the migration of the Resource module. I personally call it Scavenger design pattern.

The basic idea of the migration is that all 1.9 tables are renamed using _old suffix and new ones are created as if we were starting from scratch. Then the upgrade script goes through all 1.9 records, transforms data into new formats and inserts them into new tables (marking old records as processed). Once finished, the new workshop core tables are fed up with the old data. Later during the installation, new workshop subplugins get created (allocators, grading strategies and grading evaluators). They find the old dead workshop tables and start picking data from them. At the end of the day, the new tables are populated with the old data.

Such a procedure requires that 1.9 workshop instances are at least in sort-of-well-defined-coma state. Therefore I started to fix the most critical functional and security bugs in workshop 1.9. So far the most important one may be that the workshop in Moodle 1.9.7 will push grades into gradebook as expected during the Synchronise legacy grades procedure. Big thanks to all patient users who help me with testing both new features and fixes.

It was one of that discussions when people prefer leaving the chat room before they say something they would regret later. The Pandora’s box was opened by an innocent question on the recommended procedure to customize a contributed Moodle plugin. The discussion turned into a hot topic on what the current source code management system (CVS) gives to us and what we and our contributors could gain (and lose) from switching the Moodle CONTRIB repository to git.

Imagine you have clients who pay you for implementing a solution based on a contributed Moodle plugin. The quality of the Moodle contributed code varies and some sort of the code review and clean-up is strongly recommended before using it at production server. The contributed code may contain critical security holes and such plugin may easily become an open gate to your server with a big bright sign “Welcome Attackers!” So you have reviewed the code, fixed several obvious bugs and improved the features a bit as requested by your client. And – now what?

Of course you want to deploy your changes to the client’s server – as soon as it is tested  (or at least it does not produce any PHP warnings ) And you, as a follower of the open source spirit, want to contribute your changes back to upstream. So you send your patches to the original component’s maintainer to be included in CVS. Sounds good. In a perfect world it would work with no problems. But the humans are not dead yet and things tend to get complicated. Also, some objections against the real benefits of this procedure were raised. Let me list some issues that may influence this contributing back to upstream.

1. The maintainer does not respond. Such things happen. You can ask the CVS administrator to give you write access and become a maintainer. Do not do this if you know/feel you would not have time to do it responsibly. Remember the lesson #5 from the Eric Raymond’s famous essay: “When you lose interest in a program, your last duty to it is to hand it off to a competent successor.” It’s better if it’s known there is no maintainer than if people think they can rely on you. On the other hand, do it if a silent voice inside your mind whispers that it’s the job you always wanted to do. It’s both nice and good thing to spend your life on.
2. The maintainer does not agree with your improvements. Then it’s time to discuss things. Summarize the issue and open the topic in Moodle forums. Listen to the maintainer’s arguments. Maybe he/she has some further plans with the module? Who else can profit from your changes? Try to not to fork the product unless there is a chance to reach a consensus.
3. The rubbish can get into the code after you reviewed it. Yes, it can. You must remember the state of the code when it was reviewed. Source code management software calls such signs as “tags”. You shall tag a certain point of the source code history and review exactly that point. Then you know what version is sort of safe to be deployed. Also, you can focus on the changes that came after this point later when you review the code again.
4. The maintainer does not understand your fixes. Some say that simply sending the patches to the maintainers is useless as they do not spend any effort on learning from it. I personally do not agree with this. However, I can see the importance of educating the contributors. If the contributed code contains a security bug which is exploited, who is blamed? The administrators? No,  they have always been against this community thingie. The original author? No, nobody knows him/her. Moodle itself? Sure – it’s all Moodle’s fault! And that’s not fair.

All these have nothing to do with the SCM software itself. It is about responsibility, communication, constructive peer-reviews and developer etiquette. However, I think that the features and the characteristics of the SCM may significantly help with sorting out the issues discussed above or with not running into them at all.

Switching Moodle CONTRIB area into a cluster of separate git repositories emerged as an idea during the discussion. The nature of git – a distributed versioning system – seems to perfectly fit into the nature of our contrib, the hive of really interesting features and ideas. Contributors may come with their own trees of source code and the comparison of such two trees is easy. If the maintainer stops responding to the community questions and proposals, his/her tree becomes quickly obsolete and the community may start to prefer some other tree (if such is available). The pull-based model of the the maintainer-contributor interaction requires the maintainer pays attention to the patches to be included and hopefully learns from them. On the other hand, the contributors have to keep their trees up-to-date with the recent versions of the upstream code if they wish it is included in the maintainer’s tree. The point is, with git, everybody can easily become a maintainer of his/her own clone of the product. They have to gain the respect of the community members by keeping the high quality of their version. And remember – respect is everything.

I do not think we should switch to git-only at the moment. Let us give git a chance and see what  Moodle contributors think. To help them with their first steps, I summarized my personal experiences with using git in Moodle development into a sort of tutorial at http://docs.moodle.org. You are welcome to review it and/or put comments with your own solutions.

commit eb123debc6279c0e7fe92d7cb8979f4db651a46c
Author: skodak <skodak>
Date:   Thu Oct 8 22:34:34 2009 +0000

MDL-19474 One year and a half ago we started designing
and implementing our new database abstraction layer
which included brand new moodle native drivers.
Today we are finally removing old legacy drivers
that were still using adodb internally. From now on
the adodb will be used only in some authentication
and enrolment plugins, we are probably going to
drop adodb there too later in 2.x.
Thanks everybody who helped test and improve our
new drivers, even bigger thanks to all the great
developers who created ADOdb!