David Mudrak's blog

Latest stable version 1.9.8+

There are 4 commits into the stable branch from the last development week. Tim Hunt fixed a wrong capability checking in Quiz module (MDL-22410) and several regressions in clean_param(), introduced when we were converting ereg to preg, because ereg functions are deprecated in PHP 5.3 (MDL-19418).
Martin Dougiamas committed a bunch of patches that kills the scripts expected to be included only, if they are called directly via URL (MDL-22388). During the initial setup of every script life time, Moodle core defines constant MOODLE_INTERNAL which shall be used to make sure that the script is included and not called directly (constants can’t be defined via URL even with bloody register_globals on). This trick helps to prevent libraries, forms definitions and other scripts from potential security holes. My mom told me not to talk to strangers so I personally prefer silent one-line check

defined('MOODLE_INTERNAL') || die();

instead of explaining the poor hacker that the direct access to the script is forbidden (Mahara uses this shorter form and I just like it).
Gordon Bateson found a way how to deal with an XHTML issue in Hotpot by using IFRAME instead of OBJECT for Internet Explorer, as HTML forms are not able to escape OBJECT in this browser.

Future version Moodle 2.0 Preview 2

There are 141 commits into the future release branch from the last week. It is now tagged as Preview 2. Since Preview 1, there are many improvements and bug fixes. Many thanks to all who help us with testing.

Quotes of the week

“Ha, some song just came on my radio station by a band called CSS. The song is called CSS Suxxx. Just when I was swearing at css”
– Martin Dougiamas experiences reaffirmation from the world

Watch your back!

Backup and restore support used to be my not very favourite part of Moodle modules coding. I was kind of scared (and bored) by all those direct fwrite() calls to generate XML representation of the module data. With the new backup/restore framework, written by Eloy Lafuente, this is not true any more! Eloy has prepared really well designed system that deals with a lot of tasks automatically, yet in a flexible way. Following his tutorial on backup support for Moodle 2.0 activity modules, I was able to quickly hack a working prototype of Workshop backups.
What I like the most so far is the way how module just defines a description of its database structure and lets the core to actually fetch the data a convert them into XML representation. Given the necessary amount of information, new backup system is able to “automagically” handle all relations, references to other tables and including embedded files (only those really needed by the module, of course!).
There are still parts to finish both in Moodle core and in modules – most notably the ability to backup subplugins information and to restore 1.9 backups. But I can sleep well again as backup and restore is not nightmare any more for module developers.

Post scriptum

After more than eight years, one of the characteristic symbols of Moodle, the default user icon (yes, that very well known smiley cake), was replaced by a bland avatar for the standard theme in Moodle 2.0. That is ironic that the smiley actually replaced the original “shadowhead with something a bit more positive”. The story of the “f1.jpg” image and how it resisted various attempts to replace it is sometimes quite interesting. Anyway, I will miss it… So long!

Latest stable version 1.9.8+

There are 4 commits into the stable branch from the last development week. Dan Marsden fixed a bug in SCORM backup/restore (MDL-22301) and yet another one which was caused by not using absolute file paths when including a PHP file (MDL-22340). In Moodle, the best way to include other files (typically libraries or forms definitions) is to include the main config.php via dirname(__FILE__) and then include all other files needed via their full path, using $CFG->dirroot or $CFG->libdir.
Eloy Lafuente updated the timezones database shipped with Moodle to version tzdata2010i (MDL-18576). This database contains information about the world’s time zones and daylight saving time and helps our in-built calendar to calculate correct dates and times for users from all around the world. Moodle uses so called Olson database, a public-domain collaborative compilation of timezones data, currently maintained by Paul Eggert.
Pierre Pichet committed a patch fixing buggy behaviour of calculated question type on duplicate (save as new question).

Future version Moodle 2.0 Preview 1

There are 158 commits into the future release branch from the last week. On Thursday, Martin Dougiamas announced the release of Moodle 2.0 Preview 1 version, the first milestone on about 10 weeks long track to reach final Moodle 2.0. See the preview release notes for more information.

Quotes of the week

“It is a good idea, so I put it in the tracker. The point of the tracker is that it remembers things, so we can forget about them.”
– Tim Hunt on tracker.moodle.org

“I’m really looking forward to 2.1 and working on modules again.”
– Martin Dougiamas knows that many of improvements in 2.0 lie in the dark deep of Moodle architecture internals and are never to be actually seen by users

Installing Moodle from the command line

I love Linux for many reasons and not only because it is free as in being free after drinking beer. Since the first days with Linux, I’ve been impressed by the power of command line utilities, following the Unix philosophy of doing just one thing, doing it well and cooperating with other small programs. From version 2.0, Moodle supports installation and upgrades from the command line. Of course I was happy I could spend some time on improving these routines a bit.
CLI programs for common administration tasks (installation, upgrade, switching the maintenance mode and resetting a forgotten user password are supported at the moment) can be found in admin/cli/ directory of standard Moodle 2.0 source code. You should run them as the same user as is used for the webserver process (apache or wwwrun are used usually).
This is an example of how I installed a fresh Moodle 2.0 at my notebook today:

$ php admin/cli/install.php --lang=en
--wwwroot=http://localhost/moodle20
--dataroot=/var/www/moodledata/moodle20
--dbname=moodle20 --dbuser=apache
--dbpass=xxxxxx --dbsocket

In this example, the script runs in an interactive mode, asking for all needed data and using the provided ones as default. There is also a possibility to execute the script in non-interactive mode, providing all required information as CLI parameters. Such an option may be interesting for hosting providers as it allows to automate the process of creating a new Moodle site.

Post scriptum

http://www.projectcartoon.com/cartoon/1

Latest stable version 1.9.8+

There are 10 commits into the stable branch from the last week. Half of them, committed by Dan Marsden, deal with various SCORM issues (MDL-22168, MDL-22046, MDL-21568, MDL-18202, MDL-21444). Tim Hunt fixed three Quiz issues (MDL-22241, MDL-20586, MDL-22257). Thanks to Gordon Bateson, Hotpot module now uses HTML OBJECT tag instead of IFRAME, due to XHTML 1.0 strict compatibility (MDL-17702).

Unstable development version 2.0dev

Total of 294 commits found their way into the future version branch last week. Several various subsystems and major patches were included – for example the new My home page, Moodle Hubs support and many others. Petr Å koda’s recent patch, that reimplements the whole enrolment subsystem and changes some of the key concepts of roles and their assignments, is still being reviewed in a separate branch. Martin Dougiamas is going to announce the release of Moodle 2.0 – Preview 1 version, just working on the release notes yet.

Quotes of the week

“Users will remember for almost forever a bad/broken release; but will quickly forget that a stable/fine/working release was a year late.”
– Greg Mushial summarizes his experience from 40+ years involved in software development

“Beta version software is likely to be useful for internal demonstrations and previews to select customers. Some developers refer to this stage as a preview, a prototype, a technical preview (TP) or as an early access.”
– Wikipedia, Software release life cycle

Help tooltips in Moodle 2.0

Among many other (and definitely more significant and more important) improvements in Moodle 2.0, there is a new way how the help tooltips are handled. From the users’ point of view, help is still available via those question mark icons. Instead of opening a pop-up window in the browser, Moodle 2.0 uses YUI3 overlay to display a “bubble” with the help text. The behaviour of the tooltip had been a subject of a discussion (MDL-22067) and at the end, it was decided to follow the functionality found in Mahara or Github.

From the developer’s point of view, help texts are not stored in separate files any more. In the dark past, help files used to play a role of inline help, documentation, manuals or even tutorials on using Moodle. Nowadays, with Moodle Documentation portal being actively growing thanks the community contributions, we are trying to revert helps to what they are supposed to be – useful hints on various user interface elements, their meaning and usage. Therefore, the help files are being migrated into proper Moodle strings in 2.0.

For every help, two strings must be defined. One for the label of the thing the help is describing. In the forms, this is the label of the input element in almost all cases. The second string is the help text itself. The identifier of the second string must be the same as the identifier of the first one, just with “_help” suffix, for example:

$string['subject'] = 'Subject';
$string['subject_help'] = 'Message subject will be used as a title at the message view page.';

Let us say we want to add such help in a Moodle form, for ‘msgsubject’ input field. You would probably have a code like:

$mform->addElement('text', 'msgsubject', get_string('subject', 'mymodule'));
$mform->addHelpButton('msgsubject', 'subject', 'mymodule');

That would use the string ‘subject’ defined in your ‘mymodule’ language file as a title of the help and the string ‘subject_help’ defined in the same file as the help text. To get the same result in Moodle 1.9, you would have the help defined in a HTML file, let us say help/messagesubject.html a the code in the form would look like:

$mform->setHelpButton('msgsubject', array('messagesubject', get_string('subject', 'mymodule'), 'mymodule');

To display a standalone help icon anywhere at your page, use help_icon() method of the core renderer instance:

echo $OUTPUT->help_icon('subject', 'mymodule');

Help strings can use Markdown syntax for basic HTML formatting. We use that just for producing bullet list from the lines starting with asterisk.

To give the translators something to start with instead of rewriting all help strings from scratch, we are copying the contents of the translated help files into new help strings wherever it is possible.

Post scriptum

I can’t think of anything to write here today

Latest stable version 1.9.8+

There were 12 patches committed into to the stable branch during the last development week. Helen Foster once again reworded the help describing the behaviour of a recently added  gradebook feature as Elena Ivanova suggested (MDL-21218). Pasi Häkkinen discovered a malicious incorrect usage of HTML entity that led to Flash version detection being done by AJAX call at every page load instead of once per session and Petr Å koda promptly committed Pasi’s patch (MDL-22154). Dan Marsden backported Vlas Voloshin’s patch fixing the display format of exhibit time periods in SCORM 2004 packages (MDL-18835) and fixed yet another issue with First Access and Last Access dates in SCOs (MDL-16184, MDL-21423). Sam Marshall fixed a bug producing empty links when there was no group picture. Such empty links created an invisible tabstop which is confusing for screenreader and other keyboard users (MDL-22174). Tim Hunt backported some parts of unit test failure display improvements (MDL-22174). Eloy Lafuente fixed a bug in backup/restore machinery so the manually added links to the course main page are transformed into new URL, reported and tested by Larry Elchuck (MDL-22176). Petr Å koda committed a patch provided by Gordon Bridge. That fixed a bug in the Assignment module which overwrote the grade item identifier in the Gradebook with a number (column “id” of the Assignment module instance in course_modules) instead of the value defined for the activity in modedit form (column “idnumber” in that table). This critical bug had been reported before but could not be reproduced until Gordon described it in details and found the nasty line (MDL-22181).

Unstable development version 2.0dev

There are 141 commits into the main development branch last week. To highlight one, Eloy Lafuente merged first parts of the new backup/restore backend he was working on in a separate git repository recently. Sam Hemelryk is going to work on the backup/restore user interface. As many other parts of Moodle 2.0, backup/restore subsystem will be fully implemented during the beta period (which starts in six days!). The last huge  change that is going to land before the feature freeze next week is Petr Å koda’s new baby – the reimplementation of enrolment plugins. After that, Moodle 2.0 APIs and database structures should be considered more or less stabilised and the team will focus on fixing the regressions only.

Quotes of the week

“That github interface is very nice”
– Martin Dougiamas after reviewing a patchset before it was committed into the main CVS repository

“Would be nice if core developers actually tried to run the code before they commit it into the repository.”
– David Mudrak gives a hint on how to prevent the most obvious bugs

“Whoa, that was quick!”
– Pasi Häkkinen found his bug fixed in 57 minutes after it had been reported

I am, YUI

All modern web applications use advanced JavaScript based techniques to increase the usability of the user interface. Things like inline editing, re-populating the parts of the page via AJAX calls or saving the forms without reloading the whole page are quite common for users today. To ease the integration of JavaScript and skip fiddling about with its different implementation in various browsers and their versions, web projects use some JavaScript frameworks like jQuery, Dojo Toolkit or YUI. The latter one, Yahoo! User Interface library, has been chosen as the official JavaScript framework for Moodle since long time ago. Moodle 2.0 ships with the most recent version YUI 3.1.0 while still providing a way how to use some widgets from earlier 2.8.0 version.

Moodle core does a lot of work for you behind the scene with including YUI libraries and their dependencies in correct location of HTML produced by your code. The most straightforward way to include JavaScript on your page is to define the code in the file module.js in your plugin directory. To avoid collisions of global variables and functions, all your code is supposed to be wrapped in a namespace. Since 2.0, we put all Moodle JavaScript code into a global M object and plugins are expected to use their own M.plugintype_pluginname namespace. For example if you write an activity module called foobar, all your JavaScript functions should be defined in M.mod_foobar namespace.

To include your code at the page and to initialize it from PHP, use $PAGE->requires->js_init_call(), providing the name of the initial JavaScript function as the first parameter. This initialization function must accept YUI instance object as the first parameter which we call Y, as is common in YUI 3. Once you have YUI instance, you can do whatever magic this library offers to you, including the possibility to load additional YUI modules with Y.use(). The ‘Hello world’ example of using YUI in Moodle 2.0 could look like this:

Let us say you are working on FooBar activity module. In your PHP script, for example /mod/foobar/view.php, add a line

$PAGE->requires->js_init_call('M.mod_foobar.init');

Then create the file /mod/foobar/module.js with the following contents

/**
 * @namespace
 */
M.mod_foobar = M.mod_foobar || {};

/**
 * This function is initialized from PHP
 *
 * @param {Object} Y YUI instance
 */
M.mod_foobar.init = function(Y) {
    alert('Hello world');
}

Or, instead of just displaying the alert, to modify the current HTML code (for example to add some elements via JavaScript or to remove some non-JS support code) just use

M.mod_foobar.init = function(Y) {
    Y.one('#mycustomholder').set('innerHTML', 'Hello world');
}

See YUI 3 documentation for more great examples. You may also find this blog post useful (I did, thanks Eloy for the link).

Post scriptum

I love xkcd.

Latest stable version 1.9.8+

There are 17 commits into the stable branch from the last week. Tim Hunt fixed a silly bug that led to bad performance problems in the grader report (MDL-22098). Andrew Davis fixed a regression in the gradebook overview report so it displays hidden items correctly now (MDL-21218). Iñaki Arenaza committed new upstream version of phpCAS library (MDL-20029).

Unstable development version 2.0dev

There are 70 commits into the main development branch from the last week. Martin Dougiamas has set-up Moodle 2.0-beta release day in HQ planning calendar to May 3rd 2010 and the team is focussing on bringing Moodle 2.0 beta down onto the runway. Hopefully the wind conditions are good for clear landing.

Quotes of the week

“What happened to ‘We will release a solid product when it is done.’ ???”
– Tim Hunt thinks we should not promise any release dates

“Just 10 secs ago, by mistake, I emptied the block_instances table completely. It’s amazing how better Moodle 2.0 looks without blocks! A M A Z I N G !”
– Eloy Lafuente improves the look and feel of future Moodle version

“Most of the things I put in Moodle to push my view of courses as communities have been ground down over the years.”
– Martin Dougiamas stands alone against the world

The role who must not be named

While working on help files clean-up, Helen Foster re-opened an internal discussion on the “official” Moodle terminology. How should the role archetypes teacher and student be called in help and documentation? It is clear that these two do not fit everybody — for example when Moodle is used for company training. Martin hates the model these two terms promote and would prefer if Moodle highlighted course scenarios based on facilitation and moderation, instead of teaching. But it is difficult to come up with a word that would work for university and corporate as well as schools. There is an issue with using participant for students, too. As Helen noted, teachers are also participants, learning from their students so the term should be used for all users that have some role assigned in the course.
This question is not new kid on the block. We talked long and hard about this years ago and ended up having different people using different words in different places, says Martin. The point is that neither moderator nor participant are really used in common forums-speak. Folks tend to use teacher for the person who knows and student is the person that does not know, although the underlying principle of Moodle doesn’t work that way, Tomaz points.
Years have jaded me, sighs Martin. As our benevolent dictator for life, he makes an executive decision to use participants for all people with a role, teachers for people with some sort of a facilitation/editing role, students for people primarily there to learn, and users for people across the site. Dammit, everyone knows what they mean, he closes.
In the new translation interface I am currently finishing (to be available together with Moodle 2.0-beta), there is a way how to quickly filter all strings containing a given phrase. I can imagine Moodle partners offering a customized language packs for various clients (schools, universities, business) with all these keywords searched and replaced.

Post scriptum

make install && not war

Latest stable version 1.9.8+

There are 7 commits into the stable branch from the last week. Tim Hunt committed David Binney’s patch fixing a question bank SQL query at Oracle (MDL-21828). Iñaki Arenaza fixed an old issue when he committed his patches that significantly increase the quality of images produced by TeX filter. If the binaries of latex, dvips and convert are available at the system, Moodle can generate nice PNG formulas. Otherwise, “ugly” GIFs are created by mimetex (MDL-10197). Another Iñaki’s committed patch allows to use HTML editor when editing login page instructions (MDL-19053) and yet another one fixes small typo in CAS, DB and LDAP authentication plugins (MDL-18689).

Unstable development version 2.0dev

There are 160 commits at the main development branch from the last week, many of them done by Petr Å koda during the weekend strings clean-up. Patrick Malley added a new test theme called Boxxie based on his Greenie theme for Moodle 1.9. On Wed Apr 7 00:31:20 2010 UTC, the last ever run of Eloy’s installer_builder generated and committed strings for Moodle installer and was turned off. In the near feature, it will be replaced with a new script taking the data from our new strings repository called AMOS. So long installer_builder, and thanks for all the strings!

Quotes of the week

“I like git because it might actually force us to fix our sloppy/missing review processes before things hit CVS”
– Petr Å koda guards the code against eager commits – get alarmed!

“NOBUG: Bombing BOM: BOM bombed”
– Eloy Lafuente and his bombastic commit message

Milliseconds count

At the moment, I have 3,740,067  records (yes, 3.74 millions of records) in the new database repository of Moodle strings. The repository (basically one huge SQL table) contains one record per every modification of every single string that happened since Moodle 1.6 in any language. It takes approximately two hours to re-populate the table from our git mirrors of Moodle core and Moodle langs CVS repositories. There are some critical queries that are used quite frequently – for example to take a snapshot of the most recent string values for the given component, language and branch. That is, for example, “what is currently saved in lang/en/moodle.php at the main development branch?” (or what was there at the given timestamp). Writing this query was a big lesson for me. Not only it was actually for the first time I ever used an SQL subselect. Having so many records, the query optimising became very important as I refused to wait dozens of seconds to actually see some results (and I can imagine translators  would not be happy with such delays, too).

Basically, all Moodle strings and their history is now stored in one database table containing fields branch (integer code for Moodle branch, like 1900 representing MOODLE_19_STABLE), lang (language code like ‘es’), component (the string domain – or the file where the string is defined, like ‘forum’), stringid (the key is $string array), text (the string itself) and finally timestamp (since when the change described by this record  is valid). There are other fields defined in the table – to hold the author of the change, commit message, deletion flag and others. There is composited index created for fields  (component, language, branch).

This is the query I was using during the optimisation and performance comparison. It returns the most recent version from several branches, several languages and several components – which is exactly what I need for the new translation interface.

SELECT r.id, r.stringid, r.text, r.timemodified, r.deleted
  FROM  mdl_amos_repository r
  JOIN (SELECT branch,lang,component,stringid,
               MAX(timemodified) AS timemodified
          FROM mdl_amos_repository
         WHERE branch IN (2000,1900)
               AND lang IN ('en','cs')
               AND component IN ('moodle','workshop')
             GROUP BY  branch,lang,component,stringid) j
    ON (r.branch = j.branch
        AND r.lang =  j.lang
        AND r.component = j.component
        AND r.stringid = j.stringid
        AND  r.timemodified = j.timemodified)
  WHERE r.branch IN (2000,1900)
        AND r.lang IN ('en','cs')
        AND r.component IN ('moodle','workshop')
  ORDER BY  r.component, r.stringid, r.lang, r.branch;

The basic idea of the query is to get know (in subselect) which one is the most recent version of every string. Then (via the table self join) get some additional information about such string. I was experimenting with various variants of the query -like putting those WHERE conditions into the inner subselect or the outer main select. None of them was good enough until Petr Å koda gave me a world-saving hint: put the same WHERE conditions into both inner subselect and outer main select! The inner conditions help to filter the rows that are going to be aggregated by GROUP BY. The outer conditions help to join results quickly.

Using this, I managed to get to times like 1500ms at my laptop and 200ms at the testing server, both using PostgreSQL. First attempts to run the same query on MySQL led to significantly worse figures. I am going to do more tests to be sure that both engines have similar resources/conditions available. That massive usage of IN() is supposed to be a challenge for MySQL…

Post scriptum

I fell in love running Moodle 2.0 upgrading script via command line interface – see admin/cli/*.php scripts

Latest stable version 1.9.8+

There are 6 commits into MOODLE_19_STABLE from the last development week (from Tuesday to Monday). Tim Hunt fixed a gradebook bug occurring during repeated imports of grades that are normally pushed into the gradebook from activity modules or are calculated and can be overridden (MDL-21987). Petr Å koda fixed incorrect usage of E_ALL constants which lead to deprecation warnings in PHP 5.3 and therefore break output when generating images. Developers must use our DEBUG_xxx constants only (MDL-22006). I committed two patches that could not land at the stable branch during our recent code freeze. The first one may be useful for developers who are debugging some emailing feature in Moodle. It allows to divert all outgoing emails generated by Moodle to a single recipient (MDL-18903). The second one fixes a bug in multilang support for custom profile field labels (MDL-21845). Helen Foster (accessing CVS via command line as all real hackers do!) improved wording of a recently introduced gradebook option and added a missing help file for it (MDL-21218).

Unstable development version 2.0dev

There are 51 commits into the main development branch.

Quotes of the week

“It’s easier to say sorry than to ask permission”
– Penny Leach on committing procedure

Pruning the trees of code

The spring comes to the central Europe. For gardeners, it is time to clean up the orchards and prune the apple trees (which may turn into kind of physical exercise both with and without the chain saw). For hackers, this season is perfect for cleaning up the source codes and getting rid of zombies hiding here and there. We are currently working on localization subsystem in Moodle and there was a good opportunity to remove some not-so-nice hacks and rules that were present for a long time.

If you have ever developed a plugin for Moodle 1.x, you probably know Moodle defines strings as items in an associative array, for example

$string['greeting'] = 'Hello world';

For some known reasons, there are several rules that developers must follow when adding new strings into the English language pack (and translators when editing the packages manually) at Moodle 1.x. Some of them are not much intuitive, like double quotes must be always escaped as in

$string['author'] = 'Petr \"skodak\" Skoda';

When you want to display a percentage sign, you must remember to actually type double %% to get just one:

$string['maxgrade'] = 'Max grade exceeds 100%%';

Replacing placeholders in Moodle with their values has been always done by eval(). So everybody had to properly quote all other occurrences of the dollar sign in order not to get it replaced by the real value of producing an error (or white screen of PHP death):

$string['greeting'] = 'Hello $a->firstname, how are you?';
$string['configerror'] = 'Your \$CFG->wwwroot is not correct';

As you can imagine, it was pretty easy to make a mistake. And various syntax-related mistakes were really done by developers and by translators as well.

Luckily, the format of $string[] items was significantly simplified for Moodle 2.0. The basic principle now is what you type is what you get. That means that developers no longer need to think too much about the $string syntax. In Moodle 2.x, all the following strings will be displayed as expected:

$string['author'] = 'Petr "skodak" Skoda';
$string['maxgrade'] = 'Max grade exceeds 100%';
$string['configerror'] = 'Your $CFG->wwwroot is not correct';

If the string contains a placeholder to be replaced with a value, it must be enclosed in curly brackets:

$string['foo'] = 'This {$a} will be replaced with a value';
$string['bar'] = 'This $a will be printed as a dollar sign followed by the a letter';

Good news is we no more use eval() on the whole string so it is much easier to detect and fix errors. Even better news is that all existing 1.x strings will be automatically converted into the new syntax format. The current plan is to switch to the new strings format during the next week, after we debug the new implementation of get_string() a bit and migrate all core strings into the new syntax.

Latest stable version is 1.9.8

On 25th March 2010, Moodle 1.9.8 was released. See Release notes for details. As usually, admins are advised to upgrade production servers.

There are 6 commits into MOODLE_19_STABLE from the last week done by humans. As always in this blog, commits by Moodle Robot are excluded from these statistics. Moodle Robot tirelessly increases the build number in version.php every day (yes, paradoxically that line with “Human-friendly version name” comment).  By the way – thanks to this job, Moodle Robot is the 10th most active contributor into Moodle code and even has reached Kudo Rank 8 at ohloh.net

In the last commit before Moodle 1.9.8 was released, Petr Skoda fixed proper handling of HTTPS wwwroot in Flash version detection (MDL-21910).

Unstable development version 2.0dev

There were 89 commits into the main development branch last week (from Tuesday 00:00 to Monday 23:59 in my git clone). Among other core developers, Sam Hemelryk committed his recent work on new themes layout (MDL-21862). In Moodle 2.0, there is kind of system theme called “base” which defines just very basic CSS. All other themes (including the new “standard”) are built upon this base theme. Web designers should follow http://docs.moodle.org/en/Development:Theme_changes_in_2.0 when preparing the themes for the incoming major release. I committed a series of patches that move language files into their plugin space (MDL-21694). In Moodle 2.0, English strings for activity modules and other plugin types are stored in the plugin space so there is no difference between core plugins and contributed plugins.

Quotes of the week

“2.0 beta is so much a priority right now I’ve stopped eating”
– Martin Dougiamas

“I think reinventing the wheel is usually a good idea personally, but that’s just me. ”
– Sam Marshall

“In PHP, every dog and his master wants to write their own framework.”
– Penny Leach

Eins, Zwei, Polizei

If you ask a mother how many children she had, you do not expect her actually counting them. She just knows. Similarly, if you ask PHP array how many items it contains using count() function, you expect it will be very cheap call. Knowing the number of contained items is so natural thing that array should return the value almost immediately with almost no cost. Well, apparently not in PHP. The following trivial simple script demonstrates the performance difference between using count() function compared with keeping the number of items in a separate counter:

define('MAXITEMS', 10000);

function using_array_count() {
    $a = array();
    while (count($a) < MAXITEMS) {
        $a[] = 1;
    }
}

function using_own_counter() {
    $a = array();
    $i = 0;
    while ($i < MAXITEMS) {
        $a[] = 1;
        $i++;
    }
}

$start = microtime(true);
using_array_count();
$t1 = microtime(true) - $start;

$start = microtime(true);
using_own_counter();
$t2 = microtime(true) - $start;

printf("%f %f %f \n", $t1, $t2, $t1/$t2);

At my notebook, the variant using count() is about five times slower than the one using the own counter. Not big deal, right? But Jens Eremie from Humboldt Universität in Berlin realized that in more complicated scenario (two arrays involved, resetting the array, keeping the number of items under a given limit etc.) the difference may be significantly higher – even hundreds times. Such a scenario that Jens was dealing with is Moodle cache_context() function. In MDL-19702 you can find a test script showing quite interesting figures. At large Moodle installations with many courses and categories (therefore many context) using count() in a loop makes caching a pain. Therefore, places like MyMoodle page run into a unacceptable poor performance and can easily reach PHP max_execution limit. After some not-so-complicated changes, things just run.

During a nice discussion at MoodleMoot in Berlin, Jens explained me several caching improvements he proposes. Getting rid of using count() is just one part of them. Jens realized that it does not make sense to shift the internal cache array (that is to remove the first cached item) after reaching the limit of cached items. As the contexts come into the cache in some order and they may be requested in the same order, the cache may actually never hit if the number of contexts is higher than cache limit. It is better to randomly pick instead of using the first item always. Then the cache works better in first-in first-out scenarios. Jens also proposes to follow the common wisdom (used for example in microprocessor caches) to prune items down to 80% of cache capacity at once instead of freeing items one by one when needed. And last, but not least, one the cached context is hit, it will be probably used more than once in Moodle code. Therefore the items should stay as long as possible in the cache once they have been hit so the random picking should prefer items with not hit so far.

Disclaimer: if I am not right or even if I am wrong, there is a bug in my understanding of the issue. I write here what I remember from the discussion with Jens and chances are I misinterpreted something. Blame me, not him. Praise him, not me.

Post scriptum

PostgreSQL’s EXPLAIN ANALYSE rocks.

Latest stable version 1.9.7+

There are 15 commits into the stable branch from the last development week (from Tuesday 16/03/2010 until Monday 22/03/2010 including). Helen Foster fixed a bunch of help files to ensure XHTML strict compliance (thanks to David Horat) and improved the wording of a glossary help file (thanks to Bente Olsen). On Friday, Martin Dougiamas announced code freeze before 1.9.8 release and since then, only really important patches found their way into the stable CVS branch. Petr Škoda committed  several security related issues. Eloy Lafuente fixed upgrade failures on MS SQL and Oracle and fixed backup restore routines to properly check for permission before creating or overriding roles and their capabilities (credit goes to Daniel Neis).

Unstable development version 2.0dev

During the weekend, there was the first ever Moodle 2.0 sprint to get beta released soon. There are 114 commits into the main development branch from the last week.

Quotes of the week

“A major Moodle release and a wedding are not good things to try and do at the same time :\”
– Andrew Davis

“I think the problem was the line got truncated when I patched I need a bigger monitor”
– Penny Leach

“Comment is a reserved word for Oracle. I reserve my comments about it ”
– Eloy Lafuente

Building the Wall in Berlin

At the MoodleMoot in Berlin, me and Petr Skoda have just finished a pre-conference Moodle Development workshop. During the two-day session, our nine attendees could hear a lot about the Moodle development procedures, new APIs, standard coding patterns and ways to customize Moodle to fit the institution needs. We were demonstrating these things on an actual coding. After an overview of Moodle code base structure and plugin types available, our participants were writing a brand new activity module for Moodle 2.0 called Course wall. Such an activity brings a feature well known from Mahara, Facebook and other social networking environments.

We started with an analysis of the requested functionality. The wall module should allow course participant to leave short messages on a virtual wall inside the course. To make it simpler for the workshop purposes, wall posts can’t be edited, deleted or approved (such features would be added later). At the wall main page, a list of the recent wall posts is displayed together with a form to submit a new post. We demonstrated using UI mockup tool (like Balsamiq we have integrated into our tracker) to design the user interface of the module.

Once the functionality and the UI was agreed, we used our NEWMODULE template as a scaffold for our Wall. After the template was extracted in a proper location and initial settings was done (mass search and replace to set the correct module name), we used XMLDB do define the tables used by the module.  Of course, we had to describe the basic structure of Moodle database tables and relations between the Moodle core tables (user, modules, course and course_modules) and the module tables.

The new way of producing HTML content by plugin renderers was explained. Security aspects of the code were intensively discussed at both theoretical level (general web security principles) and at the application level (how to write a secure code for Moodle). Sanitizing user input, checking sesskey and formatting the output were introduced as the holy trinity to respect during the development. Finally we added some capability checking and tried to wrote a proper upgrade procedure for our module after adding a new field into the database.

The resulting output code of this new plugin is not available for public yet. I am thinking about extending it a bit and using it for some sort of tutorial course on Moodle 2.0 development, in a similar fashion to our current “Writing a new block” course.

Post scriptum

They should do something with the beds here in Dorint hotel in Berlin. A blocked nerve at my back constantly sends a painful  signals into my brain. If I sleep on the floor today, it is not because of the amount of mojito drunk.

Latest stable version 1.9.7+

There were 7 commits into the stable branch. Helen Foster fixed the confusing wording of Course creator role description as suggested by Ray Lawrence (MDL-21016). Petr Å koda fixed buggy displaying decimals in s() function, spotted by Tim Hunt (MDL-21789). Dan Poltawski has fixed a bug that was causing block disappear after moving it. The issue is still not completed, an upgrade step will be necessary to fix all eventual broken block instances. Andrew Nicols provided SQL statement to detect such problems (MDL-21805). Dan Marsden committed two fixes in SCORM/AICC module, credit goes to Matteo Scaramuccia. Gordon Bateson fixed a malicious regexp issue in Hotpot module (MDL-21817).

Unstable development version 2.0dev

There were 32 commits into the main development branch last week. Everybody is working hard on this future release. Among other contributors, Helen Foster continues on her crusade to beat obsolete help files and Dongsheng Cai committed a set of various patches in repositories, JavaScript and other places.

Moodle 2.0 beta release has been re-scheduled to April 2010, production release still planned for July 2010.

Quotes of the week

“I would rather wait another whole 12 months for Moodle 2.0 than have gone for [another LMS] and regret it…”
– Mary Cooch

“Moodle installs should phone home and compare version numbers and whine bitterly on notifications page ‘I AM OUT OF DATE AND SAD, PLEASE UPDATE ME ‘, getting increasingly sad and red and large, with tears, and animated gifs … ok, that’s going too far”
– Penny Leach

You shall not pass!^*

There are several Moodle core functions that every developer should know, even when they were woken up in the middle of the night after a long party (actually that does not make sense as “middle of the night” is not compatible with “after a long party”, anyway…). These include require_login() and require_course_login(), both defined in lib/moodlelib.php. They are part of the Moodle security scheme and it is important to understand what they do and how they differ.

If you are not sure, use require_login(). This function checks that the current user is logged in. You should call it very early in your scripts, typically immediately after you get course and course module objects. Once you load these objects, you should pass them to require_login() so that it not only checks the current user is logged in but also makes sure they are allowed to be in a particular course and view a particular course module instance. It checks against hidden activity restrictions and proceeds groupmembersonly access control and conditional activity access control. If the user is a guest then the function treats them according to the course policy about guests and/or asks them whether they want to enrol into the course (yes, that “do you want to enrol into this course” screen).

Regardless the function name, require_login() sets up and checks a lot more things behind the scene. If available, it tells the global $PAGE object what course and course module it displays and sets the page layout. It checks whether the user should change the password, that the user account is properly set up (do you remember those redirects to your user profile to fill-in missing fields?), that the user has agreed to a site policy (if there is one) or that the site is not in maintenance mode.

Uff, quite a lot of stuff in one function! And what about require_course_login() then? Well, it is good to know that this one was primarily intended for the Resource module. When a resource is published on the Moodle front page, no login is required to view it. But if the resource is created inside the course, normal access control checks should be done. And that is exactly what require_course_login() does. It requires user being logged in only if we are inside the course. If we are on the front page, just a basic checks are done (hidden activity? forced login? etc). It is important to realise that this is allowed only in those scripts where no input is expected from the user. Once the script handles creating some content (submitting a post in Forum, responding in Choice or taking an attempt in Quiz, for example), full require_login() must be used. It is a general rule in Moodle that only authenticated users are allowed to produce a content.

Post scriptum

I decided to release these Moodle development traffic (MDT) posts on Tuesdays instead of Mondays. Tuesday is our code review day before we release weekly build on Wednesday so it makes sense to synchronize it.