One of the things that I’ve been doing is retrofitting Screengrab to use my new include architecture. You could think of it as a proof of concept. It has actually worked out really really well. I’m very impressed with the way the code hangs together now. Version 0.94 will be interesting for those who like to view the source.

I’ll be picking up BookSmarts again very soon. I also have another new project on the go. It doesn’t have a real name yet (it’s working title is Transfoxite), but it is essentially going to be a website that provides a large number of reference translations extracted from Firefox extensions for the purpose of making localisation of said extensions a little easier. Things like “Save As” and “Default save location” are used over and over again in apps, so I’m hoping to make a website that will allow people to select the phrases they want and then get most of their translations done for free. I’ll be leaning pretty heavily on the work done by babelzilla.org. It’s not meant as a replacement, I would just like to be less of a burden on their over-stretched community.

I’m now using highlight which seems to happily colorize all types of code.

A word of warning though - it only provides a framework for doing code highlighting, by dynamically adding css classes to code fragments. You still have to pick which colors and format you want the classes to map to.

I didn’t realise this so I kind of “borrowed” some of the css from their website. I hope they don’t mind. I think it looks awesome. I didn’t take the little pictures in the corners though. That would be too rude if you don’t have permission.

Thank you to the guys at Software Maniacs!

var list = new Packages.java.util.ArrayList();

Basically, to use any standard Java packages from within Javascript, you just use the fully qualified package name and append “Packages” to the front of it. This is kind of cumbersome, but you can ‘include’ classes by just assigning them to a local variable:

var ArrayList = Packages.java.util.ArrayList;

var list = new ArrayList();

Now, this is all fine and it will get you through most Java work in Javascript (these concepts have worked fine for Screengrab!), but what if you need to implement a Java interface from within Javascript?
Well, I’ve tried. When I was trying to get clipboard support for the Java-based grabber in Screengrab I looked for answers and tried a few things, but I just couldn’t do it. I eventually gave up and currently Screengrab will only copy images to the clipboard using the Canvas-based grabber.

The only way to do this is to somehow create a Java class externally and then load it from within Firefox. This is possible, because Firefox lets you create components in other languages and link them in using some magic. The problem with components is that there is a massive amount of crap to write to get one to work.

Here is a tutorial on creating a Java component for extension development that I found. This is the Java class it is trying to get access to. And this monolithic file is the Javascript code that you have to write to get Firefox to understand it. I haven’t read it. I stopped about 2 pages in and thought “there has got to be a way to do this that doesn’t suck”.

And there is…

From this page at Mozillazine we find “the easy way” (and actually, as it turns out, the easy way comes from the same guy that did the hard way.

As you can see from the tutorial, it boils down to using a ClassLoader. Essentially, since we can access most standard Java from inside Javascript, it stands to reason that we can create ClassLoaders and use reflection. And that’s what this code does. What it doesn’t do is tell you how to do this with a local url, pointing to a jar within your extension. Here’s how…

var id = "booksmarts@getbooksmarts.org";
var ext = Components.classes["@mozilla.org/extensions/manager;1"]
    .getService(Components.interfaces.nsIExtensionManager)
    .getInstallLocation(id)
    .getItemLocation(id);

var cl = new Packages.java.net.URLClassLoader(
    [new Packages.java.net.URL(
        new Packages.java.io.File(ext.path + "/components/tiny.jar").toURL())]);

var aClass = Packages.java.lang.Class.forName("Tiny", true, cl);
var instance = aClass.newInstance();

ftst.logger.debug(instance.getMessage());

I’m pointing it to the components directory in my extension folder. When the extension is installed, the xpi is unzipped, but the chrome.jar isn’t, so if you want to load the your classes, you’ll need to put them somewhere that gets extracted. “components” seems like a good place for that, since it contains libs and the like.

Now obviously this is a bit ugly, but it could easily be wrapped in some code that does most of the work for you. To the point where you should be able to write code that looks like the following (or much much better):

var tiny =
    jarLoader.using("tiny.jar")
        .createNew("org.getbooksmarts.Tiny", arg1, arg2, arg3);

So I didn’t have to adapt log4js to get it to play nicely in chrome at all. However, I did have to change it to make it a little more Andy friendly.

Basically, I wanted the log output to hit the console (using dump()) and to look like…

startTime logLevel [fileName:lineNum] message\n
eg.

23/03/07 22:10:11:342 DEBUG [overlay.js:14] A debug message
23/03/07 22:10:12:589 DEBUG [overlay.js:21] Another message

Log4js can’t log to a command prompt using dump(). That was an easy change, we just needed to add another appender, as below.

/**
 * Appender writes the logs to the Shell of Firefox when it is launched with
 * firefox -console (and has the necessary properties enabled)
 * PLEASE NOTE - Only works in Firefox
 * @constructor
 * @extends Appender
 * @param logger log4js instance this appender is attached to
 * @author Andrew Mutton
 */
function FirefoxConsoleAppender(logger) {
...
	this.layout = new MozStackLayout(true);
}

FirefoxConsoleAppender.superclass = Appender.prototype;
FirefoxConsoleAppender.prototype = {
	/**
	 * @see Appender#doAppend
	 */
	doAppend: function(loggingEvent) {
		dump(this.layout.format(loggingEvent));
	},
...

So now we can dump to the console. Notice that there’s a new Layout there too? You probably didn’t, since not many people use log4js I imagine.

The new layout is there to take advantage of the information we can get from Components.stack - notably, line numbers and source files.

/**
 * MozStackLayout is a layout that takes advantage of stack data from Mozilla.
 *
 * startTime logLevel [fileName:lineNum] messagen
 *
 *
 * @constructor
 * @extends Layout
 * @author Andrew Mutton
 */
function MozStackLayout(showDate) {
	this.LINE_SEP  = "n";
	this.showDate = showDate;
}
MozStackLayout.prototype = {
	format: function(loggingEvent) {
	    if (this.showDate && Log4js.simpledate) {
    	    var d = new Log4js.simpledate.SimpleDate(loggingEvent.startTime.getTime());
          	return d.toFormattedString("~d/~k/~y ~H:~m:~s:~S") + " " + loggingEvent.level.toString() + " [" +  loggingEvent.getStackFileName() + ":" + loggingEvent.getStackLineNum() + "] " + loggingEvent.message + this.LINE_SEP;
	    } else {
	        return loggingEvent.level.toString() + " [" +  loggingEvent.getStackFileName() + ":" + loggingEvent.getStackLineNum() + "] " + loggingEvent.message + this.LINE_SEP;
	    }
	},
...

this is the bit that uses simpledate: to format the date in a less ugly way than you usually get for free from Javascript. I honestly have no idea why there isn’t a nice date formatter built into the language. It is ridiculous.

There’s a few more changes here as well. I had to change the log() method so that it attempts to get stack data (if it’s available).

log: function(message, logLevel) {
    if (Components && Components.stack) {
        var stack = Components.stack.caller;
        if (stack.toString().indexOf("log4js.js") != -1) {
            stack = stack.caller;
        }
        var loggingEvent = new Log4js.LoggingEvent(this.category, logLevel, message, this, stack.toString());
    } else {
        var loggingEvent = new Log4js.LoggingEvent(this.category, logLevel, message, this);
    }
...

and also LoggingEvent, so that it can give the stack information

getStackFileName : function() {
    if (this.stackInfo != null) {
        var split = this.stackInfo.split(" :: ");
        var fileName = split[1];
        if (fileName.lastIndexOf("/") != -1) {
            var pathSplit = fileName.split("/");
            fileName = pathSplit[pathSplit.length - 1];
        }
        return fileName
    }
    return null;
},

getStackLineNum : function() {
    if (this.stackInfo != null) {
        var split = this.stackInfo.split(" :: ");
        var lineNum = split[3].split(" ")[1];
        return lineNum;
    }
    return null;
}

So there you have it. Not very interesting today, but this should be the last infrastructure post. I think we can begin doing things properly now.

I’m very interested in getting log4js to be configurable via a properties files (ala log.properties in log4j). This doesn’t seem too hard, so I may have to check it out. An alternative to log.properties would be to write the configuration in JS. Something like…

logger("booksmarts.log").atLevel("DEBUG")
    .withAppender("FirefoxConsoleAppender")
    .andLayout("MozStackLayout");

this would be too simple though. Maybe something more elaborate. Who can really say eh?

There’s a reasonable example of its use in the code that sets it all up, include.js. In particular, you should take a look at the ftst.configureLogger() method and the ftst.include(ftst.getMain()) call at the bottom.

Basically, a call to ftst.include will include the specified file in that place. A call to ftst.includeInScope will include the file within a confined scope and return an object that holds that scope.

Why do I care about scoping so much? Because all extension development in Firefox is in a shared scope. Your code is in everyone else’s way and theirs is in yours. That’s why it’s so important to compartmentalise your code. This isn’t such a problem in normal web-development. In a webpage all of your JS shares the same scope, but you control it all. In extension dev there can be all sorts of stuff running around, which is a nightmare. Because of this, it’s best to limit your footprint as much as possible.

In include.js, you can see that I’m using Log4js to do logging, and I’m including it in a hidden scope, so that it doesn’t get into the global namespace. I’ve changed it (I’ll explain why in my next post) and now it also uses this date formatting code, which I configure into it after it’s initialised (from another hidden scope). Why don’t I just include it directly? Well, because there are…

Problems with mozIJSSubScriptLoader and scope
Remember that this is the code that sets up the loader.

var jsLoader = Components.classes["@mozilla.org/moz/jssubscript-loader;1"]
    .getService(Components.interfaces.mozIJSSubScriptLoader)

then we call (1)jsLoader.loadSubscript(filename) to load a JS file into the global scope, or (2)jsLoader.loadSubscript(filename, localScope) to load a JS file into the variable localScope.

The complication happens when we try to include one JS from within another. You can call (1) from within (1), (1) from within (2) and (2) from within (1). What you absolutely can’t do is call (2) from within (2). You can’t include something in a localScope when you are a script that is being included in a localScope no visible error occurs, but your script will halt and nothing will happen.

That would be alright, if it weren’t for this problem: A script included in a global scope from a script that is being included in a localScope will be globally available outside the local scope. If you include (1) from (2), (2) is confined to its localScope, but (1) is completely global. This seems wrong to me. It could be argued both ways, but I think that a script included from (2) should be confined to (2).

Anyway, what this means is that for Log4js to use simpledate, simpledate must either be included globally (by include.js or Log4js) or it can be included locally by include.js and then set into Log4js after it is loaded. I care a lot about scope (as you can tell) so I’m doing the latter.

The very worst thing about Firefox extension development is the shared JS context that every extension shares.

PS
I know that dojo does something similar to this, but from reading, it doesn’t seem to do it right (particularly for chrome) and also does a lot more than I want.

There’s two ways that I know of to do this… The first outputs to a shell and the second uses the Error Console.

Using dump()
Firefox has a function called dump() that will just dump whatever you give it to a standard console. To use it you have to…

• Start Firefox in console mode - from a command prompt type “/path_to_firefox_app/firefox -console”
• Set “browser.dom.window.dump” to true in about:config - the excellent Extension Developer’s extension can do this for you.
• Use dump("My message\n"); to dump to the console.

It isn’t pretty, but it works.

The Console Service
The other way is to use the console service, which will put debug statements into Firefox’s Error Console.

var consoleService = Components.classes["@mozilla.org/consoleservice;1"]
        .getService(Components.interfaces.nsIConsoleService);
    consoleService.logStringMessage("My Message");

It’s easier, and requires less config, but it isn’t working so well for me right now. Sigh…

What we really need for logging in Firefox is a log4j style logging framework. That would be super sweet!

The MIME type for XPIs is “application/x-xpinstall”.

(this is mostly just a mental note for me - I always forget to configure this on my servers)

The dynamic injection of script was stupid. It won’t work in XUL (but it will work in HTML).

The secret is found in two things.

1. An XPCOM interface (isn’t it always?): mozIJSSubScriptLoader
2. A JS hook into an XPCOM interface: http://www.mozilla.org/scriptable/components_object.html#_stack

The first one lets you load Javascript dynamically within XUL apps. You call it as follows…

var jsLoader = Components.classes["@mozilla.org/moz/jssubscript-loader;1"]
    .getService(Components.interfaces.mozIJSSubScriptLoader);
jsLoader.loadSubScript("chrome://booksmarts/content/overlay.js");

The cool thing is that you can also pass it in a scope object, so all of the members of the JS file you load will be attached to it. If overlay had a function called jon(), then the code above would’ve loaded it up into the global scope and I could just call jon() to access it.

But if I’d called loadSubScript(chrome://booksmarts/content/overlay.js, anObject) then I would call anObject.jon() to access it. Brilliant!

So we can load Javascript. But I said I wanted to do it with one file. Now this is where it gets hacky. In HTML, my one-file plan involved navigating the DOM to find my script element and then getting the src value from it. So the script tag would look like <script src="include.js?main=another.js">.

Well it turns out that doesn’t work in XUL. As far as I can tell, the script tags are destroyed after they’re loaded in an overlay (document.getElementById can’t find it, so it must be lost). So instead, I discovered (accidentally while I was trying to find out how to load scripts) that you can get a Javascript stack dump from XPConnect by calling Components.stack. This yields something like:

JS frame :: chrome://booksmarts/content/include.js?main=chrome://booksmarts/content/overlay.js :: anonymous :: line 56

Notice that Firefox loaded the script perfectly, despite the URL params AND it preserved them into the stack trace. AWESOME!

If you assign that to a string and split it with split(" :: ") you get an array, the second value of which is your url chrome://booksmarts/content/include.js?main=chrome://booksmarts/content/overlay.js.

From that, a handy bit of URL parsing courtesy of a helpful person on the internet and you get your parameter. Brilliant.

That’s basically all of the pieces you need to do dynamic imports in Javascript. Which I’ll continue next time…

Instead, I started dealing with one of my pet Javascript peeves. I hate having to put multiple <script> tags in my code to include lots of JS files. If one piece of Javascript depends on another piece of Javascript then I WANT IT TO SAY SO!

I’d hoped that someone else would’ve figured out a neat way of doing this by now, but they haven’t (dojo comes close, but it’s not good enough).

Here’s what I want…

1. import(”org.getbooksmarts.SomeClass”) should load the JS at org/getbooksmarts/SomeClass.js from the root.
2. the root should be definable by a parameter
3. my importer should be able to start my other js. It should act as a bootstrap loader using something like <script src="import.js?main=org.getbooksmarts.Booksmarts">
4. ideally it will work in chrome and html, but I can live with just chrome for now

It’s conceptually not too hard. The addition of new JS files is done by adding new <script> tags into the DOM. That works fine in HTML, I’m just having trouble in chrome for some reason.

This is why Venkman. Everyone thinking about developing extensions NEEDS the Mozilla Javascript debugger. This is not an optional step.

So I took the theme from Screengrab (because I’m so original and clever) and tweaked it a bit so that I can tell which site I’m writing things for. Also I made a logo for this project (why should Screengrab get all the logos?).

Actually… when I say “made” I mean “took”…

… Thanks go, once again, to Nick La of N.Design Studio for the site template and for the awesome free vector clipart on his site, which a man with zero graphic design skills (me) cunningly took and used.