Wednesday, October 1, 2008

DataSource dependencies

DataSources are one of the main attractions we had to SmartClient to start this project. Our current client is driven by metadata pulled from a database. A DataSource is exactly the same thing handling most of the features we already expect. However we have the need for user-specific security to drive the availability of fields, selections of values in comboBoxes, or even default values for new records.

So how do we provide DataSources customized for the user? Well, we could easily pull the metadata from the database, merge in the user's security and then write the DataSources into the web page (we are using ASP.Net). The same could be accomplished by publishing a Url that will do the same thing and make an RPC call to load them. As there will ultimately be around 200 DataSources in the application the time to build every one of them at login time was considered too long. Note that our main menu (a treeGrid) is user-specific based on security as well so it is loaded from the server through a DataSource but the DataSource definition will be the same for all users.

Now that pulling all DataSources at login time was rejected (at least at present), we need a way to have a form specify what DataSources it depends so they can be loaded before it is created. First thing we needed was an easy way to load a DataSource on demand from the server. To do this, the DataSource class was extended to load on demand. Note this feature is normally only available with the SmartClient Server.

isc.DataSource.addClassMethods({

// locating dataSources
dataSourceBaseURL: "data/datasources/",

// loadSchema - attempt to load a remote dataSource schema from the server.
// This is supported as part of the SmartClient server functionality
loadSchema: function(name, callback, context) {
this.logDebug("Attempt to load schema for DataSource '" + name + "' from " +
this.dataSourceBaseURL + name);

isc.RPCManager.sendRequest({
evalResult: true,
useSimpleHttp: true,
httpMethod: "GET",
actionURL: this.dataSourceBaseURL + name,
callback: this._loadSchemaComplete,
clientContext: {
dataSource: name,
callback: callback,
context: context
}
});

return null;
},

_loadSchemaComplete: function(rpcResponse, data, rpcRequest) {
var clientContext = rpcResponse.clientContext
var name = clientContext.dataSource;
var callback = clientContext.callback;
var context = clientContext.context;

// Now that the dataSource is loaded, we can leverage the DataSource.getDataSource()
// method to make the callback.
var ds = isc.DataSource.getDataSource(name);
context.fireCallback(callback, "ds", [ds], context);
}
});

This is great but if we have a DataSource dependency, how can we make sure it is loaded before the form is created or used? We were looking for a way to use the form naturally, such as:

_rightPane.addTab({
ID: tabId,
title: name,
canClose: true,
pane: form
});

After some thought and research I thought about early prototype use of ViewLoader. It does something similar to what we have to do with loading on demand except we want to load a DataSource, not the view (form). Taking a look at the ViewLoader implementation revealed a simple solution to our issue. The ViewLoader acts a proxy for the view until it has been loaded. This would be a perfect prototype for a ViewDependencyLoader. Our new usage target then became:

_rightPane.addTab({
ID: tabId,
title: name,
canClose: true,
pane: ViewDependencyLoader.create({
autoDraw: false,
viewClass: className
})
});

where className is the name of the class holding the DataSource dependency list (dataSources) and a function to create the form called createForm of all things.

Looking further, deriving from ViewLoader didn't seem to be ideal. What was needed was to refactor the ViewLoader into a base class called ViewProxy. Hear is what I came up with:


// NOTE: we are a subclass of Label as a means of showing the loading message
isc.ClassFactory.defineClass("ViewProxy", isc.Label);

isc.ViewProxy.addProperties({

//> @attr viewLoader.loadingMessage (HTML : "Loading View..." : IR)
// Message to show while the view is loading
//
// @group viewLoading
// @visibility external
//<
loadingMessage: "Loading View...",
align: isc.Canvas.CENTER,

// so that we get allocated space in Layouts, instead of autoFitting
overflow: "hidden"
});

isc.ViewProxy.addMethods({

initWidget: function() {
this.Super("initWidget", arguments);

// if we've been given a placeholder widget, add it
if (this.placeholder) this.addChild(this.placeholder);
// otherwise show the loading message
else this.contents = this.loadingMessage;
},

draw: function() {
if (!this.readyToDraw()) return this;
this.Super("draw", arguments);

if (this.view) {
this.addChild(this.view);
this.view.show();
} else if (!this.loadingView()) {
// all view loading
this.loadView();
}
return this;
},

// simple layout policy just fills the view
layoutChildren: function() {
this.Super("layoutChildren", arguments);
var children = this.children;
if (!children || children.length == 0) return;

var child = this.children[0],
width = this.getWidth(),
height = this.getHeight();

// don't resize a loaded view that has specific sizes set on it
if (child._userWidth != null) width = null;
if (child._userHeight != null) height = null;

// NOTE: we intentionally occlude styling such as borders, if any, which are only meant to
// exist while we are showing the loading message
child.setRect(0, 0, width, height);
},

destroy: function() {
if (this.placeholder) this.placeholder.destroy();
if (this.view) this.view.destroy();
this.Super("destroy", arguments);
},

// dynamically sets a custom placholder
setPlaceholder: function(placeholder) {
if (this.placeholder) this.placeholder.destroy();
this.placeholder = placeholder;
this.addChild(placeholder);
this.placeholder.sendToBack();
},

loadView: function() {
if (this.placeholder) {
this.placeholder.show();
this.placeholder.bringToFront();
}
// change contents back to loading message on reload
if (this.view != null) {
this.view.hide();
this.setContents(this.loadingMessage);
}

// Overrides should be written as:
// loadView: function() {
// this.Super("loadView", arguments);
// ...do setup here...
// this.setView(view);
// // Notify observers
// this.viewLoaded(this.view);
},

loadingView: function() {
return false;
},

setView: function(view) {
if (view != null && view == this.view) return;

this._viewSet = true;
this.setContents("&nbsp;");

if (this.view) this.view.destroy();
this.view = view;

if (view == null) return;

// add the view as a child, suppressing drawing until we have a chance to size it
this.addChild(view, null, false);
this.layoutChildren();
view.draw();
this.logInfo("showing view: " + view);

if (this.placeholder) this.placeholder.hide();
// hide loading message
this.contents = "&nbsp;";
},

getView: function() {
return this.view;
},

viewLoaded: function(view) {
// observable/overrideable
}

});

This offers a good base to build a view dependency loader. A new ViewLoader should also be possible on top of the ViewProxy but I have not refactored it yet. Here is the ViewDependencyLoader:

isc.ClassFactory.defineClass("ViewDependencyLoader", ViewProxy);

isc.ViewDependencyLoader.addMethods({

//> @attr viewDependencyLoader.viewClass (URL : null : IR)
// Name of view (form) class to setup.
//
// @visibility external
//<
//viewClass: null,

loadView: function() {
this.Super("loadView", arguments);

// Setup dependencies
var viewClass = isc.ClassFactory.getClass(this.viewClass);
if (!viewClass) {
// Class not found.
}

if (viewClass.dataSources) {
// cast dataSources to an array if we need to
var dataSources = isc.isA.Array(viewClass.dataSources) ? viewClass.dataSources : [viewClass.dataSources];

// if no dataSources, view is ready
if (dataSources.length == 0) {
this.setView(viewClass.createView());
this.viewLoaded(this.view);
return;
}

// We will ask the DataSource factory to get each dataSource and call us back
// when it is ready. This may be instant if the dataSource is already loaded
// or may be after the dataSource is loaded from the server. To know when we
// are done, save the list of required dataSources and remove each as we get
// the callback.
this._waitingDataSources = [];

for (var i = 0; i < dataSources.length; i++) {
if (!isc.DataSource.getDataSource(dataSources[i])) {
this._waitingDataSources.push(dataSources[i]);
/*ignore return value*/isc.DataSource.getDataSource(dataSources[i], this._dataSourceLoaded, this);
}
}

this._setViewIfDone();
}
// Overrides should be written as:
// loadView: function() {
// this.Super("loadView", arguments);
// ...do setup here...
// this.setView(view);
// // Notify observers
// this.viewLoaded(this.view);
},

_dataSourceLoaded: function(ds) {
this._waitingDataSources.remove(ds.ID);
this._setViewIfDone();
},

_setViewIfDone: function() {
if (this._waitingDataSources.length == 0) {
// All dataSources have been loaded. View is now ready.
var viewClass = isc.ClassFactory.getClass(this.viewClass);
if (viewClass) {
this.setView(viewClass.createView());
this.viewLoaded(this.view);
}
}
}
});


So, now we can specify form DataSource dependencies cleanly and worry about the more important stuff... See this post for details.

3 comments:

Charles Kendrick said...

Nice work David, and thanks so much for sharing your solution. I've gone ahead and blogged about it on the Isomorphic blog.

http://blog.isomorphic.com/?p=61

Some suggestions to take your technique even further - you could provide an API on the viewClass that allows the ViewDependencyLoader to find out what data the view will need when it starts up, and automatically retrieve that data along with the DataSources themselves. Use TreeGrid.initialData and/or ResultSet.initialData (via ListGrid.dataProperties) to populate a DataBoundComponent with an initial dataset but still have it be capable of continuing with load on demand for large datasets.

Similarly, you might have a kind of ViewManager that tracks and manages a set of ViewDependencyLoader classes, providing an LRU or similar strategy that allows views to be pooled and re-used.

Anyway, great work and thanks again for sharing it.

Cheers,
Charles

Anonymous said...

Hi, Am investigating the use of the tool which I saw at AjaxWorld.com. We are 100% .Net shop and connect to sql server via ado and datareaders. It appears to me that we need install and use the jdbc driver to connect to the database for the smartclient visual builder.

We feel a little nervous about this as we are adding another layer of technology we are not familar with.

Was this a concern to you guys? Did you have java experience in house?

Obviously like most software groups we try to minimise the layers of technology in use.

Thanks
David

david.bilbow@customercommunity.com.au

dave said...

David,

We are not using the visual builder at all and do not have java experience in house. During evaluation we used the visual builder to understand some of the object relationships but after that found it easier to put together forms, etc. by hand.

We did build smartclient-specific ASP.Net web services so we can drive much of the client from server metadata (create user-specific data sources, etc.). Doing a similar project on top of SQL instead of another business layer would not be too difficult and I highly recommend this effort.

So far, the amount of javascript code necessary to build forms is much smaller than I anticipated. Once the remaining infrastructure we need is in place new forms and features should be very manageable. Sort of a hybrid declarative/event handling architecture...