EVALUATING MODELLING SOFTWARE: THE HIDDEN FEATURES P Banfield Introduction There is general agreement that the costs/ben efit balance of hyd raulic modelling is now heavily in its favour. T he massive advances in both speed of b uild and ease of use over recent years make modelling the best approach to a far greater range of issues rhan was the case even a few years ago. H owever, as many users know as they cry to select modelli ng sofrware, comparative evaluations of sofrware packages are difficult. T here are several reasons for chis, which are perhaps inevitable given the breadth and depth of function with in modern modell ing sofrware, including: • there are very few people with equally detailed knowledge and experience of several products that is necessary for true comparison; • it is almost impossible to devise an evaluation harness, in formal or forma l, that does not contain a built-in bias cowards one product or another; • geography also plays a part in evaluation, with some prod ucts looking good for a specific lo cation but not being as applicable elsewh ere; • the usual binary tick or cross approach to a feature list for software selection is an oversimplification of how well each produce meets each criterion - scales of 1 to 5 are more appropriate; • che weigh tings that should be applied to the values in a rick list vary from customer to customer, and even from proj ect to project, accord ing to their specific needs, and identifying these weightings requires extensive experience of the use of modelling; • produce com parison is a lengthy and therefore expensive process, so any published results are likely to have a commercial agenda behind chem rather than being a truly independent approach; • one of the most important attrib utes of any sofrware product is ics usab ility, and ch is is che most su bjective and therefore the hardest to quantify. The detailed technical functional ity of different packages has been extensively covered in various publications. Taken together, these provide a fair view of the engineering functionality of produces -
whether or not they use rhe St Venant equations, and if so, how they solve rhem, whether the modelling is dynamic, uses variable timesceps, can represent Real Time Controls, and similar important fun ctionality. Clearly any modelling package muse meet a certain level of calculation accuracy to p roduce valid resu lts on which capital and operational decisions can be soundly based. In addition, depending on che complexity of che system it is required co model, ic must have a certai n bread ch of feacures. Bur producing accurate results th rough a fast and stable simulation engine is only pare of che story for hydraulic modelling packages, a necessary bur not suffi cient condition. A number of ocher features chat lie outside this core technical/math ematical fu nctionality are just as important, but are freque n tly not to the fo re in the evaluation process. Hard to measure in many cases, they dictate the usability, productivity, and ulcimately the cost/benefit balance of modelli ng.
It is worth putting in the effort to determine the true cost/benefit. Productivity Features The prod uct ivity of th e modeller or modelling ream is the key determinant of the cost of modelling: good modelling software contains a range of productivity aids that can dramatically lower the effo rt of building, calibrating, running and maintaining a model. Prod uct ivity features can be categorised u nder rwo headings - technical features, and interface feat ures.
Technical features The best modelling sofrware undertakes a number of rhe simple but time-consuming tasks ch at are required during the model build process chat ocher sofrware products leave to the user. Data checking and cleaning provides an example. This activity is someti mes thought to require human intervention, but in fact once the rules have been established the checking is merely examining each number or attribute against a check list and flagging anomalies - a task
Journal of the Australian Water Association
chat sofrwa re can undertake far better, and at a much lower cost, than people. Equally, if required, rhe sofrware can apply inference rules where errors are id entified, based on interpolation berween valid data points, and autocorrect the data. Both these processes must cake place u nd er fu ll user control, but the drudgery and elapsed time of data checking can be greatly reduced. The same principle applies to con nectivity data. Because modelling data is often imported from GIS, and GIS is nor particularly concerned with co nnectivity, connectivity errors are common in the first pass at the model. Checking th is automatically is possible in a network modelling system that has the concept of conn ectivity bui lt- in, provi ding another major productivity aid in a viral validation area. Assigning roughness coefficients can also be au tomated. Although some roughness factors are selected to d irectly reflect a specific and unusual p ipe condition, most factors relate solely to properties of the pipe, such as age and material. T he software can allocate the maj o rity of roughness factors automatically from irs knowledge of rhe pipe attrib utes, and the few unusual instances can be specifically entered. Calibration of a model is typically a major component of the bu ild effort, and irs automation can make a major contributio n to reducing build cost. Finally on tech nical features, the run rime and stability of the model is a determinant of effi ciency - in many cases run time is dead time for the modelli ng staff, and unstable models that fail d uring runs multiply th is dead time and create a hidden ad dition to the coral cost equation. Speed and stability are vital attributes to include in so frware selectio n criteria.
Interface features T he issue of interfaces can be addressed as three separate aspects chat are all keys to good productivity: the user interface, multiuser operation, and interfacing to ocher sofrware. Most modelling sofrware now has a windows-like GUI as the user interface. But for maximum productivity, ic is vital chat the interface is effective, in tuitive for W indows users, and critically, consistent