As opposed to the numerical evaluation of performance, the second side to the artifact based evaluation are the qualitative aspects. These attributes relate to the unquantifiable pros and cons with respect to the simulation (continuous system), the case-specific CFB-model, and the quality metrics for the associated service.
The general dimensions for comparison to simulations are (Table 13): real system, other modelling approaches, and organizational effects (Robinson 2004, Page 8). All of these aspects are set in favor of simulations as oppose to the opponent; for instance, develop-ment with using real systems (rather than simulations) to study phenomena is likely to cost
more, be less controllable, and consume more time (Table 13). While the benefits to the development with using simulations (Table 13) are considerable, there are some obvious dis-advantages to toggle - if compared to development using real systems, simulation does not produce a product, it gives an estimate.
Table 13. Simulations as tools of development (Robinson 2004, Page 8)
Benefit type Factor
Simulation versus Real System Cost, control, and time
Simulation versus other modelling approaches Modelling variability, restrictive assump-tions, and transparency
Simulation: the management perspective Fostering creativity, creating knowledge and understanding, visualization and communication, and consensus building
The key problems to simulation based development are internal. Disregarding the common problems also associated with stateless applications, still leaves the problems surrounding the state space (Figure 4); where missing the "acceptable" output range is likely to result in unforeseeable consequences for the system output. The specific problems associated to modelling the simulation (continuous system) are:
• The output is not within the available state space - the solver is terminated due to constraint(s) being triggered;
• The solver is unable to take the next time-step forward - the solver is terminated due to the configuration;
• The model exceeds the maximum time without a solution - the output is not within the state space; and
• The model fails unexpectedly - due to the complexity of the simulation, they are more likely to fail than the stateless applications.
To open up the simulation built process and to evaluate the performance of the artifact, a secondary goal of this research is to evaluate the simulation based on the service design principles. What makes the Simulink-Apros integrated CFB-model to constitute as a service,
is essentially caused by the interface to bridge between the two platforms. APROS does not access the Simulink routines directly, nor does the interface contain any direct references to any specific Simulink model. The communication format for the service provider and client to have is, therefore, request-response driven (Figure 9).
Part of the process for evaluating the viability of simulation to function under service archi-tecture is to establish metrics to quantify them. The primary metrics for IT systems can be described to six categories; these metrics are qualitative because they are relative and thus, used in interval scales (Dobson 2004):
• Bandwidth (or throughput) -> speed of operation,
• Latency (or delay) -> speed of operation,
• Jitter -> failure Semantics,
• Error Rate -> accuracy of operation,
• Availability (or up-time) -> dependability (including availability, reliability, security and safety), and
• Security -> dependability (including availability, reliability, security and safety).
While these categories can capture the plausible problem areas for generic applications, they do not do so for specialized domains like the mathematical modelling and simulation. Due to the inherent complexity of simulated systems, the factors of interest may vary widely depending on the system type: continuous or discrete, dynamic or static, and stochastic or deterministic. A usable QoS specification for simulations is, therefore, likely to be modular and expandable. It should not necessarily attempt to define an exhaustive list of metrics, but to allow domain-specific extensions to adopt to the requirements for the more complex specifications.
The practical standpoint to the service quality metrics and simulation is the MDA. The ar-chitecture sets specification, terminology, and technology together. The major difference to using MDA is that it is neither technical or domain based approach; for it combines both.
Reflected by this research; the modelling aspect to MDA is addressed in Chapter 2 and the technical aspect for service is addressed in Chapter 3.
6 Discussion
The solution proposed for the integration between the Simulink and APROS platforms is based on the adoption of techniques more common to the service-oriented paradigm. The main product of this research is the shared interface enabling the two simulation platform to execute on the same state space by using the client - provider setup as the schema for interaction. As the result, the process to bind the two simulation platforms together to ex-ecute on the same state space has both the negative and positive consequences. The main negative cause is due to the increase in the overall solution complexity for the CFB-model;
as an effect, the stability of the entire system is affected. However, the capability to execute build on the CFB-model to DLL enables steps to be taken to automate the CFB-model with external hardware or software.
The method used to encapsulate the work is based on the principles of design science. Be-cause the work is centered around an artifact, it is constructive. The process used to describe the motivation and findings is detailed in Figure 1, with the approach detailed in steps one to four. These steps are: problem (motivation), theoretical approach, design, development, and evaluation. As a constructive research, the evaluation is based on quantifiable and qualita-tive findings. In essence, the goal was to produce pragmatic, grounded, and interdisciplinary work.
Besides, detailing the implementation and evaluation, this research also contains two models.
The behavioral model (Section 2.2), is defined to capture the continuous system and the information model (Section 3.3) is defined to capture the interaction requirement for the model to model communication. Because the communication requires distributing the logic and using a mediator interface to relay commands, the required interactions resemble the service driven approach. As such, the theoretical background of this research was based on the principles of SOA and MDA.