General Messaging Performance

The test scenario that we used for the full MTS performance test consisted of a client on Clement and a server on Beagle. Again, we used all of the three available networks. We compared the BEEP mapping of AMME against regular Apache Axis, and Apache Axis using persistent HTTP con-nections, as shown inTable 7.11.

In our scenario the client sends messages as quickly as it can. A single message consists of a string, a date, and a floating point value that all re-main constant throughout the test, and a sequence number that increases

Table 7.11: Protocols of the MTS experiments Protocol Description

HTTP The default HTTP 1.0 shipped with Apache Axis using XML with synchronous invocations

PHTTP A version of HTTP 1.0 hacked to keep connections persistent using XML with synchronous invocations

AMME The MTS with the BEEP mapping of AMME using Xebu with asynchronous invocations

Figure 7.1: Per-invocation times over the LAN connection

for each message. The server verifies that it receives the messages in se-quence and responds with the sese-quence number. The number of messages that we used varied between 50 and 250.

The first results in Figures7.1, 7.2, and 7.3show the total time for the experiment as well as the total time divided by the number of invocations for each of the three networks. The total times are drawn as regression lines whereas for the per-invocation times the average values are connected. In both cases, error bars are shown at one standard deviation.

The MTS has a significant overhead over the LAN connection when compared to Axis. As the implementation of the MTS we used was a wrap-per around Axis, this is as expected. However, even on the WLAN connec-tion we are starting to see the benefits of asynchronous invocaconnec-tions when AMME catches up to even PHTTP, and would for a larger number of mes-sages surpass it.

The deviation in the times for HTTP over WLAN is very large. We note that a lost TCP SYN segment will, with the implementation in Linux, cause a 1.5-second timeout before retrying. As the HTTP implementation needs to constantly open new connections, the likelihood of this happening grows. With the small latencies of the WLAN, such delays effect a large variation in the results.

HTTP

Figure 7.2: Per-invocation times over the WLAN connection

HTTP

Figure 7.3: Per-invocation times over the GPRS connection

In the measurements over GPRS the benefits of asynchronous invo-cations materialize most clearly. Here the dominant factor in the timing is network latency. As the MTS uses both asynchronous invocations and the message bundling of AMME, the effects of network latency are much smaller for it. Comparing the two HTTP protocols we see that one network round trip is spent by plain HTTP to open each connection.

We also measured the total amount of data sent in the 250-message ex-periment, shown inFigure 7.4. The data is split into three parts, the ap-plication data (indicated as XML), the data used by the apap-plication proto-col (HTTP or AMME), and finally TCP segments containing no application data.

Clearly the amount of XML data sent by AMME is much lower, since it uses Xebu instead of XML. We also note that the amount of protocol overhead for AMME is much lower than for the two HTTP protocols. The two HTTP protocols send approximately the same amount of application data, but the constant opening of new TCP connections makes HTTP send significantly more data in total.

LAN

Figure 7.4: Amounts of total data sent

30

Figure 7.5: Per-invocation times using a mobile phone

We can see that the amount of overhead caused by the AMME proto-col in this scenario is only approximately 25 % of the total data sent, even at its highest over the LAN. This appears to be similar to the overhead caused by HTTP for XML. However, taking absolute figures, we can see that Xebu combined with HTTP would make the protocol overhead to be approximately 75 % of the total message size. This therefore provides val-idation for our observation insection 3.1 that a binary format for XML is not sufficient on its own.

Finally, we ran a similar scenario with the 3660 phone as the client using the HTTP mapping of AMME. There is no comparison point for these mea-surements, soFigure 7.5shows only the total time and time per invocation for this single case. Note that we used a fewer number of messages for this experiment.

By extrapolating the figure for total times we can see that the overhead of the experiment is approximately 30 seconds. As we started each exper-iment from a clean slate, this includes all network setup that was required for the phone, and is therefore understandable. If network connectivity had

been available from the start, this delay would have been much smaller.

The average time per invocation appears to settle to around 0.5 seconds, which is much higher than it was for AMME inFigure 7.3. This is explained by the use of the HTTP mapping with its thread-based flow control. Since in our phone implementation only one message can be in transit in either direction at the same time and message size is bounded, the largeness of this figure is also explained.

Chapter 8

Conclusions

We have used the MTS described here as a component of the mobile mid-dleware platform built by the Fuego Core project. While we consider it usable for basic XML messaging, further experience with real-world sce-narios has revealed certain areas of improvement. We first describe what we have found useful and then consider some simple enhancements of the current system as well as larger areas of future work.