|
| |
1. Functional semantic gauges
2. Data semantic gauges
3. Interoperability gauges
4. Resource and performance gauges
1.
Functional semantic gauges
The functional semantic gauges measure functional interchangeability among components,
i.e., compatibility between a component requirement specification and alternative
candidate components. This allows the user and the service broker to determine if
components exist that conform with functional requirements, and to select among conforming
components based on the estimated effort of making them usable.
 | Subsumption-based
functional gauge
 | Definition: measure the similarity in functionality of components |
| Use or significance: this gauge allows the user and the service broker to
determine most appropriate components by comparing the goal functionality with candidate
components |
| Measures: functional semantic distance as spatial distance between the two nodes
in functional ontology graphs |
| Units: number of edges |
| Measuring instruments: subsumption relationship |
| Example: Figure 1 shows an example of comparing functional semantics of different
route planner components. Gauges display the spatial distance from the goal functionality.
‘Lat-long based’ and ‘Street address based’ route planners are
geographical route planners that can be directly connected given the type of location data
available. To use the general ‘Shortest-path’ algorithm, an extra component
would be needed to convert geometric data to/from geographic locations |
|
Figure 1.
Measuring functional similarity by using the subsumption-based gauges
| Inference-based
functional gauge
| Definition: measure the complexity of a composite function |
| Use or significance: This allows the users to select among functionally similar
implementations based on implementation complexity. |
| Measures: length of the component service chain required to construct desired
functionality. |
| Units: length of component chain |
| Measuring instruments: forward and backward chaining inference algorithms |
| Example: The bar-chart gauge in Figure 2 shows the complexity of each route
planner service found (or composed). The composite component (C3) shows higher complexity.
First, a GIS Server has to create the relevant street graph using the start/end points of
the route. Then, the shortest path algorithm generates the route based on the street graph
and start/end points. This inference-based functional gauge will be useful at the system
design step to choose components with appropriate complexity level. However, the higher
component complexity may not always result in lower performance. The run-time performance
gauges have to be applied to measure the actual performance of the services. |
|
Figure 2.
Comparing complexity of components that have same functionality by using the
inference-based functional gauges
2. Data
semantic gauges
| Content-based
data gauges
| Definition: measure the similarity in contents of the data |
| Use or significance: This allows the user to determine the
difficulty in semantically transforming from one representation to another. |
| Measures: content distance as spatial distance between two
nodes in the content ontology graph |
| Units: number of edges |
| Measuring instruments: Subsumption relationship; Forward
chaining inference algorithm |
| Example: Figure 3 shows an example of comparing different
semantics of geographical location data. Gauges display spatial distance between data
components. Converting between two points represented by different coordinate systems is
semantically feasible, but between a point and a line is not feasible without additional
contextual knowledge. |
|
Figure 3.
Measuring content similarity of data components by using the content-based gauges
| Structure-based
data gauges
| Definition: measure the similarity in structure of the data |
| Use or significance: This allows the user to determine the difficulty in
syntactically transforming from one representation to another. |
| Measures: structure distance as spatial distance between two nodes in the
organization-structure ontology graph; length of conversion process between the structure
types when they are convertible |
| Units: number of edges; number of converters |
| Measuring instruments: Subsumption relationship; Forward and backward chaining
inference algorithms |
| Example: The gauges in Figure 4 measure the compatibility between different
syntactic representations of location information. Since an image of a geometric object
can be generated by performing the object rendering function on a geometric description,
the G2 gauge shows higher compatibility level. The G1 gauge indicates that it is
impossible to convert an object image to a geometric description. |
|
Figure 4.
Measuring compatibility between data organization structures by using the structure-based
gauges
3.
Interoperability gauges
| Definition: measure architecture-level semantic compatibility,
i.e. functional interoperability of connecting components together |
| Use or significance: This allows the user and service composer
to determine the appropriate (semantically interoperable) components to be included in a
composite service so that the semantic interoperability can be assured during the design
cycle. |
| Measures: length of the connector between the components |
| Units: number of intermediate functional components and data
converters |
| Measuring instruments: Forward and backward chaining inference
algorithms. Requires function and data semantic gauges. |
| Example: Figure 5 shows an example of using the
interoperability gauges to measure the semantic interoperability level between instances
of the route planner and the geo-line displayer components. The gauge G1 indicates that
the ArcView route planner result can be directly visualized by the ArcView geo-line
displayer. Also, the gauge G2 shows the MapQuest route planner result can be displayed by
a Web browser. Since the ArcView route planner result can be displayed on a Web browser by
converting it to a GIF image, the gauge G3 indicates high interoperability level. However,
the gauge G4 shows that the MapQuest route planner cannot be connected to the ArcView
geo-line displayer because there is neither a semantic match nor a data converter between
the output data of the MapQuest route planner and the input data of the ArcView geo-line
displayer. |
Figure 5.
Measuring the connectability between components using the interoperability gauge
4.
Resource and performance gauges
Using the schema definition facilities, resource and performance
schemas can be quickly defined to construct resource and performance gauges.
| Monitor gauges
| Definition: measures computing resource usage and quality of
service. |
| Use or significance: This allows the user to monitor resource
consumption of individual components as well as the entire system. Also, it allows users
to determine if the system is providing the desired quality of service. |
| Measures: CPU usage, memory usage, network usage, throughput,
refresh rate, user defined evaluation criteria, and so on. |
| Units: second, byte, byte per second, operations per second,
frame per second and so on. |
| Measuring instruments: Probes to catch events and report
usage. Use user provided architecture (ADL) description augmented with sequence/activity
diagrams to understand event sequences. Use component interface and component wrappers to
insert probes. Underlying operating system and language must provide methods to access
basic resource usage. |
| Example: Figure 6 shows a composite gauge. The three smaller
gauges measure compute time to generate street graph, communication time to send the graph
to the shortest path algorithm, and compute time to generate the shortest path. The small
gauge in the middle show that the communication time is taking the longest. As a result
the large summary gauge show that the overall performance suffers. |
|
Figure 6.
Probes monitoring input/output events and resource usage
| Comparison gauges
| Definition: These gauges measure difference between actual
performance and design-time estimated performance |
| Use of significance: This allows the user to tune the
design-time performance estimation model. Also, this allows the user to detect system
bottlenecks by comparing the actual performance against the underlying system capacity. |
| Measures: Delta of CPU usage, memory usage, network usage,
throughput, refresh rate, user defined evaluation criteria, and so on. |
| Units: percentage difference or user defined criteria. |
| Measuring instruments: Probes similar to resource usage
gauges. In addition need model of underlying hardware and network environment and
performance estimation (from other DASADA participants). |
| Example: Determine that the cycles per second is 20% less than
expected, and that the bottleneck is in the network connection between the Route Planner
and the Displayer. |
|
|
