Appendix - Methods for Data Retrieval in a Distributed Environment

1. BROKERAGE VERSUS HIGHLY HARMONISED SYSTEMS - GENERAL CONSIDERATIONS

As it is unlikely that a fully integrated, purpose-designed, harmonised system incorporating all data and services needed for travel will become a reality, at least in the near future, the task of providing high quality traveller services from door-to-door will tend to be broken down into the four following subtasks:

1. Identification of customers actual needs

2. Information retrieval out of a huge number of distributed data sources

3. Intelligent user-oriented composition of partial elements

4. Presentation of results and offers to the customer

All subtasks can already be performed to a greater or lesser degree of sophistication. However, high-quality services cannot be built by collecting data with unstructured search methods. It requires well-standardised and harmonised systems at a high level with clearly defined interfaces. Furthermore, an intermodal seamless routing service is only possible if the contributing local data sources and information systems offer a high level of accuracy in terms of time, location and reliability, etc.

For these reasons, as already stated, the achievement of this scenario should be the long term aim, and requires a considerable amount of hard work. But in the more immediate future a new style of Value Added Service Provider (or Transport Information Broker) offering one-to-one services will probably play a central role. It may also retain certain importance in the longer term as there are always new offers created in the web-economy which do not fulfil standardisation requirements from the very beginning and therefore require unstructured searches in an unstructured environment.

The following is a status report on data retrieval methods in a distributed environment and identifies possible software developments and adaptations needed to efficiently operate a broker service in the travel field. As good progress is already being made in the deployment of seamless intermodal systems for traveller services, the paper also reflects on more sophisticated methods of harmonisation, co-operation and integration of systems. In addition, a brief description of some mathematical methods which help to establish users needs is given.

2. REVIEW OF INFORMATION RETRIEVING METHODS

Information is provided in the Internet, digital libraries and (distributed) databases. In general, we distinguish between four methods for retrieving information, which are presented below:

• using a search engine
• sending out a robot/agent
• querying a (distributed) data bank by an interface
• use of an information system by a front end

2.1 Search engines

Search engines are programs that search documents for specified keywords and return a list of the documents where the keywords were found. Although the search engine is really a general class of programs, the term is often used to specifically describe systems like Alta Vista and Excite that enable users to search for documents on the World Wide Web. Typically, a search engine works by sending out a spider to fetch as many documents as possible. It's called a spider because it crawls over the Web compiling the enormous lists of URLs that are the heart of any search engine. Another term for these programs is webcrawlers. Spiders are subsets of robots/agents (see below). Because most Web pages contain links to other pages, a spider can start almost anywhere. As soon as it sees a link to another page, it goes off and fetches it. Large search engines have many spiders working in parallel. Another program, called an indexer, then reads these documents and creates an index based on the words contained in each document. Each search engine uses a proprietary algorithm to create its indices such that, ideally, only meaningful results are returned for each query.

There are numerous search engines, total number exceeds 1000. The following table provides a short list of some of well-known ones. In order to cope with the situation, so called meta search engines (e.g. webcrawler.de) have been installed which make use of many search services.

Table 1: Examples of search engines

The main obstacle for retrieving relevant information from the web is the absence of a well defined underlying data model. Search engines use more or less unstructured methods to conduct searches on poorly structured databases. If we consider the search engines on the web today, we conclude that they continue to use indexes in a very similar to those used in libraries one century ago [2]. This is why most search engines provide too many matches not relevant for the user.

These difficulties have attracted new interest in information retrieval solutions. The recent developments of modern and inexpensive graphical user interfaces (GUI) and mass storage devices have also had impacts on the science of information retrieval itself.

The main problems generating research needs are to ensure

• relevance of retrieved information
• quick response times and to develop
• efficient and friendly user interaction

Search engines are undergoing a more profound evolution at the moment, the refinement of their special tools. It is expected that very soon the Web will evolve standards, such as standard categories, ways of automatically classifying information into these categories, and the search tools to take advantage of them, that will really improve the search process.

2.2 Bots / Robots / Agents

A robot (bot is short for robot) is a software tool for digging through data. The user gives a robot directions and it brings back answers. These computer programs run automatically without human intervention and have an unusual degree of autonomy. Typically, a robot is endowed with some artificial intelligence so that it can react to different situations it may encounter.

The term (ro)bot has become interchangeable with agent, to indicate that the software can be sent out on a mission delegated to it on behalf of somebody, usually to find information and report back. (Some robots operate in a specific place, for example, a robot in Microsoft Front Page automates work on a Web page).

• autonomous - act on their own
• perceptive - perceive their environment
• re-active (event based) - react to changes in their environment
• pro-active (goal based) - attempt to change the real world
• co-operative - co-operate to achieve their goals
• adaptive - learn from experience

Table 2: Characteristics of agents (source: [3])

The following table lists up all different categories of robots for all purposes:

Table 3: Categories of robot (source: www.botspot.com)

The wide-spread distribution and intelligent self-generation of the search mechanisms of agent technology makes the difference to other methods. That is why agents can search passive documents in the web or digital libraries and also adapt to active information systems front ends. However, it has to be stated, that agent technology up to now has not fulfilled the original promise, but is a powerful means with a huge potential.

Additionally, the application of multi-agent systems (system where agents communicate with other agents) requires a great deal of research work as there are not yet answers to the question of how they behave if their number increases.

2.3 (Distributed) databanks

Unlike the Internet which represents a practically unstructured data base, which means that search procedures are often rather similar to browsing, the contents of databanks are much better known, well-structured and, in addition, limited to certain subjects. This applies to central and distributed data bases architectures as well as the hierarchical and relational organisation of the data.

For this reason, the search procedure differs considerably from that used for browsing the web, as with robots and search engines. Search in distributed databases requires access to the data bank by an interface. Then a query has to be constructed using the methods provided by the database. Often there is the need to use a proprietary query language and also meta knowledge like a semantic model. In addition, a technical model of the data contents is needed for efficient exploration.

2.4 Information systems

Information systems rely on underlying databases. Unlike passive data bases, to which users are simply given access, but then are left to generate queries on their own, using meta knowledge, information systems support the user by providing intelligent calculation algorithms to search within their own data and compute optimal results on that basis. These are proprietary search mechanisms and calculation algorithms according to the structure of the database. The user must formulate a query by filling out the front end or interface completely, and with correct input. On the other hand, there is no possibility for users to create their own queries other than the ones offered.

In both databases and information systems, a well designed interface helps save time in the information retrieval process in contrast to browsing the web with search engines or robots.

3. IDENTIFICATION OF CUSTOMER NEEDS

The ability to provide individual services represents one of the key issues for the success of a service. Here, we are not concerned with market research procedures which try to find out what customers want to know, but with the mathematical and technical methods needed to identify customers information requirements when initiating a query, as this has impacts on the user interaction with the system.

Unlike data retrieval, which aims to collect all objects satisfying clearly defined conditions, the characterisation and identification of users' (information) needs is not a simple problem. Information retrieval usually deals with natural language, which is not always well structured and could be semantically ambiguous [2]. Additionally, there is often uncertainty in users' wishes themselves. To be effective in its attempt, the identification tool must somehow interpret the content of the information items in a collection, and rank them according to a degree of relevance to the user query. The difficulty is not only to extract the information, but to perform filtering functions and decide its relevance.

The following are the most appropriate methods for solving this task:

• Fuzzy logic for modelling similarity
• Modelling of users' decision problems using multi-attribute decision theory [4]

These general remarks are also valid for the task of information retrieval in the area of travel and tourism.

4. REVIEW OF INFORMATION RETRIEVAL FOR TRAVELLER SERVICES

4.1 Basic conditions and resulting problems

Value Added Service Providers (VASPs) have to provide exhaustive and detailed traffic and travel information, covering the users needs in the geographic area of interest.

Internet is the most popular channel since it is low-cost and easy to access, but as the information is not organised, it has to be retrieved using specific methods and involved accessing huge amounts of data, as stated above.

A service provider (VASP) considering how to provide traveller information in a conurbation area faces many problems. First of all, the VASP has to be authorised to access the data of each of the local data providers. This involves keeping an updated list of local transport service providers, as well as acquiring knowledge of the data structure adopted and access methods available. Even when a VASP possesses the full set of data required to meet the user's request, a good deal of further elaboration and information retrieval still has to be done. In fact, the available information is usually raw data (e.g. position of the bus in the network, flow status in the network) and specific knowledge and elaboration are required to work out the information desired (e.g. the expected bus-stop arrival time, congestion warnings, etc.). When multimodal trip planning is required this involves further complexity, since it requires additional information (such as interchange nodes), usually not available and difficult to retrieve. The difficulty of the problems encountered increases with the number of cities involved.

The organisational and technical problems involved are summarised below:

1. The VASP needs to be able to carry out complex processing of a huge amount of distributed data, usually represented by raw data on the local traffic and travel conditions.
2. Data providers for the same mode of transport may well provide non homogeneous kinds of information.
3. Data providers for different transport modes may provide incoherent reference networks, i.e. without clear identification of the interchange nodes required for the provision of intermodal services.
4. The VASP must have a continuously updated list of all the available data providers serving the area of interest, including all access methods and data formats.
5. The VASP needs to be confident of the quality and availability of the data collected from each data provider.

These difficulties represent a serious obstacle to the VASP in achieving its objective, i.e. supplying the user effectively with information of the level of detail required.

To overcome the problems listed above, the following are important requirements:

1. The presence of "engines" to reduce the degree of technical knowledge and amount of processing required by the VASP.
2. Adoption of open solutions, if possible using de-facto or emerging standards for data exchange.
3. Different modes of transport should adopt compatible location references to favour intermodal application.
4. Availability of a reliable and easy-to-access data catalogue.
5. Data providers must guarantee the quality and availability of the data provided, based on agreement and contract templates.

Figure 3 - The current situation

4.2 One solution: the telematics platform

One possible solution consists of setting up of open telematics platforms. These are systems (both hardware and software components) capable of providing a structured data pool accessible through a set of standard or de-facto interfaces with multiple physical connections. The platform is able to meet the requirements listed above, since it has the following features:

• A standardised and harmonised platform with clearly defined interfaces for accessing the data. Emerging and de facto standard are adopted following the state-of-the-art in the ITS field. This includes reference data structures and location referencing methods for the provision of the information. The overall result is a reduction in the information required as well as in the elaboration and data access time.
• Raw and elaborated data from all the local providers. The platform collects the traffic and travel data made available by the various local transport service providers. The raw data can be further processed to provide the VASP with the specific information required.
• Engines. The presence of engines capable of complex data processing (e.g. expected bus-stop time arrivals, delays on customised selected routes, predicted levels of car park occupation)
• A single agreement. A VASP wishing to access the platform data would need to sign only one agreement, the 'Platform-Client Agreement'.

4.3 Platform Clients

A platform client is an organisation/company interacting directly with the platform obtaining and/or providing data and information. Two main types of client can be identified:

1. The Data Provider who deals with tasks like:

• keeping the relevant sub-systems in operation
• collecting raw data
• providing processed data
• handling historic data
• implementing modelling

2. The Value Added Service Provider who provides value-added services based on the data retrieved from the platform, dealing with further tasks such as:

• presenting the content
• customising the information
• combining different sources
• handling user profiles
• managing communication media

4.4 The Platform Owners

The platform should be promoted and funded by the following actors:

• City Councils interested in the promotional aspect of providing the local authorities and citizens with a platform
• Data providers (e.g. public transport companies, traffic management organisations), interested in acquiring greater scope on the market through the platform.

4.5 Possible scenarios of use

The platform owner represents a local content provider who uses the platform to provide the information to interested VASPs, with agreements covering the economic, organisational and technical specifications.

The platform owner can also decide to act as a local VASP, with the provision of services based on the available data. In this case, other VASPs using the platform would run the risk of being bypassed. In fact, the traveller could decide to directly access the local travel information through the local VASP.

The VASPs themselves could be organised according to a hierarchical structure - local VASP, regional VASP, national VASP etc. - depending on the extent of the area of interest.

As the more important real-time features become, the more essential local and regional providers or platforms will become.

Mobility VASPs could also act as part of a portfolio of general service providers coming under the umbrella of a wider portal. Customers would possibly pay more willingly for travel services within the context of a complete service offer.

Transport operators are also increasingly likely to play the role of a VASP, with the tendency to associate with recognised travel brokers, as the importance of well-known brands cannot be underestimated in the web economy.

4.6 Key features

• The platform Management Team must include representatives from all the organisations (data sources) involved to ensure that the platform provides the required information without reflecting any interested bias from management members.
• The platform data sources must include the majority of the transport service providers in the area concerned to ensure that the information provided is exhaustive and user oriented. At the same time, this pushes transport service providers towards the platform so that they can avoid losing a good part of the market.
• A Co-ordination Force is essential. Its mission is to run the platform, specify updates to system components required, also in terms of accessing methods, and maintain clear documentation for the data access.
• The Co-ordination Force has the task of guaranteeing the level of quality of the information provided, as this will determine the quality of the Services provided by the platform.
• The Co-ordination Force also has to guarantee emerging, de facto or standards methods for accessing the information. If data providers adopt proprietary formats for the information, the platform has the task of converting these data, using open solutions which meet the requirements of external VASPs.

Figure 2 - A VASP with telematics platforms

4.7 A real-life example

A real-life example of implementation of this solution is TITOS (Torino ITS 2000 Open Showcase), the open platform which was set up in Turin in occasion of the 7th ITS World Congress. It is currently in operation and is being used to develop new ITS services for the city of Turin and give a strong impetus to the commercial deployment of ITS technology in Europe.

The platform provides its clients free of charge with a set of resources (data, communication channels and user terminals) used by organisations or companies wishing to promote their equipment and services by simply plugging into the platform.

4.8 ITS test-bed

The TITOS platform serves as a neutral demonstration environment for ITS operators. An important objective is to promote the integration and interoperability of ITS technologies, and to reinforce norms and standards at the European level. TITOS is supported by EC funding as part of the SMITH project and will continue to be operational as a "test bed" until the end of 2001. The platform provides the opportunity for testing and upgrading products or services. The necessary technical assistance for the Clients is provided free of charge.

The TITOS platform experience could be replicated in any city where traffic and travel information services are to be developed. To proceed, the key features of the platform have to be considered and analysed as reported in the following sections.

Figure 3 - Real life plug and play environment

4.9 Basic principles

As shown in Figure 4, the two main interfacing elements of the platform physical architecture are the Resources and the Clients.

TITOS Resources consist of data, communication channels or user terminals available through the platform. The Clients access the Resources using the TITOS Interfaces. The Interfaces allow the Clients to access the Resources using defined, emerging or de-facto standards. As a result, Clients can obtain data/information, provide data/information, provide services, demonstrate their equipment, exhibit and test their applications.

The TITOS platform is supported by a number of 'Actors' who make available their infrastructures to form the TITOS environment. The role of main Actor is played by 5T, which is committed to providing the basic traffic and transport real-time data for the whole transport network of Turin, and providing most of the Interfaces to access this data. Other Actors have already made available other important Resources (e.g. RAI for the FM and DAB transmitting infrastructure, ATM for public transport information and data, MIZAR Automazione for the WAP gateway). Other Actors include private companies contributing by enlarging the range of TITOS Resources available. This mechanism allows a constant evolution and updating of the platform features.

Figure 4 - Elements of the TITOS architecture

4.10 TITOS Resources

Resources are classified according to the following categories:

Data
The "background scenario" consists of the traffic and transport network of Turin. Detailed real time data is provided by 5T, the source of core data. This data relates to private and public transport over the whole Torino city transport network, including traffic flows, travel time, vehicle density, queue length, and arrival times (for public transport), etc. historical profiles, forecasts and targets are also available. There is, in addition, real time data on pollution levels and free spaces in city centre car parks. The platform is also fed with traffic and travel information from other data providers. This includes traffic flows on Italian motorways and the surrounding Trans European Network, as well as tourist information, flight and bus-link timetables for Turin airport. The TITOS data can then be split down into the following:

• Private transport data: Static and dynamic (real-time and historical) data concerning urban transport network traffic conditions.
• Public transport data: Static and dynamic (real-time and historical) data concerning public transport services.
• Location reference data: Location reference database to exchange data and information between systems. In fact, the efficient use of a wide range of data types and detail has required the development of an appropriate location referencing technique. TITOS has therefore made efforts to make data accessible using alternative location referencing methods. So far, these include: TMC codes (both national and regional), Geocodes, ILOC (Intersection Location Coding), LLOC (Link Location Coding) as well as the 5T reference network.
• Car-park data: Car park data.
• Environment data: Pollution condition data.
• General purpose data: Navigation maps, Turin tourist information and points of interest, differential GPS data.

Another classification of data is based on the update frequency of the information provided, so from this point of view data can be defined as static or dynamic.

Data can be provided as a consequence of a customised request i.e. a Client may require route guidance information which concerns specific city intersections. In this case, the answer is based on the results of dedicated processing engines.

The platform uses specific engines to provide information involving complex data processing. The TITOS engines are described in the following:

• Travel time on specified routes
  An on-line processing engine monitors a set of arcs including the paths of interest, which can be defined by the user, and updates the observed travel for the paths (sum of travel times of the arcs defining the path).
• Congestion/incident alarms
  An on-line processing engine monitors a set of arcs including the paths of interest, which can be defined by the user, and updates a list of paths suffering congestion and/or anomalies.
  The congestion warning is raised from the comparison between the observed travel time and the historical travel time for the path, considering also the presence of active events along the path.
• Virtual VMS
  The 5T system is able to provide VMS information for all the city intersections, regardless of the physical presence of a VMS panel or not. This the reason for speaking about Virtual VMS. More specifically, for each defined destination, the Virtual VMS gives the suggested exit arc from the arc where the variable message sign (virtual or real) is located. The suggested exit depends on the current control strategy for the network where available and for a best route to the destination when the control policy is not available.
  The output of the control strategy is the turning flow per destination. An on-line engine processes the desired and observed turning percentages and applies a control algorithm in order to work out the desired movement from the arc where the VMS is located for each destination.
• Parking availability
  This engine monitors the car parking space available. This information covers the car parks equipped with car sensors at the gates. On-street parking spaces in Turin are not equipped with car-sensors, so that this kind of information is not available.
• Dynamically updated public transport times
  The data provided is a mixture of observed, predicted and desired data. For each bus/tram-stop of the network there is the list of public transport services (in terms of vehicles running) and related time passages on a time window of one week. The normal data is the 'desired data' relating to the official timetable schedules. Moreover for a horizon of 30 minutes ahead, starting from the current time, the scheduled data are replaced with forecast data elaborated by the Automatic Vehicle Monitoring subsystem.
• Warnings on PT route disruptions
  Information on disruptions of public transport services is made available specifically to those directly affected because they are using the service in question at that moment. Warnings are given to travellers through the media which can be chosen by them.

Communication channels
These are the telecommunication chains needed to reach the final users via different media channels. The communication channels available through TITOS include SMS and WAP, available as information channels on the GSM network, a multimedia channel (DAB/ DMB) for data broadcasts and the TMC (Traffic Message Channel) for transmitting information via FM/RDS radio. Information is also accessible on TV channels (Televideo and Teletext) and, of course, the Internet, expected to be the channel most used by TITOS final users.

User Terminals
This is the equipment needed by the traveller, who is the final user of TITOS. The services available through the TITOS platform can be accessed through a wide range of user terminals, available either as commercial products or demonstrated by equipment suppliers as prototypes. Users will be able to access most services using mobile phones, home PCs, PTA, car navigation systems, car radio or TV. A number of prototypes are being demonstrated. These include new generation in-vehicle systems (currently known as on-board "infotainment" devices), which integrate the navigation function with mobile phones and displays with DAB receivers, offering interactive information services.

4.11 TITOS Clients

The TITOS platform is designed to be used by:

• Data providers, who make information available through the platform. They are represented by clients providing different kinds of static and/or dynamic transport data using defined, emerging or de-facto standards, e.g. a fleet management system providing floating car data or a DATEX node providing information on the current traffic conditions.
• Value-added service providers, who have access to data and communication infrastructures for demonstrating their services, e.g. navigation systems, real time traveller information for public transport. The scenarios for VASPs connected to the platform may be the following:
  • Provision of value added services, based on the transport data/information accessible through TITOS, e.g. a company which provides an intermodal trip planning service to satisfy the final user who wants to reach the destination via both public transport (bus, trams) and private vehicle (taxi, car). In this case, the basic information needed is obtained from TITOS, but this data has to be further processed to provide the specific value added service.
  • Provision of other services via TITOS communication channels, using the communication channels provided by TITOS to reach the Final User, e.g. a company which provides its traffic information service through the GSM operator or the DAB communication channels, both available in TITOS.
  • Provision of value added services via TITOS communication channels. This scenario consists of a combination of the previous two. The service provider obtains information through TITOS to provide a value added service to the Final User via the communication channels, again available through TITOS. An example is a company using the TITOS WAP communication channel to provide a multimodal trip planning service, consisting of the optimal route to reach a destination based on the current traffic conditions, available from TITOS, and various customisation criteria.
• Equipment suppliers, who have access to services for demonstrating products, e.g. traveller information panels, mobile information terminals. They are represented by clients using TITOS to demonstrate the capability of their equipment in interfacing with TITOS Resources or to deliver services to the users. E.g. a DATEX node manufacturer which demonstrates the correct interfacing of its node with the node available in TITOS or a GSM mobile phone manufacture with WAP technology or a DAB receiver manufacturer.

The basic traveller assistance services of the platform are freely accessible to the general public in Turin. A focus group including a sample of average people was set up to obtain feedback and user reactions.

4.12 TITOS Services

For users registered with TITOS, it is possible to have access to both on-event and on-demand information Services. These Services include:

• TMC services: providing traffic messages via FM/RDS and DAB.
• Mobility warnings: customised real-time information on congestion available to travellers via GSM/SMS, e-mail, etc.
• WAP traveller information: real-time mobility information via mobile phones.
• Real time information on traffic status via DAB, Internet, and Televideo.
• Trip planning and navigation: based on actual traffic conditions and multimodal links.
• Public transport information: real-time information about public transport services.

In some cases, similar services are provided by different providers, allowing users to test and compare the various features. The strength of TITOS resides in the fact that it is not simply an academic exercise, but a real life environment where solutions can be tuned and finalised after confronting practical implementation problems.

4.13 Evaluation of the platform

The TITOS platform has undergone an evaluation process. As well as tests of the physical functioning of the system through built-in evaluation software modules, questionnaires were distributed to evaluate the user acceptance of the platform. The first impressions which emerged from the TITOS Forum held at the 7th World Congress in November 2000 identified the following:

• the need to complete the technical work with studies to identify and propose the organisation and institutional issues necessary to transform the services demonstrated into commercial services
• the importance of the Internet connection in accessing the available data. The following table shows a list of the TITOS Interface and Physical Connections available to access them. Considering the considerable interest in using the Internet channel TITOS has planned to make available also the DATEX connection over the Internet.

• the need to transform TITOS into a permanent ITS test-bed (beyond the end of 2001) with the support and commitment of the municipality.

REFERENCES

[1] Traveller Information - Functional specifications, system architecture issues, 2nd Intermodality Area Workshop, CODE project, Aix-en-Provence, 1999.

[2] Ricardo Baeza-Yates and Berthier Ribeiro-Neto: Modern Information Retrieval, Addison Wesley, ACM press, 1999.

[3] Josef Withalm: Agents solving strategic problems in tourism ENTER 2000, Barcelona, April 2000

[4] T.Kämpke, F.J.Radermacher, P.Wolf: Supporting preference elicitation - The FAW preference elicitation tool. Decision Support Systems,Vol. 9, No.4, 1993.

[5] TITOS Technical Support: http://www.torino2000.itscongress.org/showcase/technica.htm.

[6] TITOS Services home page: www.5t-titos.it.

[7] eEurope Initiative home page: http://europa.eu.int/information_society/eeurope/index_en.htm