iCommerce.com Corporation
eCommerce


Search our
entire site
Enter your search
terms below, or visit
our search page



Search
architecture only
Enter your search
terms below:




For the table
of contents and
hyperlinks to
general topics
proceed to toc


























Scaling Object Service Architectures to the Internet

FINAL REPORT

September 28, 1998  

Executive Summary

Web technology and distributed object technology need to be better integrated to realize the benefits of both. The Web is already moving beyond documents to provide distributed applications.  Without an architectural framework, the web environment will build services parallel to but different from the object middleware community, and enterprise architects will be forced to make idiosyncratic tradeoffs in constructing systems that need to use both architectures together.

The overall thesis of our project is that an Internet Services Architecture (ISA) can help to unify both Web and ORB architectures.  During the life of the project, we developed the broad outline for a general ISA architecture and explored its utility in the virtual office and command and control domains.  We then focused on an instantiation of the general ISA which we call an Intermediary Architecture (IA), which provides a new family of ways to add middleware services into Web architectures by inserting component middleware between Web client and server.  We constructed a series of prototypes to demonstrate parts of the IA architecture.

Background

Object Services Architectures (OSAs) are distributed software backplanes with plug-in component object services.  OSAs promise vast improvements over conventional stovepipe software architectures in reducing software life cycle costs and providing modular software design. The Object Management Group (OMG) Object Management Architecture (OMA) (partly based on our previous DARPA research) is an important OSA because it is backed by a consortium of over 800 companies, is developed by an open process, and is being adopted into large-scale industry and DoD enterprise-critical systems and standards profiles including DoD AITS-JTF/ATD, NIMA, DMSO HLA, and DII COE.

Problem Statement

Today the Web consists of thin clients, which provide universal browser interfaces, and web servers, which provide back-end access to global information resources (that's where the data is).  Meanwhile OSAs provide access to suites of middleware services (that's where the system management and application extensibility mechanisms are).  As important as OSAs are becoming, they are a new kind of software architecture and industry still lacks theory and experience in scaling OSA systems to levels to support large-scale, world wide operations like command and control.  At present, OSAs are not well connected to control access to web data sources.  Without better integration, the web community will build protocol extensions (e.g., W3C WEBDAV) parallel to but different than the OSA community, and enterprise architects will be forced to make idiosyncratic tradeoffs in constructing systems that need to use both architectures together.   Furthermore, even if we can unify these two sorts of architectures, there is still a question of what new Internet services might be useful that are not already identified by OMG, how to compose these, and how to federate them in large WAN environments (many of these questions are still unanswered for pure ORB architectures).  Another problem is the kind of object model that is appropriate for both Web and ORB.  Finally, there is the problem of how enterprises can deal with the incredibly complex Internet environment where technologies change daily.

Objective

There is a family of architectures for composing ORBs and the Web.  Together, these architectures open up a wide new spectrum of services and facilities that can enable a broad class of future architectures.  We call this class of architectures an Internet Services Architecture (ISA) by analogy with the OMG Object Services Architecture.  The overall thesis of our project is that an Internet Services Architecture can help to unify both Web and ORB architectures. Some approaches for instantiating ISA architectures are already known (developed over the last three years):  e.g., the three tier architecture where Web servers access ORBs via CGI scripts; the approach pioneered by Visigenic and Netscape of downloading an ORB client via an applet; and the recent W3C HTTP-NG project where HTTP is re-implemented on a distributed object backplane.  Our project focused on another variant ISA that appears to be broadly useful, which we called an Intermediary Architecture because it inserts middleware services between Web client and server.

 

Technical Accomplishments

In this section, we divide our main technical accomplishments into three layers of description corresponding roughly to the three years of the contract.

Internet Tools Survey and the Virtual Office

In FY96 we organized Object Services and Consulting, Inc. (OBJS) as a virtual office (distributed office), dependent on Internet, Web, database, distributed object, and related infrastructure technologies.  Our main focus that year was on identifying, describing, and installing a large family of software that constitutes the core of Internet, Web, and distributed object technology for virtual enterprise computing.  A main contribution that year was the Internet Tools Survey, described in the Technology Transition section below.  The Internet Tools Survey provided a detailed look at many technologies needed to Internet-enable applications but that so far did not clearly fit into an overarching software architecture and were then beyond the realm of OMG middleware concern.  To change that situation, we began to move OMG in the direction of a rendezvous with Internet and Web technologies by co-chairing the OMG Internet Special Interest Group.

Internet Services Architecture (ISA)

In FY97 we worked with OMG to develop a high level industry roadmap for an Internet Services Architecture to unify both Web and ORB architectures.  The ISA consists of a suite of possible standards that OMG might adopt over a several year period to make Web-ORB integration easier.   The ISA extends the OMG OMA and draws on the strengths of both OSA and Web technologies. OSAs can provide the web with a principled model for composing object services, federating like services with possibly varying policies and implementations, combining these into systems of systems, and controlling binding times of these services so new services can be added to existing systems. Both the OMG and Web communities have sufficient momentum that neither can change instantaneously, making an evolutionary approach the only one with a likelihood of success.

Intermediary Architecture (IA)

In FY97 and FY98 our work focused on a specific instantiation of the Internet Services Architecture that we call an Intermediary Architecture.  The IA splices ORB (or DCOM or other) componentware services in between Web clients and servers to provide a management layer that extensibly allows many new kinds of services to be plugged in.

The Intermediary Architecture is open and flexible and can be used for many purposes including security, versioning, internationalization, query augmentors, shielded pages, rating systems, rerouting, filtering, logging, system management instrumentation, clocking, micropayments, disconnected, intermittent access, tracking URL usage patterns of a community of interest, background indexing of pages you have seen before so you can find them again, indexing streams of audio or video, translingual translation service, closed caption in multiple languages, and augmenting a web page with voice access to URLs.  This makes the Intermediary Architecture an excellent candidate integration harness for testing other DARPA technologies.

Advantages of the Intermediary Architecture are:

  • it provides a new kind of plug-in and an improved way for third-parties to build extensible web applications;
  • it is non-intrusive requiring no change to existing Web infrastructure*;
    • * we did not find this to be 100% true.  See below.
  • it generalizes many special-purpose efforts to add specific services to Web applications;
  • it provides a general framework for reasoning about how to federate and scale Web applications.

      The Intermediary Architecture is shown in the above figure.  In its simplest form, it consists of:

  • an intermediary, also known as an interceptor, that inserts (and composes) service behaviors
  • one or more service behaviors
  • metadata associated with services (stored in a persistent metadata repository) as well as policy management controls
  • a management function for querying, viewing and manipulating metadata and controls

 The architecture can appear in many variations:

  • there are many kinds of services that can be inserted via an intermediary -- see the list above.
  • there can be more than one intermediary and different roles can control different intermediaries.  For instance, different groups in an organization could manage security, caching, filtering, system management and logging, functions like annotation, the ability to add new functions, etc.
  • intermediaries can be implemented as adjunct to the client or to the server or somewhere in between.  We can talk about client intermediaries, server intermediaries, and proxy intermediaries.
  • more than one service can be inserted via intermediaries.  Services can compose in series or in parallel.  In general, the area of service composition needs more work since there are many ways to wire up even two services (see lessons learned on the Network Performance Management project where it is possible to capture performance for just user-specified URLs or all GIFs as well).
  • services can be inserted dynamically at run-time.  One way to do this is via a Trader that provides service discovery.  The trader can do this once at the beginning or can be continuously polling to locate new or better service providers or to make the system more robust if some providers fail.
  • a given service can store metadata it collects in some repository.  If local to the Web client, the service might provide personal services (e.g., personal Web annotations -- see the Annotation Service below).  If shared in a group, the service can provide group services (e.g., group annotations).  If shared within a community or on the server it can provide public or community services.
  • repositories could be federated together to allow more sharing.
  • repositories can be different for different services or they could be shared across services (e.g., so that both the performance monitor and the annotations services could share one repository).
  • it is important to be able to view and query the metadata.
  • federation can appear in many places within this architecture:  individual services can be federated, repositories can be federated.

The overall claim is that the Intermediary Architecture provides a framework for service insertion, composition, and federation.  We do not claim to have thoroughly explored this space.  There is still much we do not understand though limited prototyping has taught us a fair amount about the Intermediary Architecture.

To learn about the properties of the Intermediary Architecture, we completed a series of separate prototypes of parts of the architecture (each is separately described):

  • Intermediary Architecture Interceptor- this mechanism is installed at insertion points to provide a way to plug object service components and before-after filters into the communication path between Web client and Web server to enable the installation of new services.
  • Web Object Model - this project examined how XML, DOM, PIC-NG, Dublin Core, LORE, OMG IDL, Java, and other information and metadata representations can add object model capabilities to the web. An associated prototype is XML to Java Translator - this mechanism provides an easy way to deserialize XML DTD schemas into Java classes and is likely to be widely useful stand-alone.
  • Annotation Service - this prototype shows how to use the Intermediary Architecture to plug in a service that downloads annotations on a document, providing a way for third parties to augment web content.  Different architectures for the service could yield private, group, or public annotations and voting or rating mechanisms.
  • Personal Network Performance Monitor Service - this project shows how to use the Intermediary Architecture to provide network performance and trend data when accessing remote sites.  This data can be used as a basis for QoS negotiation in higher level tools.
  • NLI Query Interface - this project focused on how to use an existing natural language query system to access metadata stored in IA metadata repositories.
  • Trader Service - this project provides two matchmaking services for service discovery.  One implementation is scalable and based on Internet search engine; the other shows how to extend an OMG trader to locate Web resources.

Prototyping involved reuse of a number of state-of-the-art commercial and experimental tools including ORB systems, Web systems like W3C Jigsaw, object DBMSs, and query systems.

Technology Transition Accomplishments

Lessons we learned about the Internet Services Architecture and the Intermediary Architecture were fed back to industry and DoD through our active participation in architecture initiatives, principally the Object Management Group (OMG), the World Wide Web Consortium (W3C), DARPA/ISO Advanced Information Technology Services (AITS) , DARPA Dynamic Database Panel II, the National Industrial Information Infrastructure Protocols (NIIIP) Consortium, MCC Object Infrastructure Project (OIP), and X3H7/ODP as described below.

The following reports were completed under this contract.  All appear on the OBJS public web page to better disseminate the results.  See OBJS Technical Reports and Publications.   Several were presented in workshops, conferences, or journals or at standards meetings where they were used to build consensus for an Internet Services Architecture.

Virtual Office and DoD C4I and Doctrine Application Domains

We identified two broad domains in which to develop application scenarios that would use ISA technologies:  virtual office and command and control.  Both domains require technology support for collaboration-at-a-distance.  We organized Object Services and Consulting, Inc. as a fully geographically distributed virtual office environment, making us our own testbed for collaborationware.   We documented virtual office and collaboration technology requirements and "lessons learned" in provisioning our own virtual office domain. We identified DoD doctrine as a good domain for future application scenarios in command and control. We are not aware of much DARPA work in this area though DoD has spent a substantial effort working to document its doctrine-related practices.

Internet Tools Survey

Defense Information Infrastructure Common Operating Environment (DII COE) is a suite of standards that DoD enterprises build on, but it does not contain as much information on Internet technologies as is needed to Internet-enable future DoD applications.  Mostly in the first year of our contract, we assembled a complementary Internet Tools Survey to provide a description of key technologies we feel must be understood to Internet-enable enterprises, virtual offices, and command and control. New additions after the first year include work on web object models, web-ORB integration architectures, and traders.

Publications

We published the following papers:

Workshops

We organized two industry workshops:

Standards

DARPA Reviews

  • We participated in annual IC&V and I*3/BADD PI meetings and project reviews throughout the life of the project.

Related Technology Transfer Projects.  We provided OSA componentware architecture expertise to DARPA and industry research programs.

Conclusion

We succeeded in advancing the state-of-the-art and the state-of-practice in several ways during the life of the project.  Our main contributions are:

  • We developed a new software architecture for Web-ORB integration which we call an Intermediary Architecture because it inserts middleware services between Web client and server.   We extensively reviewed the Web-ORB integration architecture literature and, until recently, found little other work on extending the web with intermediaries though there is prior work on Web proxies for use in firewalls and for caching and ORB interceptors for security.  We developed prototypes to show how useful this intermediary architecture is and that it provides a generally useful approach for connecting Web and ORB architectures.  Lessons learned include the fact that today's web clients are not open enough to directly support this architecture but a universal intermediary could provide this openness in a portable manner.
  • The particular prototypes we developed (interceptors, annotations, network performance monitoring, query interface, trader) and the web object model all provide insights into the general architecture and at the same time are interesting and useful in their own right.
  • Our work on virtual office scenarios is one of the best documented examples of how to use Internet-based technologies to build small business virtual offices.
  • Our architecture and technology transfer activities are making a direct impact on OMG Object Management Architecture and OMG's platform technology roadmap as well as directly benefiting a number of DARPA and industry initiatives: DARPA AITS (ISO) Architecture, NIIIP, and MCC OIP.
Architecture
Home
Distributed
IA Definition
Java Based
Mismatch
Network Monitor
Scaling Objects
Trader-Based
Web annotations