WO2004075466A2 - Semantic knowledge retrieval management and presentation - Google Patents

Semantic knowledge retrieval management and presentation Download PDF

Info

Publication number
WO2004075466A2
WO2004075466A2 PCT/US2004/004674 US2004004674W WO2004075466A2 WO 2004075466 A2 WO2004075466 A2 WO 2004075466A2 US 2004004674 W US2004004674 W US 2004004674W WO 2004075466 A2 WO2004075466 A2 WO 2004075466A2
Authority
WO
WIPO (PCT)
Prior art keywords
semantic
user
request
information
smart
Prior art date
Application number
PCT/US2004/004674
Other languages
French (fr)
Other versions
WO2004075466A3 (en
Inventor
Nosa Omoigui
Original Assignee
Nervana, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to CA002555280A priority Critical patent/CA2555280A1/en
Priority to AU2004213986A priority patent/AU2004213986A1/en
Priority to EA200501304A priority patent/EA200501304A1/en
Priority to MXPA05008670A priority patent/MXPA05008670A/en
Priority to BR0407451-3A priority patent/BRPI0407451A/en
Priority to JP2006503648A priority patent/JP2006522388A/en
Application filed by Nervana, Inc. filed Critical Nervana, Inc.
Priority to EP04711863A priority patent/EP1599811A4/en
Publication of WO2004075466A2 publication Critical patent/WO2004075466A2/en
Publication of WO2004075466A3 publication Critical patent/WO2004075466A3/en
Priority to US11/931,793 priority patent/US20080147788A1/en
Priority to US12/358,224 priority patent/US20100070448A1/en
Priority to US13/168,785 priority patent/US20120191716A1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/30—Definitions, standards or architectural aspects of layered protocol stacks
    • H04L69/32—Architecture of open systems interconnection [OSI] 7-layer type protocol stacks, e.g. the interfaces between the data link level and the physical level
    • H04L69/322—Intralayer communication protocols among peer entities or protocol data unit [PDU] definitions
    • H04L69/329—Intralayer communication protocols among peer entities or protocol data unit [PDU] definitions in the application layer [OSI layer 7]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/951—Indexing; Web crawling techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/40—Data acquisition and logging

Definitions

  • This invention relates generally to computers and, more specifically, to information management and research systems.
  • the present invention is directed in part to a semantically integrated knowledge retrieval, management, delivery and presentation system, as is more fully described in my co-pending parent application (U.S. Application Serial No. 10/179,651 filed June 24, 2002).
  • the present invention and system includes several additional improved features, enhancements and/or properties, including, without limitation, Entities, Profiles and Semantic Threads, as are more fully described in the Detailed Description below.
  • FIGURE 1 is a partial screenshot overview and FIGURE 2 is an expansion of a dialog box of FIGURE 1 for a scenario of a Patent Examiner using the preferred embodiment in a prior art search, a screenshot of where "Magnetic Resonance imaging" occurs in a Pharmaceuticals taxonomy.
  • FIGURE 3 shows the Sharable Smart Request System Interaction, which is the binary document format that encapsulates the SQML buffer with the smart request and also illustrates how the extension handler opens a document.
  • FIGURE 4A is a partial screenshot overview of document files.
  • FIGURE 4B shows an illustration of two .REQ documents from FIGURE 4A (titled 'Headlines on Reuters Related to My Research Report (Live)' and 'Headlines on Reuters (as of January 21 2003, 08 17AM)' on the far right) with a registered association in the Windows shell.
  • FIGURE 5 is a Diagram Illustrating the Text-to-Speech Object Skin and shows an illustration of an email message being rendered via a text-to-speech object skin.
  • FIGURE 6 is a Diagram Illustrating a Text-to-Speech Request Skin.
  • FIGURE 7 is a Diagram Illustrating Knowledge Modeling for a Pharmaceuticals Company Example.
  • FIGURE 8 is a Diagram Illustrating Client Component Integration and Interaction Workflow.
  • FIGURES 9 - 11 show three different views of the Explore Categories dialog box.
  • FIGURES 12 and 13 show sample screenshots of the Dossier Smart Lens in operation.
  • FIGURE 14 shows how the server-side semantic query processor processes incoming semantic queries (represented as SQML).
  • FIGURE 15 illustrates the semantic browser showing two profiles (the default profile named "My Profile” and a profile named “Patents”). Observe how the user is able to navigate his/her knowledge worlds via both profiles without interference.
  • FIGURE 16A-C illustrate how a user would configure a profile (to create a profile, the user will use the "Create Profile Wizard" and the profile can then be modified via a property sheet as shown in other Figures).
  • FIGURE 17 shows how a user would select a profile when creating a request with the "Create Request Wizard.”
  • FIGURE 18 shows a screenshot with the 'Smart Styles' Dialog Box illustrating some of the foregoing operations and features.
  • FIGURE 19 illustrates the "Smart Request Watch” Dialog Box.
  • FIGURE 20 illustrates a Watch Window displaying Filtered Smart Requests (e.g., Headlines on Wireless).
  • Figure 20 is an Illustration of the Watch Window with a Current Smart Request Title (e.g., "Breaking News").
  • FIGURE 21 illustrates Entity views displayed in the Semantic Browser.
  • FIGURE 22A and 22B show the UI for the Knowledge Community Subscription.
  • FIGURE 23 illustrates a semantic thread object and its semantic links.
  • FIGURES 24 through 46B are additional screen shots further illustrating the functions, options and operations as described in the Detailed Description.
  • FIGURE 47 as a sample semantic image for Pharmaceuticals/Biotech industry (DNA helix).
  • FIGURE 48 is an illustration of a semantically appropriate image visualization for the Breaking News context template.
  • FIGURE 49 is a Visualization — Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc. (Headlines).
  • FIGURE 50 is a Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc. (Two people working at a desk).
  • FIGURE 51 illustrates a semantic "Newsmaker” Visualization or Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
  • FIGURE 52 illustrates a semantic "Upcoming Events” Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
  • FIGURE 53 is a Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc. (Petri Dish).
  • FIGURE 54 illustrates a semantic "History” Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
  • FIGURE 55 illustrates a semantic Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc. (Spacecraft).
  • FIGURE 56 illustrates a "Best Buys" Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
  • FIGURE 57 illustrates a semantic Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc. (Coffee).
  • FIGURE 58 illustrates a semantically appropriate Sample Image for "Classics" for smart hourglass, interstitial page, transition effects, background chrome, etc. (Car).
  • FIGURE 59 illustrates a semantically appropriate "Recommendation" Visualization - Sample Image for the contextual/application elements of smart hourglass, interstitial page, transition effects, background chrome, etc. (Thumbs up).
  • FIGURE 60 illustrates a semantic "Today” Visualization - Sample Image for the elements smart hourglass, interstitial page, transition effects, background chrome, etc.
  • FIGURE 61 illustrates a semantic "Annotated Items” Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
  • FIGURE 62 illustrates a semantic "Annotations” Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
  • FIGURE 63 illustrates a semantic "Experts" Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
  • FIGURE 64 illustrates a semantic "Places" Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
  • FIGURE 65 illustrates a semantic "Blenders” Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
  • FIGURES 66 through 84 illustrate semantic Visualizations for the following Information Object Types, respectively: Documents, Books, Magazines, Presentations, Resumes, Spreadsheets, Text, Web pages, White Papers, Email, Email Annotations, Email Distribution Lists, Events, Meetings, Multimedia, Online Courses, People, Customers, and Users.
  • Figure 85 illustrates a semantic "Timeline" Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc..
  • RWLs Request Watch Lists
  • RWGs Groups
  • the Examiner could conduct a search, and with no additional effort or time as presently invested, obtain search results relevant to patentability even if they did not contain a single word in common with the key words chosen by the Examiner. Therefore, the system would obtain results relevant to the Examiner's task that would not ordinarily be located by present systems because it can locate references on the basis of meaning.
  • the Examiner is assigned to examine an application directed to computer software for a more accurate method of interpreting magnetic resonance data and thereby generating more accurate diagnostic images.
  • the Examiner would: a. Using the Create Entity wizard, create a "Topic" entity with the relevant categories in the various contexts in which "Magnetic Resonance Imaging” occurs. As an illustration, Figures 1 and 2 show where "Magnetic Resonance Imaging” occurs in a Pharmaceuticals taxonomy. Notice that there are several contexts in which the category occurs. Add the relevant categories to the entity and apply the "OR" operation.
  • the profile is preferably configured to include the "Patent Database” knowledge community. This launches a new
  • the request can be created by using the Create Request Wizard. To do this, select the Dossier context template and select the "Patent Database" knowledge community as the knowledge source for the request.
  • the wizard intelligently suggests a name for the request based on the semantics of the request.
  • the wizard also selects the right default predicates based on the semantics of the "Magnetic Resonance Imaging" "Topic” entity. Because the wizard knows the entity is a "Topic,” it selects the right entities that make sense in the right contexts. Hit Finish.
  • the wizard compiles the query, sends the SQML to the KISes in the selected profile, and then displays the results. hi the foregoing example, the results could be drawn, ultimately, from any source.
  • the results would have originated on the Web, some on the PTO intranet, some on other perhaps proprietary extranets.
  • the system by use of the system they have been automatically processed, and automatically "read” and “understood” by the system, so that when the Examiner's query was initiated, and also "read” and "understood” semantically, and by context, the system locates all relevant, and only relevant results. Again, not perfectly, but radically more accurately than in any prior systems. Note also that the system does not depend on any manual tagging or categorization of the documents in advance.
  • the Examiner may also wish to use additional features of the preferred embodiment of the invention.
  • the Examiner may wish to consult experts within the PTO, or literature by experts outside the PTO, as follows (note that Experts in Magnetic Resonance Imaging would be included in the Dossier on Magnetic Resonance Imaging; however, the examiner might want to create a separate request for Experts in order to track it separately, save it as a "request document," email it to colleagues, etc.). Find all Experts in Magnetic Resonance Imaging: a. Follow steps 1-4 above. b.
  • Biotech companies are research intensive, not only in laboratory research, but in research of the results of research by others, both within and outside of their own companies.
  • the research tools available to such companies have shortcomings.
  • Proprietary services provide context-sensitive and useful results, but those services themselves have inferior tools, and thus rely heavily on indexing and human effort, and subscriptions to expensive specialized journals, and as consequence are very expensive and not as accurate as the present system.
  • biotech researchers can search inexpensively using Google D, but it shares all the key word based limitations described above. i contrast, using the search features of the preferred embodiment of the present invention, a biotech researcher could more efficiently locate more relevant results. Specifically, the researcher might use the system as follows.
  • the researchers may wish to Find all Experts in Marketing or Research: a.
  • b. Drag and drop the "Marketing or Research” entity to the Experts (special agent or default knowledge request) icon in the desired profile. This launches a new request/agent appropriately titled “Experts in Marketing or Research.”
  • the semantic browser selects the right default predicate "in” because it "knows” the entity is a "Team” entity and the context template is a "People” template (Experts).
  • the default predicate is selected based on the intersection of these two arguments ("in") since this is what makes sense.
  • Dossier On Marketing or Research, as follows: a. Follow steps 1-4 above. b. Drag and drop the "Marketing or Research" entity to the Dossier (special agent or default knowledge request) icon in the desired profile. This launches a new Dossier request/agent that displays each special agent (context template). Each special agent is displayed with the right default predicate as follows:
  • the researchers may be interested in Finding "Breaking News on my Competitors", and would do so as follows: a. For each competitor, create a new "competitor" entity (under “companies") using the Create Entity Wizard. Select the right filters as needed. For instance, a competitor with a well-known English name - like "Groove” should have an entity that includes categories in which the company does business and also the keyword. b. Using the Create Entity Wizard, create a portfolio (entity collection) and add all the competitor entities you created in step a. Name the entity collection "My Competitors.” c. Using the Create Request Wizard, select the Breaking News context template and add the portfolio (entity collection) you created in step b. as a filter.
  • the semantic browser will now display a watch pane (e.g., a ticker) showing "Breaking News on My Competitors.”
  • a watch pane e.g., a ticker
  • the researchers may wish to keep records of competitors for future reference, and to have them constantly updated.
  • the system will create and update such records, by the researchers instructing the system to Show a collection of Dossiers on each of our competitors, as follows: a. Create entities for each of your competitors as described in 4a. above. b. For each competitor entity, create a new Dossier on that competitor by dragging the entity to the Dossier icon for the desired profile - this creates a Dossier on the competitor. c. Using the Create Request Wizard, create a new request collection (blender) and add each of the Dossier requests created in step b. above to the collection (you can also drag and drop requests to the collection after it has been created in order to further populate the collection). Hit Next — the wizard intelligently suggests a name for the request collection. Hit Finish. The wizard launches a request collection that contains the individual Dossiers. You can then add the request collection as a favorite and open it everyday to get rich, contextual competitive intelligence.
  • the researchers may wish to review a particular dossier, and can do so by instructing the system to Show a Dossier on the CEO (e.g., named John Smith): a. Using the wizard, launch an information-type request/agent for People with the keywords "John Smith”. b. Select the result and click "Save as Entity” - this saves the object as a "Person” entity (because the semantic browser "knows” the original object is a person - as such, a "Person” entity makes sense in this context). c. Using the Create Request Wizard, select the Dossier context template, and then select the "John Smith” entity as a filter. Hit Next - the wizard intelligently suggests a name for the request based on -the semantics of the request.
  • the wizard also selects the right default predicates based on the semantics of the "John Smith" person entity. Hit Finish.
  • the wizard compiles the query, sends the SQML to the KISes in the selected profile, and then displays the results (as sub-queries/agents) as follows:
  • the Smart Selection Lens is similar to the Smart Lens feature of the Information Nervous System information medium, hi this case, the user can select text within the object and the lens will be applied using the selected text as the object (dynamically generating new "images" as the selection changes). This way, the user can "lens” over a configurable subset of the object metadata, as opposed to being constrained to "lens” over either the entire object or nothing at all.
  • This method gets the SQML from the clipboard, takes the argument SRML for the object, and dynamically creates new SQML that includes the resource in the SRML as a link in the SQML (with the default predicate "relevant to”). The method then returns the new SQML. The Presenter then calls the method:
  • This method passes the generated lens SQML and then retrieves the number of items in the results and the SRML results, preferably asynchronously. For details on this call, see the specification "Information Nervous System Semantic Runtime OCX.” The Presenter then displays a preview window (or the equivalent, based on the current skin) with something like:
  • the timer is preferably reset (this will avoid flashing the window when the user moves the mouse around the same area).
  • outstanding lens query requests (which are regular SQML queries, albeit with SQML dynamically generated consistent with the agent lens) should be cancelled when the Presenter no longer needs them (e.g. if the Presenter is navigating to a new page, or if we are requesting new lens info for an object).
  • such cancellation is not critical from a performance (or bandwidth) standpoint because lens queries will likely only ask for a few objects at a time. Even if the queries are not cancelled, the Presenter can ignore the results. Regardless, because the Presenter also has to deal with stale results, dropping them on the floor -the Presenter will have to do this anyway (whether or not lens queries are also cancelled).
  • the preferred embodiment has asynchronous cancellation implementations - the software component has been designed to always be prepared to ignore bad or stale results.
  • the Presenter preferably has both icons (indicating the current lens request state) and tool-tips:
  • the Presenter can put up a tooltip with the words, "Requesting Lens Info” (or words to that effect).
  • hovering will show the "Found 23 Objects” tip and clicking will show the results.
  • This interstitial tool tip can then be transitioned to the preview window if it is still up when the results arrive.
  • the smart selection lens can be applied to objects other than textual metadata.
  • the Smart Selection Lens can be applied to images, video, a section of an audio stream, or other metadata.
  • the Presenter would return the appropriate SRML consistent with the data type and the "selection region.” This region could be an area of an image, or video, a time span in an audio stream, etc.
  • the rest of the smart lens functionality would apply as described above, with the appropriate SQML being generated based on the SRML (which in turn is based on the schema for the data type under the lens).
  • the Information Nervous System (which, again, is one of our current shorthand names for certain aspects of our presently preferred embodiments) also supports the drag and drop or copy and paste of 'Person' objects (People, Users, Customers, etc.).
  • 'Person' objects People, Users, Customers, etc.
  • the server's semantic query processor merely resolves the SQML from the client using the Person as the argument. For instance, if the user pastes (or drags and drops) a person 'Joe' on top of a smart request 'Headlines on Reuters,' the client will create a new smart request using the additional argument.
  • the Reuters Information Nervous System Web service will then resolve this request by returning all Headlines published or annotated by 'Joe.' In this case, the server will essentially apply the proper default predicate ('published or annotated by') - that makes sense for the scenario.
  • the Reuters Web service (in the preferred embodiment) must return Headlines that are "relevant to Joe's expertise.” This embodiment would then require that the client take a two- pass approach before sending the SQML to the destination Web service. First, it must ask the Knowledge community that the person belongs to for "representative data (SRML)" that represents the person's expertise.
  • SRML representationative data
  • the client queries the Knowledge community from whence the person came for several objects that belong to categories on which the person is an expert, hi the preferred embodiment, the implementation should pick a high enough number of objects that accurately represent the categories of expertise (this number is preferably picked based on experimentation).
  • the reason for picking objects in this case is that the destination Web service will not understand the categories of the Knowledge community from whence the person came and as such will not be able to map them to its own categories.
  • a category mapper can be employed (via a centralized Web service on the Internet) that maps categories between different Knowledge communities, i this case, the destination Knowledge community will always be passed categories as part of the SQML, even though it does not understand those categories - the Knowledge community will then map these categories to internal categories using the category mapper Web service.
  • the category mapper Web service will have methods for resolving categories as well as methods for publishing category mappings.
  • a sharable smart request document is a binary document that encapsulates SQML (via a secure stream in the binary format). It provides a safe, serialized representation of a semantic query that, among other features, can protect the integrity and help protect the intellectual property of the specification.
  • the query itself may embody trade secrets of the researcher's employer, which, if exposed, could enable a competitor to reverse engineer critical competitive information to the detriment of the company.
  • the protection can be accomplished in several ways, including by strongly encrypting the XML version of the semantic query (the SQML) or via a strong one-way hash.
  • the sharable document has an extension (.REQ) that represents the request.
  • An extension handler on the client operating system is installed to represent this extension. When a document with the extension is opened, the extension handler is invoked to open the document.
  • the extension handler opens the document by extracting the SQML from the secure stream, and then creating a smart request in the semantic namespace with the SQML.
  • the handler then opens the smart request in the semantic namespace.
  • the client serializes the SQML representing the smart request in the binary .REQ format and saves it at the requested directory path or opens the email client with the .REQ document as an attachment.
  • Figure 3 shows the binary document format that encapsulates the SQML buffer with the smart request and also illustrates how the extension handler opens the document.
  • a similar model can also be employed for sharing results (via SRML). hi this case, a binary document encapsulates the SRML, rather than the SQML as in the case above.
  • Figure 4A and 4B shows an illustration of two .REQ documents (titled 'Headlines on Reuters Related to My Research Report (Live)' and 'Headlines on Reuters (as of January 21 2003, 08 17AM)' on the far right) with a registered association in the Windows shell.
  • the first request document is 'live' and the second one is a snapshot at a particular time (they are both time-sensitive requests).
  • the operating system has associated the semantic browser application (Nervana Librarian) with the document.
  • the semantic query gets opened in the application.
  • Extension Property Sheet this will create a temporary smart request or entity (depending on the kind of document) in the semantic environment and display the property sheet for a smart request or entity.
  • Extension Tool tips this will display a helpful tool tip when the user hovers over a librarian document (a request, .REQ or an entity, .ENT).
  • the information Nervous System also supports the sharing of what the inventor calls "Smart Snapshots.”
  • a smart snapshot is a smart request frozen in time. This will enable a scenario where the user wants to share a smart request but not have it be “live.” For instance, by default, if the user shares the smart request "Breaking News on Reuters related to this document” with a colleague, the colleague will see the live results of the smart request (based on the "current time”). However, if the user wants to share "[Current] Breaking News on Reuters related to this document," a smart snapshot will be employed.
  • a smart snapshot is the same as a smart request (it is also represented by an SQML query document) except that the "attributes" section of the SQML document contains attributes marking it as a snapshot (the flag QUERYATTRIBUTES_SNAPSHOT).
  • the creation date/time of the SQML document is also stored in the SQML (as before - the SQML schema contains a field for the creation date/time).
  • the user interface the semantic browser, Information Agent, or Librarian
  • the process described above is employed. If the user indicates a smart snapshot, the binary document is populated with the edited SQML (containing the snapshot attribute) and the remainder the process is followed as above.
  • the extension handler opens the document and adds an entry into the semantic namespace as a smart request (as described above).
  • the client's semantic query processor will send the processed SQML to the server's XML web service (as previously described).
  • the server's semantic query processor then processes the SQML and honors the snapshot attribute by invoking the semantic query relative to the SQML creation date/time. As such, results will be relative to the original date/time, thereby honoring the intent of the sender.
  • Virtual Knowledge communities refer to a feature of the Information Nervous System that allows the publisher of a knowledge community to publish a group of servers to appear as though they were one server.
  • Reuters could have per-industry Reuters Knowledge Communities (for pharmaceuticals, oil and gas, manufacturing, financial services, etc.) but might also choose to expose one 'Reuters' knowledge community.
  • Reuters will publish and announce the SQML for the virtual knowledge community (rather than the URL to the WSDL of the XML Web Service).
  • the SQML will contain a blender (or collection) of the WSDLs of the actual Knowledge communities.
  • the semantic browser will then pick up the SQML and display an icon for the knowledge community (as though it were a single server).
  • Semantic queries that are time-sensitive are preferably implemented in an intelligent fashion to account for the rate of knowledge generation at the knowledge community (agency) in question. For instance, 'Breaking News' on a server that receives 10 documents per second is not the same as 'Breaking News' on a server that receives 10 documents per month.
  • the server-side semantic query processor would preferably adjust its time-sensitive semantic query handling according to the rate at which information accumulates at the server. To implement this, general rules of thumb could be used, for instance:
  • N can also be adjusted based on whether the query is a Headline or Breaking News.
  • newsmaker queries is preferably implemented with the same time- sensitivity parameters as Headlines.
  • Text-to-speech is implemented at the object level and at the request level.
  • the object skin runs a script to take the SRML of the object, interprets the SRML, and then passes select pieces of text (in the SRML fields) to a text-to-speech engine (e.g., using the Microsoft Windows Speech SDK) that generates voice output.
  • a text-to-speech engine e.g., using the Microsoft Windows Speech SDK
  • Figure 5 shows a diagram illustrating text-to-speech object skin. When executed, the pipeline shown in Figure 5 results in the following voice output:
  • the semantic browser's presentation engine loads a skin that takes the SRML for all the current objects being rendered (based on the user-selected cursor position) and then invokes the text-to-speech object skin for each object. This essentially repeats the text-to-speech action for each XML object being rendered, one after another.
  • Web services are software building blocks used for distributed computing
  • Figure 6 shows an illustration of several email objects being presented in the semantic browser via a request skin.
  • Language translation skins are implemented similar to text-to-speech skins except that the transform is on the language axis.
  • the XSLT skin (smart style) can invoke a software engine to automatically perform language translation in real-time and then generate XML that is encoded in Unicode (16 bits per character) in order to account for the universe of languages.
  • the XSLT transform that generates the final presentation output then will render the output using the proper character set given the contents of the translated XML.
  • Semantic queries can also be invoked in a language-agnostic fashion. This is implemented by having a translation layer (the SQML language translator) that translates the SQML that is generated by the semantic browser to a form that is suitable for interpretation by the KDS (or KBS) which in turn has a knowledge domain ontology seeded for one or more languages.
  • the SQML language translator translates the objects referred to by the predicates (e.g., keywords, text, concepts, categories, etc.) and then sends that to the server-side semantic query processor for interpretation. The results are then translated back to the original language by the language translation skin.
  • the end user will be able to issue a query for a category as an information type - e.g., 'Web services.'
  • the metadata will then be displayed in the semantic browser, as would be the case for any first-class information object type.
  • Visualizations, dynamic links, context palettes, etc. will also be available using the category object as a pivot. This feature is useful in cases where the user wants to start with the category and then use that as a pivot for dynamic navigation, as opposed to starting off with a smart request (smart agent) that has the category as a parameter.
  • Categorized annotations follow from categqries being first-class objects. Users will be able to annotate a category directly - thereby simulating an email list that is mapped to a category. However, for cases where there are many categories (for instance, in pharmaceuticals), this is not recommended because information can belong to many categories and the user should not have to think about which category to annotate - the user should publish the annotation directly to the knowledge community (agency) where it will be automatically categorized or annotate an object like a document or email message that is more contextual than a category. 11. Additional Context Templates
  • Experts - The Experts feature was indicated as a special agent in my parent application Serial No. 10/179,651. As should have also been understood from that application, the Experts feature can also operate in conjunction with the context templates section. Experts are a context template and as the name implies indicate people that have expertise on one or more subject matters or contexts (indicated by the PREDICATETYPEID_EXPERTON predicate).
  • Interest Group this refers to a context template which as the name implies indicate people that have interest (but not necessarily expertise) on one or more subject matters or contexts (indicated by the PREDICATETYPEJDJNTERESTIN predicate).
  • This context template returns People that have shown interest in any semantic category in the semantic network.
  • a very real-world scenario will have Experts returning people that have answers and interest Group returning results of people that have questions (or answers), hi the preferred embodiment, this is implemented by returning results of people who have authored information that in turn has been categorized in the semantic network, with the knowledge domains configured for the KIS.
  • this context template presents the user with dynamic, semantic communities of interest. It is a very powerful context template.
  • the Interest Group context predicate in turn is interpreted by the server-side semantic query processor. This allows powerful queries like “Interest Group on XML” or “Interest Group on Bioinformatics.” Similarly, this would allow queries (via drag, and drop and/or smart copy and paste) like "Interest Group on My Local Document” and "interest Group on My Competitor (an entity).”
  • the Interest Group context template also becomes a part of the Dossier (or Guide) context template (which displays all special agents for each context templates and loads them as sub-queries of the main agent/request).
  • the context template should have a time-limit for which it detects "areas of interest.” An example of this would be three months.
  • the logic here is that if the user has not authored any information (most typically email) that is semantically relevant to the SQML filter (if available) in three months, the user either has no interest in that category (or categories) or had an interest but doesn't any longer.
  • Annotations of My Items this is a context template that is a variant of Annotations but is further filtered with items that were published by the calling user. This will allow the user to monitor feedback specifically on items that he/she posted or annotated.
  • the semantic browser will support the importation and exportation of user state.
  • the user will be able to save his/her personal state to a document and export it to another machine or vice- versa.
  • This state will include information (and metadata) on:
  • Default user state e.g., computer sophistication level, default areas of interest, default job role, default smart styles, etc.
  • Smart requests per profile
  • Local Requests per profile
  • Subscribed Knowledge Communities per profile
  • the semantic browser will show UI (likely a wizard) that will allow the user to select which of the user state types to import or export.
  • the UI will also ask the user whether to include identity/logon information.
  • the semantic browser will serialize the user state into an XML document that has fields corresponding to the metadata of all the user state types.
  • the semantic browser will navigate the XML document nodes and add or set the user state types in the client environment corresponding to the nodes in the XML document.
  • Local smart requests would allow the user to browse local information using categories from an knowledge community (agency), i the case of categorized local requests, the semantic client crawls the local hard drives, email stores, etc. extracts the metadata (including summaries) and stores the metadata in a local version of the semantic metadata store (SMS).
  • the client sends the XML metadata (per object) to an knowledge community for categorization (via its XML Web Service).
  • the knowledge community then responds with the category assignment metadata.
  • the client updates the local semantic network (via the local SMS) and responds to semantic queries just like the server would.
  • this feature can provide functionality equivalent to a local server without the need for one.
  • Integrated Navigation integrated Navigation allows the user to dynamically navigate from within the Presenter (in the main results pane on the right) and have the navigation be integrated with the shell extension navigation on the left. Essentially, this merges both stacks, hi the preferred embodiment, this is accomplished via event signaling.
  • the Presenter wants to dynamically navigate to a new request, it sets some state off the GUID that identifies the current browser view.
  • the GULD maps to a key in the registry that also has a field called 'Navigation Event,' 'Next Namespace Object ID' and 'Next Path.'
  • the 'Navigation Event' field holds a DWORD value that points to an event handle that gets created by the current browser view when it is loaded.
  • the Presenter When the Presenter wants to navigate to a new request, it creates the request in the semantic environment and caches the returned ID of the request. It then dynamically gets the appropriate namespace path of the request (depending on the information/context type of the request) and caches that too. It then sets the two fields ('Next Namespace Object ID' and 'Next Path' with these two values). Next, it sets the 'Navigation Event' (in Windows, this is done by calling a Win32 API named 'SetEvent').
  • the browser view starts a worker thread when it first starts. This thread waits on the navigation event (and also simultaneously waits on a shutdown event that gets signaled when the browser view is being terminated - in Windows, it does this via a Win32 API named 'WaitForMultipleObjects'). If the navigation event is signaled, the 'Wait' API returns indicating that the navigation event was signaled. The worker thread then looks up the registry to retrieve the navigation state (the object id and the path). It then calls the shell browser to navigate to this object id and path (in Windows, this is done by retrieving a 'PIDL' and then calling IShellBrowser::BrowseTo off the shell view instance that implements IShellView).
  • the Nervana semantic browser empowers the user to dynamically navigate a knowledge space at the speed of thought.
  • the user could navigate along context, information or time axes. However, as the user navigates, he/she might be presented with redundant information. For instance, the user can navigate from a local document to 'Breaking News' and then from one of the 'Breaking News' result objects to 'Headlines.' However, semantically, some of the Headlines might overlap with the breaking news (especially if not enough time has elapsed). This is equivalent to browsing the Web and hitting the same pages over and over again from different 'angles.'
  • the Nervana semantic browser handles this redundancy problem by having a local cache of recently presented results.
  • the Presenter indicates redundant results to the user by showing the results in a different color or some other UI mechanism.
  • the local cache is aged (preferably after several hours or the measured time of a typical 'browsing experience'). Old entries are purged and the cache is eventually reset after enough time might have elapsed. Alternately, at the users option, the redundant results can be discarded and not presented at all.
  • the semantic browser will also handle duplicate results by removing duplicates before rendering them in the Presenter - for instance if objects with the same metadata appear on different Knowledge Communities (agencies). The semantic browser will detect this by performing metadata comparisons. For unstructured data like documents, email, etc., the semantic browser will compare the summaries - if the summaries are identical the documents are very likely to be identical (albeit this is not absolutely guaranteed, especially for very long documents).
  • Client-side Knowledge Federation which allows the user to federate knowledge communities and operate on results as though they came from one place (this federation feature was described in my parent Application Serial No. 10/179,651).
  • Client-side Knowledge Federation is accomplished by the semantic browser merging SRML results as they arrive from different (federated) KISes.
  • Server-Side Knowledge Federation is technology that allows external knowledge to be federated within the confines of a knowledge community. For instance, many companies rely on external content providers like Reuters to provide them with information. However, in the Information Nervous System, security and privacy issues arise - relating to annotations, personal publications, etc. Many enterprise customers will not want sensitive annotations to be stored on remote servers hosted and managed by external content providers.
  • Reuters will provide their content to a customer like Intel but Intel will host and manage the KIS.
  • the Intel KIS would crawl the Reuters KIS (thereby chaining KIS servers) or the Reuters DSA. This way, sensitive Intel annotations can be published as 'Post-Its' using Reuters content as context while Intel will still maintain control over its sensitive data.
  • Federated annotations is a very powerful feature that allows the user to annotate an object that comes from one agency/server (KIS) and annotate the object with comments (and/or attachment(s)) - like "Post-Its" on another server.
  • a server (call it Server A) might not support annotations (this is configurable by the administrator and might be the common case for Internet-based servers that don't have a domain of trust and verifiable identity).
  • a user might get a document (or any other semantic result) from Server A but might want to annotate that object on one or more agencies (KISes) that do support annotations (more typically Intranet or Extranet-based agencies that do have a domain of trust and verifiable identity).
  • the annotation email message would include the URI of the object to be annotated (the email message and its attachment(s) would contain the annotation itself).
  • the server crawls its System Inbox and picks up the email annotation, it scans the annotation's encoded To or Subject field and extracts the URI for the object to be annotated. If the URI refers to a different server, the server then invokes an XML Web Service call (if it has access) to that server to get the SRML metadata for the object. The server then adds the SRML metadata to its Semantic Metadata Store (SMS) and adds the appropriate semantic links from the email annotation to the SRML object.
  • SMS Semantic Metadata Store
  • the destination server (for the annotation) does not have access to the server on which the object to be annotated resides, the destination server informs the client of this and the client then has to get the SRML from the server (on which the object resides) and send the complete SRML back to the destination server (for the annotation).
  • This embodiment essentially implies that the client must first "de-reference" the URI and send the SRML to the destination server, rather than having the destination server attempt to "de-reference" the URI itself. This approach might also be superior for performance reasons as it spreads the CPU and I/O load across its clients (since they have to do the downloading and "de-referencing" of the URI to SRML).
  • Semantic Alerts for Federated Annotations hi the same manner that semantic browser would poll each KIS in the currently viewed user profile for "Breaking News" relevant to each currently viewed object on a regular basis (e.g., every minute), the same will be performed for annotations. Essentially, this resembles polling whether each object that is currently displayed "was just annotated.” For annotations that are not federated (i.e., annotations that have strong semantic links to the objects they annotate), this is a straightforward SQML call back to the KIS from whence the annotated object came.
  • the semantic browser would poll each KIS in the selected profile and pass the URI of the object to "ask" the KIS whether that object has been annotated on it. This way, semantic alerts will be generated even for federated annotations.
  • every category will be qualified with at least the following properties:
  • Knowledge Domain ID this is a globally unique identifier that uniquely identifies the knowledge domain from whence the category came
  • the categories knowledge domain id (and not the name) is preferably used in the category URI, because the category could be renamed as the knowledge domain evolves (but the identifier should remain the same).
  • the knowledge domain id is c9554bce-aedf-4564-81f7-48432bf8e5a0
  • the URI type is "category”
  • the category path is "Information Technology/Wireless/80211”.
  • the semantic browser will also allow users to anonymously annotate and publish to an knowledge community (agency).
  • the metadata is completely stored (with the user identity) but is flagged indicating that the publisher wishes to remain anonymous.
  • the inference Engine can infer using the complete metadata but requests for the publisher will not reveal his/her identity.
  • the administrator will also be able to configure the knowledge community (agency) such that the inference engine cannot infer using anonymous annotations or publications.
  • the semantic browser will also have offline support.
  • the browser will have a cache for every remote call.
  • the cache will contain entries to XML data. This could be SRML or could be any other data that gets returned from a call to the XML Web Service.
  • Each call is given a unique signature by the semantic browser and this signature is used to hash into the XML data. For instance, a semantic query is hashed by its SQML. Other remote calls are hashed using a combination of the method name, the argument names and types, and the argument data.
  • the semantic runtime client For every call to the XML Web Service, the semantic runtime client will extract the signature of the call and then map this to an entry in the local cache. If the browser (or the system) is currently offline, the client will return the XML data in the cache (if it exists). If it does not exist, the client will return an error to the caller (likely the Presenter). If the browser is online, the client will retrieve the XML data from the XML Web Service and update the cache by overwriting the previous contents of the file entry with a file path indicated by the signature hash.
  • the information Nervous System can be implemented in a cross-platform manner.
  • Standard protocols are preferably employed where possible and the Web service layer should use interoperable Web service standards and avoid proprietary implementations.
  • the test is that the semantic browser does not have to "know" whether the Knowledge community (or agency) Web service it is talking to is running on a particular platform over another.
  • the semantic browser need not know whether the Web service it is talking to is ranning on Microsoft's .NETTM platform or Sun's J2EE platform (to take 2 examples of proprietary application servers), a Linux or any other "open source” server.
  • the Knowledge community Web service and the client- server protocol should employ Web service standards that are commonly supported by different Web service implementations like .NETTM and J2EETM.
  • the Knowledge community schema is preferably extended to include a field that indicates the Web service platform implementation.
  • a .NETTM implementation of the Knowledge community is preferably published with a field that indicates that the platform is .NETTM.
  • J2EETM J2EETM.
  • the semantic browser will then have access to this field when it retrieves the metadata for the Knowledge community (either directly via the WSDL URL to the Knowledge community, or by receiving announcements via multicast, the enterprise directory (e.g., LDAP), the Global Knowledge community Directory, etc.).
  • the semantic browser can then issue platform-specific calls depending on the platform that the Knowledge community is running on. This is not a recommended approach but if it is absolutely necessary to make platform-specific calls, this model is preferably employed in the preferred embodiment.
  • Knowledge Modeling refers to the recommended way enterprises will deploy an Information Nervous System. This involves deploying several KIS servers (per high-level knowledge domain) and one (or at most few) KDS (formerly KBS) servers that host the relevant ontology and taxonomy. KIS servers are preferably deployed per domain to strike a balance between being too narrow such that there is not enough knowledge sharing possibility of navigation and inference in the network and being too high that scalability (in storage and CPU horsepower needed by the database and/or the inference engine) becomes a problem.
  • KIS servers are preferably deployed where access control becomes necessary at the server level (for higher-level security) as opposed to imposing access control at the group level with multiple groups sharing the same KIS.
  • a large pharmaceutical company could have a knowledge community KIS for oncology for the entire company and another KIS for researchers working on cutting-edge R&D and applying for strategic patents. These two KIS' might crawl the same sources of information but the latter KIS would be more secure because it would provide access only to users from the R&D group.
  • these researchers' publications and annotations will not be viewable on the corporate KIS.
  • FIG. 7 illustrates an example of a possible knowledge architecture for a pharmaceuticals company.
  • the KDS can serve several subsidiary KIS', as follows:
  • Knowledge integration Server 1 Oncology
  • Knowledge Integration Server 2 (Pharmacology)
  • Knowledge Integration Server 3 Biotechnology
  • Knowledge Integration Server 4 (Cardiology)
  • Knowledge Domain Server Purge
  • the Knowledge Integration Server will allow the admin to set up 'housekeeping' rales to purge old or stale metadata. This will prevent the SMS on the KIS from growing infinitely large. These rules could be as simple as purging any metadata older than a certain age (between 2-5 years depending on the company's policies for keeping old data) and which does not have any annotations and that is not marked as a favorite (or rated).
  • the client components of the system can be integrated in several different steps or sequences, as can the workflow interaction or usage patterns.
  • the workflow and component integration would be as follows:
  • Shell User implicitly creates a SQML query (i.e. an agent) via UI navigation or a wizard.
  • Registry entry contains: Agent Name, Creation date, Agent (Request)- GULO, SQML path, Comments, Namespace object type (agency, agent, blender, etc), and attributes
  • Shell The request is handed off to the presenter: a) A registry request GULD entry is created containing (namespace path that generated the request, and SQML file URL). b) Browser is initialized and opened with command line http://PresenterPage.html#RequestGUID http://presenterpage.html/. The Presenter loads default Chrome contained in the page. c) Presenter page loads presenter binary behavior and Semantic Runtime OCX.
  • Query Manager Monitors server requests and makes callback on data. It also signals an event on request completion or timeout. The callback is into the Presenter, which mean inter-process messaging to pass the XML.
  • Presenter receives data and loads appropriate skin: a) Receives SRML data in buffer; this will happen incrementally. b) Determines if there is a preferred skin (smart style) associated with this agent, otherwise chooses default skin. c) Transforms SRML into preferred skin format via XSLT. This is multistage, for the tree of results (root is list, then objects, then Deep/Lens/BN info) as results come in. d) Display results in target DIV in page. The target is an argument to the behavior itself and is defined by the root page.
  • Presenter Calls Semantic Runtime to fill context panels (per context template), deep info, smart copy and paste, and other semantic commands.
  • the Presenter also loads the smart style, which then loads semantic images, motion, etc. consistent with the semantics of the request.
  • Figure 8 illustrates the presently preferred client component integration and interaction workflow described above.
  • the Categories Dialog Box allows the user to select one or more categories from a category folder (or taxonomy) belonging to a knowledge domain. While more or fewer can be deployed in certain situations, in the preferred embodiment, the dialog box has all of the following user interface controls:
  • Profile this allows the user to select a profile with which to filter the category folders (or taxonomies) based on configured areas of interest. For instance, if a profile has areas of interest set to "Health and Medicine,” selecting that profile will display only those category folders that belong to the "Health and Medicine” area of interest (for instance, Pharmaceuticals, Healthcare, and Genes). This control allows the user to focus on the taxonomies that are relevant to his/her knowledge domain, without having to see taxonomies from other domains.
  • Area of Interest this allows the user to select a specific area of interest.
  • this combo box is set to "My Areas of Interest” and the profile combo box is set to "All Profiles.” This way, the dialog box will display category folders for all areas of interest for all profiles. However, by using the "Area of interest” combo box, the user can directly specify an area of interest with which to filter the category folders, regardless of the areas of interest in his/her pro file(s).
  • Publisher Domain Zone/Name this allows the user to select the domain zone and name of the taxonomy publisher. This is advantageous to distinguish publishers that might have name collisions.
  • the Publisher Domain Name uses the DNS naming scheme (for instance, IEEE.org, Reuters.com).
  • the domain zone allows the user to select the scope of the domain name.
  • the options are Internet, Intranet, and Extranet.
  • the zone selection further distinguishes the published category folder (or taxonomy).
  • a fairly common case would be where a department in a large enterprise has its own internal taxonomy. In this case, the department will be assigned the Intranet domain zone and will have its own domain name - for instance, Intranet ⁇ Marketing or Intranef ⁇ Sales.
  • Category Folder this allows the user to select a category folder or taxonomy. When this selection is made, the categories for the selected category folder are displayed in the categories tree view.
  • Search categories this allows the user to enter one or more keywords with which to filter the currently displayed categories. For instance, a Pharmaceuticals researcher could select the Pharmaceuticals taxonomy but then enter the keyword "anatomy" to display only the entries in the taxonomy that contain the keyword “anatomy.”
  • the taxonomy entry "Anatomy ⁇ Cells ⁇ Chromaffin Cells” will be included in a hierarchy filter because the hierarchy includes the word “Anatomy.” However, it will be excluded from a names filter because the end-name (“Chromaffm Cells”) does not include the word "Anatomy.”
  • search options allow the user to select whether the dialog box should check for all keywords, for any keyword, or for the exact phrase.
  • Categories Tree View - the tree view displays the taxonomy hierarchy and allows the user to select one or more items to add to the Create Request Wizard or to open as a new Dossier (Guide) request/agent.
  • the user interface breaks the category hierarchy into "category pages" - for performance reasons.
  • the UI allows the user to navigate the pages via buttons and a slide control. There is also a "Deselect All” button that deselects all the currently selected taxonomy items.
  • My parent application (Serial No. 10/179,651) described how the KIS can use a consistency checker (CC) to periodically purge stale entries from the SMS.
  • this approach might impair perfonnance because the CC would have to periodically scan the entire SMS and confirm whether the indexed objects still exist.
  • An alternative embodiment of this feature of the invention is to have the client (the semantic browser) notify the server if it gets a 404 error. To do this, the semantic browser would have to track when it gets a 404 error for each result that the user "opens.”
  • the client can poll for the HTTP headers when it displays the results, even before the user opens the results. In this case, if the source web server reports a 404 error (object not found), the client should report this to the KIS.
  • the KIS When the KIS gets a "404 report" from the client, it then intelligently decides whether this means the object is no longer available.
  • the KIS cannot arbitrarily delete the object because it is possible that the 404 error was due to an intermittent Web server failure (for instance, the directory on the Web server could have been temporarily disabled).
  • the KIS should itself then attempt to asynchronously download the object (or at the very least, the HTTP headers in the case of a Web object) several times (e.g., 5 times). If each attempt fails, the KIS can then conclude that the object is no longer available and remove it from the SMS. If another client reports the 404 error for the same object while the KIS is processing the download, the KIS should ignore that report (since it is redundant).
  • This alternate technique could be roughly characterized as lazy consistency checking, h some situations, it may be advantageous and preferred.
  • the server performs duplicate detection by checking the source URIs before adding new objects into the semantic metadata store (SMS).
  • SMS semantic metadata store
  • the server does not perform strict duplicate-detection. In such cases, duplicate detection is best performed at the client.
  • the client federates results from several KISes, it is possible for the client to get duplicates from different KISes. As such, it is advantageous if the client also performs duplicate detection.
  • the client removes objects that are definitely duplicates and flags objects that are likely duplicates. Definite duplicates are objects that have the same URI, last modified time stamp, summary/concepts, and size.
  • Likely duplicates are objects that have the same summary/concepts, but have different URIs, last modified times, or sizes. For objects for which summary extraction is difficult, it is recommended that the title also be used to check for likely duplicates (i.e., objects that have the same summary but different titles are not considered likely duplicates because the summary might not be a reliable indicator of the contents of the object).
  • the semantic browser can limit the file-size check to plus or minus N % (e.g., 5%) - for instance, an object with the same summary/concepts and different URIs, last-modified times, and sizes might be disqualified as a likely duplicate if the file-size is within 5% of the file-size of the object it is being compared to for redundancy checking.
  • N % e.g., 5%
  • the client also provides the user with a seamless user experience when there are multiple knowledge communities (agencies) subscribed to a user profile.
  • the semantic browser preferably presents the results as though they came from one source.
  • the browser preferably presents the user with one navigation cursor - as the user scrolls, the semantic browser re-queries the KISes to get more results.
  • the semantic browser keeps a results cache big enough to prevent frequent re-querying - for instance, the cache can be initialized to handle enough results for between 5-10 scrolls (pages).
  • the cache size are preferably capped based on memory considerations.
  • the browser checks if the current page generates a cache hit or miss. If it generates a cache hit, the browser presents the results from the cache, else if re-queries the KISes for additional results which it then adds to the cache.
  • the cache can be implemented to grow indefinitely or to be a sliding window.
  • the former option has the advantage of simplicity of implementation with the disadvantage of potentially high memory consumption.
  • the latter option which is the preferred embodiment, has the advantage of lower memory consumption and higher cache consistency but with the cost of a more complex implementation.
  • the semantic browser will purge results from pages that do not fall within the window (e.g., the last N - e.g., 5-10 - pages as opposed to all pages as with the other embodiment).
  • the client also provides the user with a seamless user experience when authenticating the user to his/her subscribed knowledge communities (agencies). It does this via what the inventor calls "virtual single sign-on.”
  • This model involves the semantic browser authenticating the user to knowledge communities without the user having to enter his/her username and password per knowledge community.
  • the user will have a few usernames and passwords but might have many knowledge communities of which he/she is a member (especially within a company based on departmental or group access, and on Internet- based knowledge communities).
  • the ratio of the number of knowledge communities to the number of authentication credentials (per user) is likely to be very high.
  • the user specifies his/her logon credentials to the semantic browser in a server (knowledge community)-independent fashion.
  • the semantic browser stores the credentials in a Credential Cache Table (CCT).
  • CCT has columns as illustrated below:
  • logon user name e.g., an email address
  • KCEL Knowledge Community Entry List
  • the semantic browser When the user first attempts to subscribe to a knowledge community (or access the knowledge community in some other way - for instance, to get the properties of the community), the semantic browser prompts the user for his/her password and then tries to logon to the server using the supplied credentials. If a logon is successful, the semantic browser creates a new CCT entry (CCTE) with the supplied credentials and adds the KC to the Knowledge Community Entry List (KCEL) for the new CCT entry.
  • CCTE CCT entry
  • KCEL Knowledge Community Entry List
  • the semantic browser For each subsequent subscription attempt, the semantic browser checks the CCT to see if the KC the user is about to subscribe to is in the KCEL for any CCTE. If it is, the semantic browser retrieves the credentials for the CCTE and logs the user on with those credentials. This way, the user does not have to redundantly enter his/her logon credentials.
  • the semantic browser also supports pass-through authentication when the operating system is already logged on to a domain. For instance, if a Windows machine is already logged on to an NT (or Active Directory) domain, the client-side Web service proxy also includes the default credentials to attempt to logon to a KC. hi the preferred embodiment, the additional credentials supplied by the user are preferably passed via SOAP security headers (via Web Services Security (WS-Security) or a similar scheme).
  • WS-Security Web Services Security
  • the semantic browser exposes a property to allow the user to indicate whether the credentials for a CCTE are preferably purged when the KCEL for the CCTE is empty or whether the credentials should be saved, hi the preferred embodiment, the credentials are preferably saved by default unless the user indicates otherwise. If the user wants the credentials purged, the semantic browser should remove a KC from a CCTE in which it exists when that KC is no longer subscribed to any profile in the browser. If after removing the KC from the CCTE's KCEL, the CCTE becomes empty, the CCTE is preferably deleted from the CCT.
  • the virtual single sign-on feature like many of the features in this application, could be used in applications other than with my Information Nervous System or the Virtual Librarian. For example, it could be adapted for use by any computer user who must log into more than one domain.
  • the table below shows the actions that the semantic browser invokes when namespace objects are copied and pasted onto other namespace objects.
  • the Information Nervous SystemTM will support dynamic updates of ontologies and taxonomies.
  • Knowledge domain plug-ins that are published by Nervana (or that are provided to Nervana by third-party ontology publishers) will be hosted on a central Web service (an ontology depot) on the Nervana Web domain (Nervana.com).
  • Each KDS will then periodically poll the central Web service via a Web service call (for each of its knowledge domain plug-ins, referenced by the URI or a globally unique identifier of the plug-in) and will "ask” the Web service if the plug-in has been updated.
  • the Web service will use the last-modified timestamp of the ontology file to determine whether the plug-in has been updated. If the plug-in has been updated, the Web service will return the new ontology file to the calling KDS.
  • the KDS then replaces its ontology file.
  • the KDS If the KDS is running during the update, it will ordinarily temporarily stop the service before replacing the file, unless it supports file-change notifications and reloads the ontology (which is the recommended implementation).
  • Each KIS also has to poll each KDS it is connected to in order to "ask" the KDS if its ontology has changed.
  • the KIS should poll the KDS and not the central Web service in case the KDS has a different version of the ontology.
  • the KDS also uses the last modified time stamp of the knowledge domain plug-in (the ontology) to determine if the ontology has changed. It then indicates this to the KIS. If the ontology has changed, the KIS needs to update the semantic network accordingly. In the preferred embodiment, it does this by removing semantic links that refer to categories that are not in the new version of the ontology and adding/modifying semantic links based on the new version of the ontology.
  • the client purges the semantic network and re-indexes it.
  • the client polls each KIS it is subscribed to in order to determine if the taxonomies it is subscribed to (directly via the central Web service or via the KISes) have changed.
  • the KIS exposes a method via the XML Web service via which the client determines if the taxonomy has changed (via the last modified time stamp of the taxonomy/ontology plug-in file). If the taxonomy has changed, the client needs to update the Categories Dialog user interface (and other Ul-based taxonomy dependents) to show the new taxonomy.
  • the client For taxonomies that are centrally published (e.g., via Nervana), the client should poll the central Web service to update the taxonomies.
  • Dossier (Guide) queries are preferably invoked by the client-side semantic query processor by parsing the SQML of the request/agent and replacing the Dossier context predicate with each special agent (context template) context predicate - e.g., All Bets, Best Bets, Breaking News, Headlines, Random Bets, Newsmakers, etc.
  • Each query (per context template) is then invoked via the query processor - just like an individual query. This way, the user operates at the level of the Dossier but the semantic browser maps the dossier to individual queries behind the scenes.
  • the client-side semantic query processor retains every other predicate except the context predicate. This way, the filters remain consistent as illustrated by the example above.
  • dossiers can be used as a Smart Lens (just like how they can be targets for drag and drop, smart copy and paste, etc.).
  • the smart lens displays a "Dossier Preview Window" with sections/tabs/frames for each context template (special agent). Sample screenshots of the Dossier showing the UI of the Dossier Smart Lens are included in Figures 12 and 13.
  • the semantic browser adds the KC's email address to the local email contacts (e.g., in Microsoft Outlook or Outlook Express). This makes it easy for the user to publish knowledge to the KC by sending it email (via the integrated contacts list). Similarly, when the KC is unsubscribed from all profiles, the semantic browser prompts the user whether it should remove the KC from the local email contacts list. 32. Dynamic Ontology and Taxonomy Mapping
  • the Information Nervous SystemTM accomplishes this by the following algorithm:
  • Each KDS will be responsible for ontology mapping (via an Ontology Mapper (OM)) and will periodically update the central Web service (the ontology depot) with an Ontology Mapping Table (OMT).
  • the updates are bi-directional: the KDS will periodically update its ontologies and taxonomies from the central Web service and send updates of the OMT to the central Web service.
  • Each OMT will be different but the central ontology depot will consolidate all OMTs into a Master OMT.
  • the ontology mapper will create a consistent user experience because the user wouldn't have to select all items in the umbrella taxonomy that are relevant but overlapping.
  • the semantic browser will automatically handle this.
  • the KIS wouldn't have any concept of the mapper but will get mapped results from the KDS which it will then use to update the semantic network.
  • the KDS and KIS administrators would still be responsible for selecting the right KDS ontology plug-ins, however - based on the quality of each ontology/taxonomy (the ontology mapping doesn't improve ontologies; it merely maps them).
  • Semantic Alerts in the semantic browser can be optimized by employing the following rule (in order):
  • Headlines are implemented similar to Breaking News except with a larger time window. As a consequence, if there are no Headlines (in the preferred embodiment), there is no Breaking News. Also, in the preferred embodiment, Newsmakers are implemented by returning the authors of Headlines. As such, if there are no Headlines, there are no Newsmakers.
  • Corbis http://www.corbis.com
  • Getty Images http://www.gettyimages.com
  • the Information Nervous SystemTM can use these kinds of images for semantic images that are not only context-sensitive but also "fresh.” This can be advantageous in terms of keeping the user interface interesting and "new.” For instance, "Breaking News on SARS” can show not only pharmaceutical images but images showing doctors responding to recent SARS outbreaks, etc.
  • Semantic images can be dynamically and intelligently selected using the following rules:
  • the currently displayed namespace object is a request, parse the SQML of the object for categories. If there are categories, send the categories to the central Web service (that hosts the semantic image cache) to get images that are relevant to the categories. Also, send the request type (e.g., knowledge types like All Bets and Headlines, or information types like Presentations) to the central Web sendee to return images consistent with the request type
  • the request type e.g., knowledge types like All Bets and Headlines, or information types like Presentations
  • the namespace object is not a request, send the areas of interest for the current profile (if available) to the central Web service.
  • the Web service then returns semantic images consistent with the profile's areas of interest. If the profile does not have configured areas of interest, send the areas of interest for the application (the semantic browser). If the application does not have configured areas of interest, send an empty string to the central Web service - in this case, the central Web service returns generic images (e.g., branded images).
  • Dynamic Knowledge Community Contacts Membership Knowledge communities (agencies) have members (users that have read, write, or read- write access to the community) and contacts. Contacts are users that are relevant to the community but are not necessarily members. For example, a departmental knowledge community (KC) in a large enterprise would likely have the members of the department as members of the KC but would likely have all the employees of the enterprise as contacts. Contacts are advantageous because they allow members of the KC to navigate users that are semantically relevant to the KC but might not be members. The KC might semantically index
  • the index in this case would include the contacts even though the contacts are not members of the KC.
  • Core members are users that are very active in the community while peripheral members include "other" users such as knowledge hobbyists, occasional contributors, potential recruits, and even members of other relevant communities.
  • the KIS With dynamic KC contacts membership in the Information Nervous SystemTM, the KIS will add users to its Contacts table in the semantic metadata store (SMS) and to the semantic network "when and as it sees them” (in other words, as it indexes email messages that have new users that are not members). This allows the community to dynamically expand its contacts, but in a way that distinguishes between Members and mere Contacts, and "understands” the importance of the distinction semantically when operating the system (e.g., executing searches and the like).
  • SMS semantic metadata store
  • the KIS also indexes concepts (key phrases) and keywords as first-class members of the semantic network. This can be done in a domain-independent fashion as follows:
  • Entities can be defined with categories and can be further narrowed with keywords (to disambiguate the keywords in the case where the keywords could mean different things in different contexts).
  • Integrated full-text indexing allows the KIS semantic query processor (SQP) to interpret entities seamlessly - by applying the necessary sub-queries with categories and keywords/concepts to the semantic network.
  • SQL KIS semantic query processor
  • the KIS might not have the resources to store semantic links between People and objects on a per-object basis.
  • semantic-based redundancy is not the same as per-object redundancy — as in email.
  • email clients allow users to select an email message as read or unread - this is typically implemented as a flag stored on the mail server with the email message.
  • email is not a semantic system, a semantically similar or identical message on the server would not be flagged as such - the user has to flag each message separately regardless of semantic redundancy.
  • the user is able to flag an object as read not unlike in email.
  • the semantic browser extracts the concepts from the object and informs all the KISes in the request profile that the "concepts" have been read.
  • the KIS then dynamically maps the concepts to categories via the KDSes it is configured with and adds a flag to the objects belonging to those categories (in the preferred embodiment) and/or adds a flag to the semantic network with a semantic link with the predicate PREDICATETYPEID_VIEWEDCATEGORY between the categories corresponding to the concepts and all the objects that are linked to the categories.
  • the KIS should only flag those categories over a link-strength threshold (for the source concepts). This ensures that only those objects (in the preferred embodiment) and/or categories that are semantically close to the original object will be flagged.
  • the KISes When the semantic browser flags the object via the KISes, the KISes should return a flag indicating whether the network was updated (it is possible that no changes would be made in the event that the object does not have any "strong” categories or if there are no other objects that share the same "strong” categories). If at least one KIS in the request profile indicates that the network was updated, the semantic browser should refresh the request/agent.
  • the semantic browser can expose a property to allow the user to indicate whether he/she wants the KISes to return only unread objects or all objects (read or unread), in which case the browser should display unread objects differently (like how email clients display unread messages in a bold font).
  • the presentation layer in the semantic browser should then display the read and unread objects with an appropriate font and/or color to provide a clear visual distinction.
  • Multi-select object lens is an alternative implementation of the object lens that was described in my parent application, hi that embodiment, the object lens was invoked via smart copy and paste - pasting an object over another object would invoke the object lens with the appropriate default predicate. This has the benefit of allowing the user to copy objects across instances of the semantic browser, across profiles, and from other environments (like the file- system, word processors, email clients, etc.).
  • the object lens is a Dossier Lens (the context predicate is a Dossier, the filters are the source and target objects, and the profile is the profile in which the source object was displayed).
  • Multi-selection can also be used instead of copy and paste to invoke an object lens.
  • the semantic browser will allow the user to select multiple objects (results). The user can then hit a button (or alternative user-interface object) to invoke the object lens on the selected objects.
  • a Dossier Lens will be displayed (in a preview pane) with a Dossier context predicate, with the filters as the selected objects, and the current profile as the request profile.
  • the KIS (in the preferred embodiment) would only add objects to the Semantic Metadata Store (SMS) if those objects belong to at least one category from at least one of the knowledge domains the KIS is configured with (via one or more KDSes). This essentially means the KIS will not index objects it "does not understand.” The exception to this is that the KIS will index all objects from its System Inbox - because this contains at-times personal community-specific publications and annotations that might be relevant but not always semantically relevant.
  • SMS Semantic Metadata Store
  • a side-effect of this ontology-based filtering model is spam management - ontology- based indexing would be effective in preventing spam from being indexed and stored. If users use the semantic browser to access email, as opposed to their inboxes, only email that has been semantically filtered will get through.
  • the results of a request/agent can be further refined via additional filters and predicates.
  • the request/agent Headlines on Bioinformatics could be further refined with keywords specific to certain areas of Bioinformatics. This way, the end-user can further narrow the result set using the request/agent as a base.
  • the user can specify a time-window to override the default time-window. For example, the default Breaking News time-request could be set to 3 hours. The user should be able to override this for a specific request/agent (in addition to changing the defaults on a per-profile or application-wide basis) with an appropriate UI mechanism (e.g., a slider control that ranges from 1 hour to 24 hours). The same applies to Headlines and Newsmakers (e.g., a slider control that ranges from 1 day to 1 week).
  • the semantic browser invokes the XML Web Service call for each of the KISes in the request profile and passes the override arguments as part of the call. If override arguments are present, the Web service uses those values instead of the default filter values. The same applies to additional filters (e.g., keywords) - these will be passed as additional arguments to the Web service and the Web service will apply additional sub-queries appropriately to further filter the query that is specified in the agent/request SQML (in other words, the SQML is passed as always, but in addition, the filter overrides and additional filters are also passed).
  • additional filters e.g., keywords
  • a good case for filter-overrides will be for Best Bets.
  • the default semantic relevance strength for Best Bets could be set to 90% (in the preferred embodiment).
  • Exposing a relevance UI control e.g., a slider control that ranges from 0% to 100%
  • This essentially allows the user to change the Best Bets on the fly from “All Bets” (0%) all the way to "Perfect Bets" (100%).
  • a hybrid model should also be employed for embodiments of context template (special agent) implementations that involve multiple axes of filtering. For instance, Breaking News could also impose a relevance filter of 25% and Headlines and Newsmakers could impose a relevance filter of 50% (Breaking News has a lower relevance threshold because it has a higher time-sensitivity threshold; as such, the relevance threshold can be relaxed), i this case, the semantic browser should expose UI controls to allow the user to refine the special agents across both axes (a slider control for time-sensitivity and another slider control for relevance).
  • the semantic browser can display UI controls for each special agent displayed in the Dossier - the main Dossier pane can show all the UI controls (changing any UI control would then refresh the Dossier sub-request for that special agent). Also, if the Dossier has tabs for each special agent, each tab can have a UI control specific to the special agent for the tab.
  • the Information Nervous SystemTM can also be used to manage information stores such as personal email inboxes, personal contact lists, personal event calendars, a desktop file-system (e.g., the Microsoft Windows Explorer file-management system for local and network-based files), and also other stores like file-shares, content management systems, and web sites.
  • information stores such as personal email inboxes, personal contact lists, personal event calendars, a desktop file-system (e.g., the Microsoft Windows Explorer file-management system for local and network-based files), and also other stores like file-shares, content management systems, and web sites.
  • the client runtime of the semantic browser should periodically poll the store via a programmatic interface to check for items that have become redundant, stale, or meaningless. This would address the problem today where email inboxes keep growing and growing with stale messages that might have "lost their meaning and relevance.”
  • email inboxes keep growing and growing with stale messages that might have "lost their meaning and relevance.”
  • many computer users are losing the ability to manage their email inboxes themselves, resulting in a junk-heap of old and perhaps irrelevant messages that take up storage space and make it more difficult to find relevant messages and items.
  • the client runtime should enumerate the items in the user's information stores, extract the concepts from the items (e.g., from the body of email messages and from local documents) and send the concepts to the KISes in the user's profiles. In an alternative embodiment, only the default profile should be used. The client then essentially "asks" the user's subscribed KISes whether the items mean anything to them. In the preferred embodiment, the client should employ the following heuristics:
  • the client should check for meaninglessness or irrelevance.
  • the client should only check items that are "older" than N days (e.g., 30 days) by examining the last-modified time of the email item, document, or other object.
  • N days e.g. 30 days
  • the client should specify a very low threshold of meaning to the XML Web Service (e.g., 25%) for preservation. Essentially, this is akin to deleting (or flagging) those items that are very old and weak in meaning.
  • the client should specify a very low threshold (e.g., 10%) for preservation. This is akin to deleting (or flagging) those items that are fairly old and very weak in meaning.
  • the client should specify a very low threshold (e.g., 0%) for preservation. This is akin to deleting (or flagging) those items that are old (but not too old) but are meaningless, based on the user's profile(s).
  • a very low threshold e.g., 0%
  • the model for this aspect or feature of the preferred embodiment balances semantic sensitivity with time-sensitivity by imposing a higher semantic threshold on younger items (thereby preserving items that might be largely - albeit not totally - meaningless if they are fairly young. For example, fairly recent email threads might be very weak in meaning - the client should preserve them anyway because their "youth" is also a sign of relevance. As they "age,” however, the client can safely delete them (or flag them for deletion).
  • This model can also be applied to manage documents on local file-systems.
  • the model can be extended to content-management systems, document repositories, etc. by configuring an Information Store Monitor (ISM) to monitor these systems (via calls to the Information Nervous SystemTM XML Web Services) and configuring the ISM with KISes that are configured with KDSes that have ontologies consistent with the domain of the repositories to be semantically managed.
  • ISM Information Store Monitor
  • KISes that are configured with KDSes that have ontologies consistent with the domain of the repositories to be semantically managed.
  • the refinement pane in the semantic browser allows the user to "search within results.”
  • the user will be able to add additional keywords, specify date ranges, etc.
  • the date-range control can be implemented like a slide-rule. Shifting one panel in the slide-rule would shift the lower date boundary while moving the other panel will shift the upper date boundary. Other panels can then be added for time boundaries - shifting both time and date panels will impose both date and time constraints. Panels can also be added for other filter axes.
  • queries can be broken into several components: a. Context (documents, keywords, entities, portfolios (or entity collections)). b. Context/Knowledge Template (or Special Agent) or Information Template - this describes whether the request if for a knowledge type (e.g., Breaking News, Conversations, Newsmakers, or Popular Items) or for a particular information type (e.g., Documents, Email).
  • a knowledge type e.g., Breaking News, Conversations, Newsmakers, or Popular Items
  • information type e.g., Documents, Email
  • a semantic query is made up of the triangulation of context, request (or Agent) type, and the knowledge communities (or Agencies).
  • the client sends the SQML that represents the semantic query to all the knowledge communities in the profile in which the request lives.
  • the client asks for a few results at a time and then aggregates the results from one or more servers.
  • the server-side semantic query processor subdivides semantic queries into several sub- queries, which it then applies (via SQL inner joins or sub-queries in the preferred embodiment). These sub-queries are:
  • Request type sub-query this represents a sub-query (semantic or non-semantic) depending on the request type. Examples are context (knowledge) types (e.g., All Bets, Best Bets, Headlines, Experts, etc.) and information types (like General Documents, Presentations, Web Pages, Spreadsheets, etc.).
  • context knowledge
  • information types like General Documents, Presentations, Web Pages, Spreadsheets, etc.
  • Semantic context sub-query this represents a semantic sub-query derived from the context (filter) passed from the client (an example of this is categories sent from the client or mapped from keywords/text via semantic stemming).
  • Non-semantic context sub-query this represents a non-semantic sub-query derived from the context (filter) passed from the client (examples are keywords without semantic stemming - mapping to ontology-based categories).
  • Access-control sub-query this represents a sub-query that filters out those items in the semantic metadata store (SMS) that the calling user does not have access to.
  • SMS semantic metadata store
  • Figure 14 shows how the server-side semantic query processor processes incoming semantic queries (represented as SQML).
  • the semantic relevance score defines the normalized score that the concept extraction engine returns. It maps a given term of "blob” of text to one or more categories for a given ontology. The score is added to the semantic network (in the "LinkStrength” field of the "SemanticLinks” table) when items are added to the Semantic Network.
  • the relevance filter is different from the relevance score (indeed, both will typically be combined).
  • the relevance filter indicates how the SQP will semantically interpret context (note: in the currently preferred embodiment, the filtering is always semantic in this case).
  • the relevance filter is preferably used only to create sub-query "buckets" that are then used for ordering results.
  • the SQP might decide to prioritize a High relevance filter ahead of a Low relevance filter when filtering the semantic network but would still return both (with duplicates removed) in order to help guarantee that synonyms don't get rejected during the final semantic filtering process.
  • the time-sensitivity filter determines how time-critical the semantic sub-query is. There are two levels: High and Low. A High filter is meant to be extremely time-critical. Default is 3 hours (this accounts for lunch breaks, time away from the office/desk, etc.). A Low filter is meant to be moderately time-critical. The default is 12 hours.
  • the server simply returns all the items in the semantic metadata store. If the SQML has filters, the filters are imposed via an inner sub-query with no semantic link strength threshold. For instance, All Bets on Topic A will return all items that have anything (strongly or barely) to do with Topic A. b. Random Bets the preferred embodiment, for "Random Bets” queries, the server simply returns all the items in the semantic metadata store (like in the case of "All Bets” queries) but orders the results randomly. If the SQML has filters, the filters are imposed via an inner sub-query with no semantic link strength threshold. For instance, Random Bets on Topic A will return all items (ordered randomly) that have anything (strongly or barely) to do with Topic A. c. Breaking News
  • Breaking News can be implemented in a very intelligent way.
  • the table below illustrates the currently preferred ranking and prioritization for Breaking News when the server tracks what items (and/or categories) the user has read: hi the preferred embodiment, the server processes SQML for Breaking News (via the Breaking News context predicate) as follows:
  • the Semantic Network Manager should update the semantic network to indicate the "last read time" for each user to each category. This is then used in the sub-query to check whether news has been "read” or not (per category or per object - per category is the preferred embodiment because the latter will not scale).
  • the primary ordering axis (Creation Time) guarantees that results are filtered by freshness.
  • the secondary ordering axis (Relevance Score) acts as a tiebreaker and guarantees that equally fresh results are distinguished primary based on relevance.
  • Breaking News Intrinsic Alerts can be implemented on the client by limiting the Brealdng News priority to Priority 2 and by changing the Priority 1 and Priority time-sensitivity filters to high. This way, only very fresh Breaking Unread Semantic News (of both High and Low semantic relevance filters) will be returned. This is advantageous because the alert should have a higher disruption threshold than the Breaking News Request (or agent) - since it is implicit rather than explicit.
  • Unread Breaking News has a lower time-sensitivity filter than Read Breaking News because users are likely to be more tolerant of older news that is new to them than younger news that is not. hi some cases, the server might not have user-state (and "read” information). In this case, a simple implementation of Breaking News is shown below:
  • Breaking News should return only items younger than N hours (default is 3 hours).
  • Breaking News should apply the time- sensitivity filter (3 hours) to the outer sub-query and also apply a moderately strong relevance filter to the inner sub-query (off the SemanticLinks table). In the preferred embodiment, this should correspond to a relevance score (and link strength) of 50%. For instance, Breaking News on Topic A should return those items that have been posted in the last 3 hours and which belong to the category (or categories) represented by Topic A with at least a relevance score of 50%. This will avoid false positives like Breaking News items which are barely relevant to Topic A. d. Headlines
  • the High filter is 12 hours instead of 3 hours and the low filter is 1 day instead of 12 hours.
  • the time-sensitivity constraint is 1 day. This can also be made 3- days on Mondays to dynamically handle weekends (making the number of days the "number of working days"). e. Newsmakers
  • Best Bets are implemented by imposing a filter on the strength of the semantic link with the "Belongs to Category" predicate. The preferred default is 90%, although the client (at the option of the user) can change this on the fly via an argument passed via the XML Web Service. Best Bets are implemented with a SQL inner join between the Objects table and the SemanticLinks table and joining only those rows in the SemanticLinks table that have the "Belongs to Category" predicate and a LinkStrength greater than 90% (default).
  • the server-side semantic query processor must also invoke a sub-query, which is a SQL inner join that maps to the desired filters, i the preferred embodiment, this sub-query should also include a "Best Bets" filter.
  • filters e.g., keywords, text, entities, etc.
  • the outer sub-query it is advantageous and probably preferable for most users for the outer sub-query to be a Best Bet, and for the inner sub-query.
  • "Best Bets on Topic A” is semantically different from “Best Bets that are also relevant to Topic A.”
  • Best Bets which are Best Bets "ON" Topic A, will be returned (via applying the "Best Bets” semantic filter on the inner sub-query).
  • Best Bets will return Best Bets on anything that might have anything to do with Topic A.
  • the second example might return false positives because for example, a document, which is a Best Bet on Topic B but a "weak bet" on Topic B, will be returned and that is not consistent with the semantics of the query or the presumably desired results. Extending the "Best Bets" filter to not only the outer sub-query but also all inner sub-queries will prevent this from happening.
  • Other query implementations can also follow this rule (with the right sub-queries applied based on the semantics of the main query) if the SQML contains filters.
  • the SQP returns only those objects that have the object type id that corresponds to the requested information type.
  • An example is "Information TypeXPresentations.”
  • the SQP parses the SQML received from the client, it extracts this attribute from the SQML and maps it to an object type id. It then invokes a SQL query with an added filter for the object type id.
  • the SQP maps the request to a set of object type ids and invokes a SQL query with this added filter.
  • the server-side SQP has to first semantically interpret the context before generating sub-queries that correspond to it. To do this, the server sends the concepts to all KDS'es (KBS'es) it is configured with (for the desired knowledge community or agency) for semantic categorization. When the server gets the categories back, it preferably determines which of those categories are "strong" enough to be used as filters before generating the appropriate sub-queries.
  • KBS'es KDS'es
  • the server-side SQP preferably chooses only "strong categories" to apply to the sub-queries.
  • the server-side semantic query processor performs semantic stemming to map keywords, text, and concepts to categories based on one or more domain ontologies.
  • One way it does this by invoking an XML Web Service call to the KDS/KBS (or KDSes/KBSes) it is configured with in order to obtain the categories. It then maps the categories to its semantic network.
  • This form of stemming is superior to regular stemming that is based on keyword variations (such as singular and plural variations, tense variations, etc.) because it also involves domain-specific semantic mapping that stems based on meaning rather than merely stemming based on keyword forms.
  • the KIS calls the KDS/KBS each time it receives SQML that requires further semantic interpretation.
  • this could result in delays if the KDS/KBS resides on a different server, if the network connection is not fast, or if the KDS/KBS is busy processing many requests, hi this case, the KIS can also implement a Semantic Stemming Cache.
  • This cache maps keywords and concepts to categories that are fully qualified with URIs (making them globally unique).
  • the server-side semantic query processor receives SQML that contains keywords, text, or concepts (extracted from, say, documents on the client by the client-side semantic query processor), it first checks the cache to see if the keywords have already been semantically stemmed.
  • the SQP simply retrieves the categories from the cache and maps those categories to the semantic network via SQL queries. If there is a cache miss (i.e., if the context is not in the cache), it then calls the KDSes/KBSes to perform semantic categorization. It then takes the results, maps them to unique category URIs, and adds the entry to the cache (with the context as the hash code). Note that even if the context does not map to any category, the "lack of a category" is preferably cached. In other words, the context is added as a cache entry with no categories. This way, the server can also quickly determine that a given context does not have any categories, without having to call the KDSes/KBSes each time to find out.
  • the SQP can also manage the semantic stemming cache. It has to do this for two reasons: first, to keep the cache from growing uncontrollably and consuming too much system resources (particularly memory with a heap-based hash table); and, second, if the KIS configuration is changed (e.g., if knowledge domains are added/removed), the cache is preferably purged because the entries might now be stale.
  • the first scenario can be handled by assigning a maximum number of entries to the cache, the preferred embodiment, the SQP caches the current amount of memory consumed by the cache and the cache limit is dictated by memory usage. For example, the administrator might set the maximum cache size to 64MB. To simplify the implementation, this can be mapped to an approximate count of items (e.g., by dividing the maximum memory usage by an estimate of the size of each cache entry).
  • the SQP For each new entry, if the cache limit has not been reached, the SQP simply adds the entry to the cache. However, if the cache limit has been reached, the SQP (in the preferred embodiment) should purge the least recently added items from the cache. In the preferred embodiment, this can be implemented by keeping a queue of items that is kept in sync with a hash table that implements the cache itself (for quick lookups using the context as a key). When the SQP needs to purge items from the cache to free up space, it de-queues an item from the least-recently-added queue and also removes the corresponding item from the hash table (using the context as key). This way, fresh items are more likely to result in a cache hit than older items.
  • Extensible client-side user profiles allow the user of a semantic browser to have a different state for different job roles, knowledge sources, identities, personas, work styles, etc. This essentially allows the user to create different "knowledge worlds" for different scenarios. For instance, a Pharmaceuticals researcher might have a default profile that includes all sources of knowledge that are relevant to his/her work. As described in my parent application Serial No. 10/179,651, the SRML from each of these sources will be merged on the client thereby allowing the user to seamlessly go through results as though they were coming from one source. However, the researcher might want to track patents separate from everything else. In such a case, the researcher would be able to create a separate "Patents" profile and also include those knowledge communities (agencies) that have to do with patents (e.g.,. the US Patent Office Database, the EU Patent Database, etc.)
  • the user might create a profile for 'Work' and one for 'Home.
  • Many investment analysts track companies across a variety of industries. With the semantic browser, they would create profiles for each industry they track. Consultants move from project to project (and from industry to industry) and might want to save requests and entities created with each project. Profiles will be used to handle this scenario as well. Profiles contain the following user state:
  • Smart styles the smart styles to be used by default for requests and entities created with the profile.
  • Default Flag this indicates whether the profile is the default profile.
  • the default profile is initiated by default when the user wishes to create requests and entities, browse information communities, etc. Unless the user explicitly selects a different profile, the default profile gets used.
  • Profiles can be created, deleted, modified, and renamed. However, in the preferred embodiment the default profile cannot be deleted because there has to be at least one profile in the system at all times. In alternate embodiments, a minimum profile would not be required.
  • all objects in the semantic browser are opened within the context of a profile.
  • a smart request is created in a profile and at runtime, the client semantic query processor will use the properties of the profile (specifically the subscribed knowledge communities (agencies) in that profile) to invoke the lequest.
  • the client semantic query processor will use the properties of the profile (specifically the subscribed knowledge communities (agencies) in that profile) to invoke the lequest.
  • the client semantic query processor will use the properties of the profile (specifically the subscribed knowledge communities (agencies) in that profile) to invoke the lequest.
  • Figure 15 illustrates the semantic browser showing two profiles (the default profile named "My Profile” and 15Aand a profile named “Patents” 15B). Observe how the user is able to navigate his/her knowledge worlds via both profiles without interference.
  • Figures 16A-C illustrate how a user would configure a profile (to create a profile, the user will use the "Create Profile Wizard" and the profile can then be modified via a property sheet as shown).
  • Figure 17 shows how a user would select a profile when creating a request with the "Create Request Wizard.”
  • a color theme and animation theme applied to a style theme yields a "smart style”.
  • Smart in this context means the style is adaptive or responsive to the mood of its request, context panes, preview mode, handheld mode, live mode, slideshow mode, Screensaver mode, ' blender/collection mode, accessibility, user settings recognition, and possibly other variables within the system (see below).
  • the preferred embodiment comprises at least the following style Classes:
  • Implicit and Dynamic Smart Style Properties a. Mood - the smart style must convey the mood of the request (i.e., the request is a parameter passed to the smart style). This will involve semantic images, semantic motions, Visualizations, etc. that convey the semantically informed or semantically determined properties of the smart request (the context template or information type, the categories, whether there are filters (e.g., local documents), the information types of those filters, etc.) b. Context panes - e.g., deep info pane (per object), dockable preview panes, dockable contextual PU? watch groups/panes, etc. c. Preview Mode - each smart style must be able to display its results for preview (in a small window). d.
  • Handheld Mode - each smart style must be able to display its results optimized for a handheld device.
  • Live mode each smart style must have a "live” mode during which it would display real-time semantic Visualizations (per object). This can be toggled on or off (e.g. if the user does not want real-time semantic Visualizations, or to save bandwidth that results from real-time Web service calls per object).
  • Slideshow mode preferably, each smart style must be able to "play” the results of the request - like a live stream.
  • Screensaver mode preferably, each smart style must be able to "play” the results of the request as a screensaver. This is a variant of slideshow mode, except in fullscreen theater mode.
  • Blender/collection mode preferably, each smart style must change its UI appropriately if the request it is displaying is a blender/collection.
  • Accessibility preferably, each smart style must support accessibility.
  • User settings recognition - the Nervana Librarian will allow users to indicate whether they are beginners, moderate users, or power-users, and their respective job function s) (R&D, sales, marketing, executive, etc.).
  • each smart style considers (or is influenced by) these functions where appropriate.
  • each smart style is responsible, consistent with the semantics of the request, for recognizing (or discerning or perceiving) and then Visualizing (or presenting or depicting or illustrating, consistent with what should deserve the user's attention): the Mood of the Current Request (including semantic images, motion, chrome, etc. a Change in the number of Items in the Current Request the Mood of each object (intrinsically) the Mood of each object's context (headlines, breaking news, experts, etc.)
  • the characteristic is on a gradient or continuum, perceiving the relative placement along it (e.g., how breaking is breaking news?, how critical are the headlines? what is the level of expertise for the experts?, etc.) a change in each object's context (there is new breaking news, there are new annotations, etc.) the RELATIVE criticality of each object being displayed (different sized view ports, different fonts, different chrome, etc.) a request navigation and "loading" status (interstitials that INTRODUCE the mood of the new request being loaded)
  • Figure 18 shows a screenshot with the 'Smart Styles' Dialog Box illustrating some of the foregoing operations and features.
  • the Dialog Box allows the user to browse smart styles by pivoting across style classes, style themes, color themes, and animation themes.
  • a preview window shows the user a preview of the currently selected smart style.
  • Smart Request Watch refers to a feature of the information Nervous System that allows users of the semantic browser (the Info ⁇ nation Agent or the Librarian) to monitor (or “watch") smart requests in parallel. This is a very advantageous feature in that it enhances productivity by allowing users to track several requests at the same time.
  • the feature is implemented in the client-side semantic runtime, the semantic browser, and skins that allow a configurable way of watching smart requests (via a mechanism similar to "Picture-fri-Picture” (PIP) functionality in television sets).
  • PIP Picture-fri-Picture
  • one or more of the following software components are used:
  • the Watch Window 2. Request Watch Lists (RWLs) and Groups (RWGs)
  • the Request Watch List is a list of smart requests (or smart agents) that the client runtime manages. This list essentially comprises the smart requests the user wishes to monitor.
  • the Request Watch List comprises a list of entries, the Request Watch List Entry (RWLE) with the following data structure:
  • the Request Watch List contains an array or vector of RWLE structures.
  • the Request Watch List Manager manages the RWL.
  • the semantic browser provides a user interface that allows the user to add smart requests to the RWL - the UI talks to the RWLM to add and remove RWLEs to/from the RWL.
  • the RWL is stored (and persisted) centrally by the client-side semantic runtime (either as an XML file-based representation or in a store like the Windows registry).
  • the RWL can also be populated by means of Request Watch Groups (RWGs).
  • RWGs Request Watch Groups
  • a Request Watch Group provides a means for the user to monitor a collection of smart requests. It also provides a simple way for users to have the semantic browser automatically populate the RWL based on configurable criteria.
  • RWGs There are at least two types of RWGs: Auto Request Watch Groups and the Manual Request Watch Group.
  • Auto Request Watch Groups are groups that are dynamically populated by the semantic browser depending on the selected profile, the profile of the currently displayed request, etc.
  • the Manual Request Watch Group allows the user to manually populate a group of smart requests (regular smart requests or blenders) to monitor as a collection.
  • the Manual Request Watch Group also allows the user to add support context types (e.g., documents, categories, text, keywords, entities, etc.) - in this case, the system will dynamically generate the semantic query (SQML) from the filter(s) and add the resulting query to the Manual Request Watch Group.
  • SQL semantic query
  • Auto-RWGs for one or more configurable profiles, including "All Profiles" as shown in the Smart Request Watch Dialog Box in Figure 19:
  • Breaking News this tells the semantic browser to automatically add a Breaking News smart request to the RWL (for the selected profile(s)).
  • Headlines this tells the semantic browser to automatically add a Headlines smart request to the RWL (for the selected profile(s)).
  • Categorized Breaking News this tells the semantic browser to automatically add Categorized Breaking News smart requests to the RWL (for the contextual profile).
  • the semantic browser will dynamically add smart requests with category filters corresponding to each subcategory of the currently displayed smart request (and for the contextual or current profile) - if the currently displayed smart request has categories. For example, if the smart request "Breaking News" about Technology" is currently being displayed in a semantic browser instance, and if the category "Technology” has 5 sub-categories (e.g., Wireless, Semiconductors, Nanotechnology, Software, and Electronics), the following smart requests will be dynamically added to the RWL when the current smart request is loaded:
  • the RWLEs for these entries will be initialized with the RequestViewInstancelD of the current semantic browser instance. If the user navigates to a new smart request, the categorized Breaking News for the previously loaded smart request will be removed from the
  • Categorized Headlines this tells the semantic browser to automatically add Categorized Headlines smart requests to the RWL (for the contextual profile). This is similar to Categorized Breaking News, except that Headlines are used in this case. The user will then be able to monitor Categorized Headlines smart requests as a watch group or collection.
  • Categorized Newsmakers this tells the semantic browser to automatically add Categorized Newsmakers smart requests to the RWL (for the contextual profile). This is similar to Categorized Breaking News, except that Newsmakers are used in this case. The user will then be able to monitor Categorized Newsmakers smart requests as a watch group or collection.
  • My Favorite Requests this tells the semantic browser to automatically add all favorite smart requests to the RWL (for the selected profile(s)). This allows the user to watch or monitor all his/her favorite smart requests as a group.
  • My Favorite Breaking News this tells the semantic browser to automatically add all favorite breaking news smart requests to the RWL (for the selected profile(s)). This allows the user to watch or monitor all his/her favorite breaking news smart requests as a group.
  • My Favorite Newsmakers this tells the semantic browser to automatically add all favorite newsmakers smart requests to the RWL (for the selected profile(s)). This allows the user to watch or monitor all his/her favorite newsmakers smart requests as a group.
  • FIG 19 illustrates the "Smart Request Watch” Dialog Box in the semantic browser of the preferred embodiment.
  • the top half of the dialog is used to add auto-watch groups.
  • the user can select auto-watch group types and profile types ("All Profiles,” "Contextual Profile,” and the actual profile names) and add them to the auto-watch-group list.
  • the user can also remove auto- watch-groups.
  • the bottom half of the dialog box is used to add/remove smart requests to/from the manual watch group.
  • the Notification Manager is a component of the semantic runtime client that monitors smart requests in the RWL.
  • the NM has a thread that periodically invokes each smart request in the RWL (via the client semantic query processor) and updates the RWLE with the "results count” and the "last update time.” i the preferred embodiment the NM preferably invokes the smart requests every 5-30 seconds.
  • the NM can intelligently adjust the periodicity or frequency of request checks depending on the size of the RWL (in order to minimize bandwidth usage and the scalability impact on the Web service).
  • the NM For time-sensitive smart requests (like Breaking News, Headlines, and Newsmakers), the NM preferably invokes the smart request without any additional time filter. However, for non time-sensitive requests (like for information as opposed to context types or for non time-sensitive context templates like Favorites and Recommendations), the NM preferably invokes the query for the smart request with a time filter (e.g., the last 10 minutes).
  • a time filter e.g., the last 10 minutes.
  • the semantic runtime client manages what the inventor calls Watch Group Monitors (WGM). For each watch group the user has added to the watch group list, the client creates a watch group monitor. A watch group monitor tracks the number of new results in each request in its watch group. The watch group monitor creates a queue for the RWLEs in the watch group that have new results. The WGM manages the queue in order to maximize the freshness of the results. The WGM periodically polls the NM to see whether there are new results for each request in its watch group. If there are, it adds the request to the queue depending on the 'last result time' of the request. It does this in order to prioritize requests with the freshest results first.
  • WGM Watch Group Monitors
  • the currently displayed visual style (skin) running in the Presenter would then call the semantic runtime OCX to dequeue the requests in the WGM queue. This way, the request watch user interface will be consistent with the existence of new results and the freshness of the results. Once there are no more new results in the currently displayed request, the smart style will dequeue the next request from the WGM queue.
  • the Watch Pane refers to a panel that gets displayed in the Presenter (alongside the main results pane) and which holds visual representations of the user's watch groups.
  • the WP allows the user to glance at each watch group to see whether there are new results in its requests.
  • the WP also allows the user to change the current view with which each watch group's real-time status gets displayed.
  • the following views are currently defined:
  • Ticker View this displays the total number of new results in all the watch group's smart requests but also shows an animation that sequentially displays the number of new results in each smart request (as a ticker).
  • Preview View this is similar to the ticker view except that the most recent result per smart request is also displayed alongside the number of new results in the ticker.
  • the WP displays the total number of new results in all the watch group's smart requests along with a ticker that shows the number of new results in each smart request and a slide-show of all the new results per smart request.
  • the WP also allows the user to watch a watch group. The user will do this by selecting one of the watch groups in the WP and dragging it into the main results pane (or by a similar technique).
  • This WW resembles or can be analogized to TV's picture-in-picture functionality in appearance or layout, but differs in several ways, most noticeably in that in this case the displayed content is comprised of semantic requests and results as opposed to television channels are being "watched.” Of course, the underlying technology generating the content is also quite different.
  • the WW can be displayed in any of the aforementioned views. When the WW is in Deep View however, the WW's view controls are displayed. The following controls are currently defined:
  • Stop, Play, Seek, FF, RW, Speedup - these allow the user to stop, play, seek, fast- forward, rewind or speedup the "watch group request stream.” For instance, a fast-forward will advance to several requests ahead of the currently displayed one.
  • Results controls this allows the user to control the results in each request in the watch group. Essentially, the results are a stream within a stream and this will also allow the user to control the results in the current request in the current watch group.
  • FIG 20 illustrates a Watch Window displaying Filtered Smart Requests (e.g.,. Headlines on Wireless).
  • Figure 20 is an Illustration of the Watch Window with a Current Smart Request Title (e.g., "Breaking News").
  • the Watch List can be named "News Watch.”
  • the user will be asked to add/remove requests, objects, keywords, text, entities, etc. to/from the "News Watch.”
  • the "News Watch” can be viewed with a Newsstand watch pane. This will provide a spatially- oriented view of the user's requests and dynamically-created requests (via objects added to the Watch List, and created dynamically by the runtime using those objects as filters) - not unlike the view of a news-magazine rack when one walks into a Library or Bookstore.
  • Entities are a very powerful feature of the preferred embodiment of the Information Nervous System. Entities allow the user to create a contextual definition that maps to how they work on a regular basis. Examples of entities include:
  • Entities could include drags, drag interaction issues, patents, FDA clinical trials, etc.
  • an entity is a semantic envelope that is a smart contextual object. An entity can be dragged and dropped like any other smart object. However, an entity is represented by SQML and not SRML (i.e., it is a query-object because it has much richer semantics). An entity can be included as a parameter to a smart request. The user creates entities based on his/her tasks. Entities in the preferred embodiment contain at least the following information (in alternate embodiments they could contain more or less information):
  • Contextual resources these could include keywords, local documents, Internet documents, or smart objects (such as people).
  • An entity can be opened in the semantic browser, can be used as a pivot for navigation, as a parameter for a smart request (e.g., Headlines on My Project), can be dragged and dropped, can be copied and pasted, can be used with the smart lens, can be visualized with a smart style, can be used as the basis for an intrinsic alert, can be saved as a .ENT document, can be emailed, shared, etc.
  • a smart request e.g., Headlines on My Project
  • the semantic runtime client dynamically creates SQML by appending the rich metadata of the entity to the subject of the relational request to create a new rich SQML that refers to the entity.
  • Gene Project is an internal project, it would likely not exist in a public taxonomy which could be used with the semantic browser of this the preferred embodiment of my invention.
  • the researcher could create an entity named "Gene Project”, typed as a Project, and could then initialize the entity by scoping it to Genomics (which exists in broad taxonomies) and then also qualifying it with the keyword-phrase "Gene Project” (using the AND operator).
  • the server-side semantic query processor will interpret this (by mapping the SQML to the semantic network) as "Experts on any information that belongs to the category Genomics AND which also includes the phrase "Gene Project.”
  • Entities also allow the user to create a dynamic taxonomy - public taxonomies are very static and are not updated regularly. With entities, the user can "extend" his/her private taxonomy dynamically and at the speed of thought. Knowledge is transferred at the speed of thought. Entities allow the user to create context with the same speed and dynamism as his/her mind or thought flow. This is very significant. For instance, the user can create an entity for a newly scheduled meeting, a just-discovered conference, a new customer, a newly discovered competitor, etc. - ALL AT THE SPEED OF THOUGHT. Taxonomies don't allow this.
  • Taxonomies assume that topics are the only source of context. With entities, a user can create abstract contextual definitions that include - but are not limited to - topics. Examples include people, teams, events, companies, etc. Entities might eventually "evolve” into topics in a taxonomy (over time and as those entities gain “fame” or “notoriety") but in the "short-term," entities allow the user to create context that has not yet evolved (or might never evolve) into a full-blown taxonomic entry. For instance, Nervana (our company) was initially an entity (known only to itself and its few employees) but as we have grown and attracted public attention, as an entity we are evolving into a topic in a public taxonomy. With entities, users don't have to wait for context (like Nervana) to "eventually become” topics.
  • Entities allow the user to create what the inventor calls "compound context.”
  • An example of this is a meeting.
  • a meeting typically involves several participants with documents, presentation slides, and/or handouts relevant to the topic of discussion.
  • entities in the Information Nervous System a user can create a "meeting" context that captures the semantics of the meeting.
  • the Create Entity Wizard the user can specify that the entity is a meeting, and then specify the semantic filters.
  • the Presenter of the meeting might want to create an entity in order to track knowledge specifically relevant to the meeting. For instance, he/she might want to do this to determine when to schedule a follow-up meeting or to track specific action items relating to the meeting.
  • the user would add the email addresses of the participants, the handed out documents, and also the presentation to the entity filter definition.
  • the user then saves the entity which is then created in the semantic namespace/environment.
  • the user can then edit the entity with new or removed filters (and/or a new name/description) at a later date/time - for instance, if he/she has discovered new documents that would have been relevant to the meeting.
  • the semantic browser When the user drags and drops the entity or includes it in a request/agent, the semantic browser then compiles the entity and includes it in a master SQML with the sub-queries also passed to the XML Web Service for interpretation.
  • the server-side semantic query processor then processes the compound SQML by constructing a series of SQL sub-queries (or an equivalent) and by joining these queries with the entity sub-queries which in turn are generated using SQL sub-queries.
  • the user can use an AND or OR (or other) operator to indicate how the entity filters should be applied. For instance, the user can indicate that the meeting (semantically) is the participants of the meeting AND the documents/slides handed out during the meeting.
  • the SQML equivalent is used to interpret the entity (with the desired operator). This is very powerful. It means that the user can define an entity named "Project Meeting” and drag and drop that entity to the special agent named "Breaking News.” This then creates a request named "Breaking News on Project Meeting” (with the appropriate SQML referring to the identifier of the entity- which will then be compiled into sub- SQML before it is passed to the server(s) for interpretation.
  • the server then applies default predicates to the entries in the entity (based on what "makes sense” for the object). In this particular example, because of the definition of the entity, the server will then only return:
  • Entities can include other entities to allow for compound entities. For instance, if an entire team of people were involved in the meeting, the Presenter might want to create an entity that includes an email distribution list of those people, hi this case, the user might search the Information Nervous System for the distribution list and then save the result as an entity.
  • the browser will allow the user to save results as entities and based on the result type, it will automatically create an entity with a default entity type that "makes sense.” For instance, if the user saves a document result as an entity, the semantic browser it will create a "Topic" entity. If the user saves a Person result as an entity, the semantic browser will create a "Person” entity. If the user saves an email distribution list as an entity, the semantic browser will create a "Team" entity.
  • the user can save a Person result as a Person entity and then drag and drop that entity into the Project Meeting entity.
  • the Team entity that maps to the email distribution list of the meeting participants can be dragged and dropped to the Project Meeting entity.
  • the user can then create a request called "Headlines on Project Meeting" that includes the entity.
  • the semantic query processor will then return Headlines BY anyone in the email distribution list (using the right default predicate) and which is semantically relevant to ALL the handouts given out during the meeting.
  • a Dossier (Guide) on the Project Meeting will return All Bets on the meeting, Best Bets on meeting, Experts on the meeting, etc.
  • Entity A refers to Entity B and the user attempts to delete Entity B
  • the semantic browser will detect this and flag the user that Entity B has an outstanding reference. If the user deletes Entity B anyway, the reference in Entity A (and any other references to Entity B) will get removed. Alternately, in some embodiments, the user could be prohibited (whether informed or not) from deleting Entity B in the same situation, based on permissions of others within an organization associated with the entity.
  • Popular entities can also be shared amongst members of a knowledge community. Like other items in the semantic browser (like requests or knowledge communities (agencies), entities can be saved as files (so the user can later open them or email them to colleagues, or save them on a central file share, etc.).
  • a common scenario would be that the corporate Librarians at businesses would create entities that map to internal projects, meetings, seminars, tasks, and other important corporate knowledge items of interest. These entities would then be saved on a file-share or other sharing mechanism (like a portal or web-site) or on a knowledge community (agency). The knowledge workers in the organization would then be able to use the entities.
  • the Librarians can and will automatically edit their context and users will be able refresh or synchronize to the new entities. Entities could also and alternately be shared on a peer-to-peer basis by individual users. This is akin to a legal peer-to-peer file sharing for music, but instead of music, what is shared is context to facilitate meaning, or more meaningful communication.
  • Portfolios are a special type of entity that contains a collection of entities.
  • an entity can be of any size or composition, and portfolio can contain any kind or number of entities, a portfolio would not contain other portfolios.
  • a portfolio allows the user to manage a group of entities as one unit.
  • a portfolio is a first-class entity and as such has all the aforementioned features of an entity.
  • the OR qualifier is applied (by default) to its containing entities, hi other words, if Portfolio P contains entities El and E2, a smart request titled 'Headlines on P' will be processed as 'Headlines on El or E2.' The user can change this setting on individual smart requests (to an AND qualifier).
  • a pharmaceuticals 'patent' entity could include the categories of the patent, relevant keywords, and relevant documents.
  • a CIA agent could create a 'terrorist' entity to track terrorists. This could include categories on terrorism, suspicious wire transfers, suspicious arms sales, classified documents, keywords, and terrorism experts in the information community.
  • the Nervana semantic browser will allow the user to subscribe and unsubscribe to/from knowledge communities (agencies) for a given profile. These knowledge communities will be readily available to the user underneath the profile entry in the semantic environment. In addition, these l ⁇ iowledge communities will be queried by default for intrinsic alerts, context panels, and etc. whenever results are displayed for any request created using the same profile.
  • the semantic environment includes state indicating the subscribed knowledge communities for each profile.
  • the client-side semantic query processor uses this information for dynamic requests that start from results for requests of a given profile (the SQP will ask the semantic runtime client for the knowledge communities for the profile and then issue XML Web Service calls to those knowledge communities as appropriate).
  • Figures 22A and 22B show the user interface for the knowledge community subscription and un-subscription.
  • the dialog box has combo boxes allowing the user to filter by profile, to view all, new, subscribed, suggested, and un-subscribed communities, by industry and area of interest, by keywords, by publishing point (all publishing points, the local area network, the enterprise directory, and the global knowledge community directory), and by creation time (anytime, today, yesterday, this week, and last week).
  • the semantic runtime client queries the publishing point endpoint listeners (for each publishing point) using the filters. It then gathers the results and displays them in the results pane. The user is also able to view the categories of each knowledge community in the results pane via a combo box.
  • Figure 20B illustrates the bottom portion of the Knowledge Communities Dialog Box.
  • SQL Semantic Query Markup Language
  • the Nervana Semantic DHTML Behavior is an Internet Explorer DHTML Behavior that, from the client's perspective, every thing it understands as a query document.
  • the client opens 'query documents,' in a manner resembling how a word processor opens 'textual and compound documents.
  • the Nervana client is primarily responsible for processing a Nervana semantic query document and rendering the results.
  • a Nervana semantic query document is expressed and stored in form of the Nervana Semantic Query Markup Language (SQML). This is akin to a "semantic file format.”
  • SQL Nervana Semantic Query Markup Language
  • the SQML semantic file format comprises of the following:
  • Head - The 'head' tag like in the case of HTML, includes tags that describe the document.
  • RequestType - This indicates the type of request. It can be "smart request"
  • “dumb request” (indicating requests to one or more local or network resources).
  • ObjectType - This fully qualifies the type of objects returned by the query.
  • CreationTime The creation time of the document.
  • LastModifiedTime The last modified time of the document.
  • LastAccessedTime The last accessed time of the document.
  • Attributes The attributes of the document, if any.
  • Version - this indicates the version of the query. This allows the web service's semantic query processor to return results that are versioned. For instance, one version of the browser can use VI of a query, and another version can use V2.
  • Targets - This indicates the names and the URLs of the information community web services that the query document targets.
  • Type - this indicates the type of targets. This can be "targetentries,” in which case the tag includes sub-tags indicating the actual web service targets, or
  • Each "category" entry contains a name attribute and a URI attribute that indicates the URL of the Knowledge Domain Server (KDS) from which the category came.
  • KDS Knowledge Domain Server
  • Type - this indicates the type of categories. This can be either “categoryentri.es,” in which case the sub-tag refers to the list of category entries, "allcategories,” in which case all categories are requested from the information community web services, or "myfavoritecategories,” in which case the query processor gets the user's favorite categories and then generates compiled SQML that contains these categories (this compiled SQML is then sent to the server(s)).
  • Resource - The reference to the 'dumb' resource being queried. Examples include file paths, URLs, cache entry identifiers, etc. These will be mapped to actual resource managers components by the interpreter.
  • Type - The type of resource reference, qualified with the namespace.
  • defined resource reference types are: nervana:url (this indicates that the resource reference is a well-formed standard Internet URL, or a custom Nervana URL like
  • nervana filepath (this indicates that the resource reference is a path to a file or directory on the file-system), and nervana amespaceref (this indicates that the resource comes from the client semantic namespace).
  • Uri This indicates the universal resource identifier of the resource. In the case of paths and Internet URLs, this indicates the URL itself. In the case of namespace entries, this indicates the GUID identifier of the entry.
  • Type - this indicates the type of links. This can be "linkentries,” indicating the links are explicit entries.
  • the predicate "nervana:relevantto” indicates that the query is "return all objects from the resource R that are relevant to the object O," where R and O and the specified resource and object, respectively.
  • Other examples of predicates include nervana:reportsto, nervana:teammateof, nervana:from, nervana:to, nervanaxc, nervana:bcc, nervana: attachedto, nervana:sentby, nervana: sentto, nervana:postedon, nervana ontainstext, etc.
  • Examples include standard XML data types like xmkstring, xmkinteger, Nervana equivalents of same, custom Nervana types like nervana: datetimeref (which could refer to object references like 'today' and 'tomorrow'), and any standard Internet URL (HTTP, FTP, etc.) or Nervana URL (objects://, etc.) that refers to an object that Nervana can process as a semantic XML object.
  • MetadataEntry this indicates the details of a metadata entry.
  • SQML is generated in any one or more of several possible ways:
  • SQML Parsing hi some embodiments in some situations, SQML that gets created on the client might not be ready (in real-time) for remote consumption - by the server's XML web service or at another machine site. This is especially likely to be the case when the SQML refers to local context such as documents, Entities, or Smart Requests (that are identified by unique identifiers in the semantic environment). 1
  • the client generally creates SQML that is ready for remote consumption. Preferably, it does this by caching the metadata for all references in the metadata section of the document. This is preferable because in some cases, the resource or object to which the reference points might no longer exist when the query is invoked.
  • a user might drag and drop a document from the Internet to a smart request in order to generate a new relational request.
  • the client extracts the metadata (including the summary) from the link and inserts the metadata into the SQML. Because the resolution of the query uses only the metadata, the query is ready for consumption once the metadata is inserted into the SQML document.
  • the link that the object refers to might not exist the day after the user found it. In such a case, even if the user invokes the relational request after the link might have ceased to exist, the request will still work because the metadata would already have been cached in the SQML.
  • the client SQML parser performs "lazy" updating of metadata in the SQML.
  • the request When the request is invoked, it attempts to update the metadata of all parameters (resources, etc.) in the SQML to handle the case where the objects might have changed since they were used to create
  • Blenders contain references to smart requests. the relational request. If the object does not exist, the client uses the metadata it already has. Otherwise, it updates it and uses the updated metadata. That way, even if the object has been deleted, the user experience is not interrupted until the user actually tries to open the object from whence the metadata came.
  • the Nervana Semantic Runtime Control is an ActiveX control that exposes properties and methods for use in displaying semantic data using the Nervana semantic user experience.
  • the control will be primarily called from XSLT skins that take XML data (using the SRML schema) and generate DHTML+TLME or SVG output, consistent with the requirements of the Nervana semantic user experience.
  • the Nervana control encapsulates the "SDK" on top of which the XSLT skins sit in order to produce a semantic content-driven user experience.
  • the APIs listed below illustrate the functionality that will be exposed or made available by the final API set in the preferred embodiment.
  • Nervana Semantic Runtime Control API a. EnumObjectsInNamespacePath
  • the EnumObjectsInNamespacePath method returns the objects in a namespace path.
  • the CompileSemanticQueryFromBuffer method opens an SQML buffer and compiles it into one or more execution-ready SQML buffers. For instance, an SQML file containing a blender will be compiled into SQML buffers representing each blender entry. If the blender contains blenders, the blenders will be unwrapped and an SQML buffer will be returned for each contained blender.
  • a compiled or "execution-ready" SQML buffer is one that can be semantically processed by an agency. The implication is that a blender that has agents from multiple agencies will have its SQML compiled to buffers with the appropriate SQML from each agency.
  • a Nervana client application for instance, the semantic browser or a Nervana skin will call this method to compile an SQML buffer and retrieve generated "compiled code" that is ready for execution.
  • the application or skin will compile an SQML buffer and then prepare frame windows where it wants each individual SQML query to sit. It can then issue individual SQML semantic calls by calling OpenSemanticQueryFromBuffer and then have the results displayed in the individual frames.
  • the OpenSemanticQueryFromBuffer method opens an SQML buffer and asynchronously fires the XML results (in SRML) onto the DOM, from whence a Nervana skin can sink the event. Note that in this embodiment the SQML has to be "compiled" and ready for execution. If the SQML is not ready for execution, the call will fail. To compile an SQML buffer, call CompileSemanticQueryFromBuffer.
  • a Nervana client application for instance, the semantic browser
  • a Nervana skin will call this method to open a compiled SQML buffer.
  • the GetSemanticQueryBufferFromFile method opens an SQML file, and returns the buffer contents.
  • the buffer can then be compiled and/or opened.
  • a Nervana client application for instance, the semantic browser or a Nervana skin will call this method to convert an SQML file into a buffer before processing it.
  • the GetSemanticQueryBufferFromNamespace method opens a namespace object, and retrieves its SQML buffer.
  • a Nervana client application for instance, the semantic browser or a Nervana skin will call this method to open an SQML buffer when it already has access to the id and path of the namespace object.
  • the GetSemanticQueryBufferFromURL method wraps the URL in an SQML buffer, and returns the buffer.
  • a Nervana client application for instance, the semantic browser or a Nervana skin will call this method to convert an URL of any type to SQML.
  • This can include file paths, HTTP URLs, FTP URLs, Nervana agency object URLs (prefixed by "wsobject://”) or Nervana agency URLs (prefixed by "wsagency ://").
  • the GetSemanticQueryBufferFromClipboard method converts the clipboard contents to SQML, and returns the buffer.
  • a Nervana client application for instance, the semantic browser or a Nervana skin will call this method to get a semantic query from the clipboard.
  • the application can then load the query buffer.
  • the Stop method stops current open request.
  • the Refresh method refreshes the current open request.
  • a Nervana client application for instance, the semantic browser
  • a Nervana skin will call this method to refresh the currently loaded request.
  • the CreateNamespaceObject method creates a namespace object and returns its GULD.
  • a Nervana client application for instance, the semantic browser or a Nervana skin will typically call this method to create a temporary namespace object when a new query document has been opened.
  • the DeleteNamespaceObject method deletes a namespace object.
  • a Nervana client application for instance, the semantic browser
  • a Nervana skin will typically call this method to delete a temporary namespace object.
  • the CopyObject method copies the semantic object to the clipboard as an SQML buffer using a proprietary SQML clipboard format.
  • the object can then be "pasted" onto agents for relational semantic queries, or used as a lens over other objects or agents.
  • a Nervana skin will typically call the CopyObject method when the user clicks on the
  • the CanObjectBeAnnotated method checks whether the given object can be annotated.
  • a Nervana skin will typically call the CanObjectBeAnnotated method to determine whether to show UI indicating the "Annotate" command.
  • the AnnotateObject method invokes the currently installed email client and initializes it to send an email annotation of the object to the email agent of the agency from whence the object came.
  • a Nervana skin will typically call the AnnotateObject method when the user clicks on the "Annotate" menu option - off a popup menu on the object.
  • a Nervana skin will typically call the CanObjectBePublished method to determine whether to show UI indicating the "Publish" command.
  • the PublishObject method invokes the currently installed email client and initializes it to send an email publication of the object to the email agent of the agency from whence the object came.
  • a Nervana skin will typically call the PublishObject method when the user clicks on the "Publish" menu option - off a popup menu on the object.
  • OpenObjectContents method opens the object using an appropriate viewer. For instance, an email object will be opened in the email client, a document will be opened in the browser, etc..
  • a Nervana skin will typically call the OpenObjectContents method when the user clicks on the "Open" menu option - off a popup menu on the object.
  • the SendEmailToObject method is called to send email to a person or customer object.
  • the method opens the email client and initializes it with the email address of the person or customer object.
  • a Nervana skin will typically call the SendEmailToObject method when the user clicks on the "Send Email" menu option - off a popup menu on a person or customer object.
  • the GetObj ect Annotations method is called to get the annotations an object has on the agency from whence it came.
  • a Nervana skin will typically call the GetObj ectAnnotations method when it wants to display the titles of the annotations an object has - for instance, in a popup menu or when it wants to display the annotations metadata in a window.
  • the IsObjectMarkedAsFavorite method is called to check whether an object is marked as a favorite on the agency from whence it came.
  • a Nervana skin will typically call the IsObjectMarkedAsFavorite method to determine what UI to show - either the "Mark as Favorite” or the "Unmark as Favorite” command. If the object cannot be marked as a favorite (for instance, if it did not originate on an agency), the error code E_LNVALIDARG is returned.
  • the MarkObj ect AsFavorite method is called to mark the object as a favorite on the agency from whence it came.
  • a Nervana skin will typically call the MarkObj ect AsFavorite method when the user clicks on the "Mark as Favorite" command.
  • UnmarkObjectAsFavorite method is called to unmark the object as a favorite on the agency from whence it came.
  • the IsSmartAgentOnClipboard method is called to check whether a smart agent has been copied to the clipboard.
  • a Nervana skin will typically call the IsSmartAgentOnClipboard method when it wants to toggle the user interface to display the "Paste” icon or when the "Paste" command is invoked.
  • the GetSmartLensQueryBuffer method is called to get the query buffer of the smart lens.
  • a Nervana skin will typically call the GetSmartLensQueryBuffer method when the user hits "Paste as Smart Lens” to invoke the smart lens off the smart agent that is on the clipboard.
  • the OpenObjectContents method opens the object using an appropriate viewer. For instance, an email object will be opened in the email client, a document will be opened in the browser, etc.
  • a Nervana skin will typically call the OpenObjectContents method when the user clicks on the "Open" menu option - off a popup menu on the object.
  • Email Control APIs a. Email_GetFromLink bjects
  • the Email_GetFromLinkObjects method is called to get the metadata for the "From" links on an email object from the agency from whence it came.
  • a Nervana skin will typically call the Email_GetFromLinkObjects method when it wants to navigate to the "From" list from an email object, or to display a popup menu with the name of the person in the "From" list.
  • the Email_GetFromLinkObjects method is called to get the metadata for the "To" links on an email object from the agency from whence it came.
  • a Nervana skin will typically call the Email_GetToLinkObjects method when it wants to navigate to the "To" list from an email object, or to display a popup menu with the name of the person in the "To" list.
  • the Email_GetCcLinkObjects method is called to get the metadata for the "CC" links on an email object from the agency from whence it came.
  • a Nervana skin will typically call the Email_GetCcLinkObjects method when it wants to navigate to the "CC" list from an email object, or to display a popup menu with the name of the person in the "CC" list.
  • the Email_GetBccLinkObjects method is called to get the metadata for the "BCC" links on an email object from the agency from whence it came.
  • a Nervana skin will typically call the Email_GetBccLinkObjects method when it wants to navigate to the "BCC" list from an email object, or to display a popup menu with the name of the person in the "BCC” list.
  • Email_Get AttachmentLinkObj ects method is called to get the metadata for the
  • a Nervana skin will typically call the Email_Get AttachmentLinkObj ects method when it wants to navigate to the "Attachments" link from an email object, or to display a popup menu with the titles of the attachments in the "Attachments" list.

Abstract

The present invention is directed to an integrated implementation framework and resulting medium for knowledge retrieval, management, delivery and presentation. The system includes a first server component that is responsible for adding and maintaining domain- specific semantic information (item 50) and a second server component (iem 80) that hosts semantic and other knowledge for use by the first server component that work together to provide text and time- sensitive semantic information retrieval services to clients operating a presentation platform via a communication medium (item 10). Within the system, all objects or events given hierarchy are active Agents (item 90) semantically related to each other and representing queries (comprised of underlying action code) that return data objects for presentation to the client according to a predetermined and customizable theme or “Skin”. This system provides various means for the client to customize and “blend” Agents and the underlying related queries to optimize the presentation of the resulting information (item 30).

Description

SYSTEM AND METHOD FOR SEMANTIC KNOWLEDGE RETRIEVAL, MANAGEMENT, CAPTURE, SHARING, DISCOVERY, DELIVERY AND
PRESENTATION
INVENTOR
Nosa Omoigui PRIORITY CLAIM This application is a Continuation-In-Part of U.S. Application Serial No. 10/179,651 filed June 24, 2002, which claims priority to U.S. Provisional Application No. 60/360,610 filed February 28, 2002 and to U.S. Provisional Application No. 60/300,385 filed June 22, 2001. This Application also claims priority to U.S. Provisional Application No. 60/447,736 filed February 14, 2003. This Application also claims priority to PCT/US02/20249 filed June 24, 2002. All of the foregoing applications are hereby incorporated by reference in their entirety as if fully set forth herein.
COPYRIGHT NOTICE
This disclosure is protected under United States and International Copyright Laws. © 2002 - 2004 Nosa Omoigui. All Rights Reserved. A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
FIELD OF THE INVENTION
This invention relates generally to computers and, more specifically, to information management and research systems.
BACKGROUND OF THE INVENTION
The general background to this invention is described in my co-pending parent application (U.S. Application Serial No. 10/179,651 filed June 24, 2002), which is incorporated by reference herein, and of which this application is a Continuation in Part.
SUMMARY OF THE INVENTION
The present invention is directed in part to a semantically integrated knowledge retrieval, management, delivery and presentation system, as is more fully described in my co-pending parent application (U.S. Application Serial No. 10/179,651 filed June 24, 2002). The present invention and system includes several additional improved features, enhancements and/or properties, including, without limitation, Entities, Profiles and Semantic Threads, as are more fully described in the Detailed Description below.
BRIEF DESCRIPTION OF THE DRAWINGS
The preferred and alternative embodiments of the present invention are described in detail below with reference to the following drawings.
FIGURE 1 is a partial screenshot overview and FIGURE 2 is an expansion of a dialog box of FIGURE 1 for a scenario of a Patent Examiner using the preferred embodiment in a prior art search, a screenshot of where "Magnetic Resonance imaging" occurs in a Pharmaceuticals taxonomy. FIGURE 3 shows the Sharable Smart Request System Interaction, which is the binary document format that encapsulates the SQML buffer with the smart request and also illustrates how the extension handler opens a document.
FIGURE 4A is a partial screenshot overview of document files.
FIGURE 4B shows an illustration of two .REQ documents from FIGURE 4A (titled 'Headlines on Reuters Related to My Research Report (Live)' and 'Headlines on Reuters (as of January 21 2003, 08 17AM)' on the far right) with a registered association in the Windows shell.
FIGURE 5 is a Diagram Illustrating the Text-to-Speech Object Skin and shows an illustration of an email message being rendered via a text-to-speech object skin.
FIGURE 6 is a Diagram Illustrating a Text-to-Speech Request Skin.
FIGURE 7 is a Diagram Illustrating Knowledge Modeling for a Pharmaceuticals Company Example.
FIGURE 8 is a Diagram Illustrating Client Component Integration and Interaction Workflow.
FIGURES 9 - 11 show three different views of the Explore Categories dialog box.
FIGURES 12 and 13 show sample screenshots of the Dossier Smart Lens in operation.
FIGURE 14 shows how the server-side semantic query processor processes incoming semantic queries (represented as SQML).
FIGURE 15 illustrates the semantic browser showing two profiles (the default profile named "My Profile" and a profile named "Patents"). Observe how the user is able to navigate his/her knowledge worlds via both profiles without interference.
FIGURE 16A-C illustrate how a user would configure a profile (to create a profile, the user will use the "Create Profile Wizard" and the profile can then be modified via a property sheet as shown in other Figures).
FIGURE 17 shows how a user would select a profile when creating a request with the "Create Request Wizard." FIGURE 18 shows a screenshot with the 'Smart Styles' Dialog Box illustrating some of the foregoing operations and features.
FIGURE 19 illustrates the "Smart Request Watch" Dialog Box.
FIGURE 20 illustrates a Watch Window displaying Filtered Smart Requests (e.g., Headlines on Wireless). Figure 20 is an Illustration of the Watch Window with a Current Smart Request Title (e.g., "Breaking News").
FIGURE 21 illustrates Entity views displayed in the Semantic Browser.
FIGURE 22A and 22B show the UI for the Knowledge Community Subscription.
FIGURE 23 illustrates a semantic thread object and its semantic links.
FIGURES 24 through 46B are additional screen shots further illustrating the functions, options and operations as described in the Detailed Description.
FIGURE 47 as a sample semantic image for Pharmaceuticals/Biotech industry (DNA helix).
FIGURE 48 is an illustration of a semantically appropriate image visualization for the Breaking News context template.
FIGURE 49 is a Visualization — Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc. (Headlines).
FIGURE 50 is a Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc. (Two people working at a desk).
FIGURE 51 illustrates a semantic "Newsmaker" Visualization or Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
FIGURE 52 illustrates a semantic "Upcoming Events" Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
FIGURE 53 is a Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc. (Petri Dish).
FIGURE 54 illustrates a semantic "History" Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc. FIGURE 55 illustrates a semantic Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc. (Spacecraft).
FIGURE 56 illustrates a "Best Buys" Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
FIGURE 57 illustrates a semantic Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc. (Coffee).
FIGURE 58 illustrates a semantically appropriate Sample Image for "Classics" for smart hourglass, interstitial page, transition effects, background chrome, etc. (Car).
FIGURE 59 illustrates a semantically appropriate "Recommendation" Visualization - Sample Image for the contextual/application elements of smart hourglass, interstitial page, transition effects, background chrome, etc. (Thumbs up).
FIGURE 60 illustrates a semantic "Today" Visualization - Sample Image for the elements smart hourglass, interstitial page, transition effects, background chrome, etc.
FIGURE 61 illustrates a semantic "Annotated Items" Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
FIGURE 62 illustrates a semantic "Annotations" Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
FIGURE 63 illustrates a semantic "Experts" Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
FIGURE 64 illustrates a semantic "Places" Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
FIGURE 65 illustrates a semantic "Blenders" Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
FIGURES 66 through 84 illustrate semantic Visualizations for the following Information Object Types, respectively: Documents, Books, Magazines, Presentations, Resumes, Spreadsheets, Text, Web pages, White Papers, Email, Email Annotations, Email Distribution Lists, Events, Meetings, Multimedia, Online Courses, People, Customers, and Users. Figure 85 illustrates a semantic "Timeline" Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc..
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
TABLE OF CONTENTS
A. ADDITIONAL ILLUSTRATIVE SCENARIOS 9
1. Patent Examiner Prior Art Search Tool 9
2. BioTech Company Research Scenario 13
B. SUBJECT MATTER FOR THE PRESENTLY PREFERRED EMBODIMENT OF THE INFORMATION NERVOUS SYSTEM 17
1. Smart Selection Lens Overview 18
2. Pasting Person Objects Overview 20
3. Saving and Sharing Smart Requests Overview 22
4. Saving and Sharing Smart Snapshots Overview 24
5. Virtual Knowledge Communities 25
6. Implementing Time-Sensitive Semantic Queries 26
7. Text-To-Speech Skins Overview 26
8. Language Translation Skins 28
9. Categories as First Class Objects in the User Experience 29
10. Categorized Annotations 29
11. Additional Context Templates 30
12. Importing and Exporting User State 31
13. Local Smart Requests 32
14. Integrated Navigation 32
15. Hints for Visited Results 33
16. Knowledge Federation 34
17. Anonymous Annotations and Publications 38
18. Offline Support in the Semantic Browser 38
19. Guaranteed Cross-Platform Support in the Semantic Browser 39
20. Knowledge Modeling 40
21. KIS Housekeeping Rules 41
22. Client Component Integration & Interaction Workflow 41
23. Categories Dialog Box User Interface Specification 42
24. Client-Assisted Server Data Consistency Checking 45
25. Client-Side Duplicate Detection.......................... 46
26. Client-Side Virtual Results Cursor 46
27. Virtual Single Sign-On 47
28. Namespace Object Action Matrix 49
29. Dynamic End-to-End Ontology/Taxonomy Updating and Synchronization 50
30. Invoking Dossier (Guide) Queries 51
31. Knowledge Community (Agency) Semantics 52
32. Dynamic Ontology and Taxonomy Mapping 53
33. Semantic Alerts Optimizations 53
34. Semantic "News" Images 54
35. Dynamically Choosing Semantic Images 54
36. Dynamic Knowledge Community (Agency) Contacts Membership 54
37. Integrated Full-Text Keyword and Phrase Indexing 55
38. Semantic "Mark Object as Read" 56
39. Multi-Select Object Lens 57
40. Ontology-Based Filtering and Spam Management 58
41. Results Refinement 58
42. Semantic Management of Information Stores 60
43. Slide-Rule Filter User Interface 61
C. SERVER-SIDE SEMANTIC QUERY PROCESSOR SPECIFICATION 62
1. Overview 62
2. Semantic Relevance Score 63
3. Semantic Relevance Filter 63 4. Time-Sensitivity Filter 64
5. Knowledge Type Semantic Query Implementations 64
D. EXTENSIBLE CLIENT-SIDE USER PROFILES SPECIFICATION FOR THE INFORMATION NERVOUS SYSTEM 71
E. SMART STYLES SPECIFICATION FOR THE INFORMATION NERVOUS SYSTEM 73
1. Smart Styles Overview 73
2. Implicit and Dynamic Smart Style Properties 73
F. SMART REQUEST WATCH SPECIFICATION FOR THE INFORMATION NERVOUS SYSTEM 75
1. Overview 75
2. Request Watch Lists (RWLs) and Groups (RWGs) 76
3. The Notification Manager (NM) 79
4. Watch Group Monitors 80
5. The Watch Pane 80
6. The Watch Window 81
7. Watch List Addendum 82
G. ENTITIES SPECIFICATION FOR THE INFORMATION NERVOUS SYSTEM 82
1. Introduction 82
2. Portfolios (or Entity Collections) 88
3. Sample Scenarios 89
H. KNOWLEDGE COMMUNITY BROWSING AND SUBSCRIPTION SPECIFICATION FOR THE
INFORMATION NERVOUS SYSTEM 90
I. CLIENT-SIDE SEMANTIC QUERY DOCUMENT SPECIFICATION FOR THE INFORMATION
NERVOUS SYSTEM 91
1. Semantic Query Markup Language (SQML) Overview 91
2. SQML Generation 100
3. SQML Parsing 100
J. SEMANTIC CLIENT-SIDE RUNTIME CONTROL API SPECIFICATION FOR THE INFORMATION
NERVOUS SYSTEM 101
1. Introducing the Nervana Semantic Runtime Control - Overview 101
2. The Nervana Semantic Runtime Control API 101
3. Email Control APIs 112
4. Person Control APIs 115
5. System Control Events 118
K. SECURITY SPECIFICATION FOR THE INFORMATION NERVOUS SYSTEM 120
1. Authorization 120
2. People Groups 124
3. Identity Metadata Federation 125
4. Access Control 126
L. DEEP INFORMATION SPECIFICATION FOR THE INFORMATION NERVOUS SYSTEM 132
M. CREATE REQUEST WIZARD SPECIFICATION FOR THE INFORMATION NERVOUS SYSTEM 138
N. CREATE PROFILE WIZARD SPECIFICATION FOR THE INFORMATION NERVOUS SYSTEM 140
O. CREATE BOOKMARK WIZARD SPECIFICATION FOR THE INFORMATION NERVOUS SYSTEM 141
1. Introducing the Create Bookmark Wizard 141
2. Scenarios 142
3. Intelligent Publishing-Tool Metadata Suggestion and Maintenance 142
P. SEMANTIC THREADS SPECIFICATION FOR THE INFORMATION NERVOUS SYSTEM™ 143
1. Semantic Threads 143
2. Semantic Thread Conversations 146
3. Semantic Thread Management 147
Q. SAMPLE SCREEN SHOTS 148
R. SPECIFICATION FOR SEMANTIC QUERY DEFINITIONS & VISUALIZATIONS FOR THE
INFORMATION NERVOUS SYSTEM 148
1. Semantic Images & Motion 148
2. The Smart Hourglass 153
3. Visualizations — Context Templates 154 In a currently preferred embodiment, the system incorporates not only the features and functions described in my parent application and this CIP. A. ADDITIONAL ILLUSTRATIVE SCENARIOS
The following scenarios help to explain the utility and operation of the system, and will thereby make the rest of the detailed description easier to follow and understand. 1. Patent Examiner Prior Art Search Tool
Largely because of PTO fee diversion, there is a great' deal of pressure on U.S. Patent Examiners to conduct a robust prior art search in very little time. And, while the research tools available to Examiners have improved dramatically in the last several years, those tools still have many shortcomings. Among the shortcomings are that most of the research tools are text based, rather than meaning based. So, for example, the search tool on the PTO website will search for particular words in particular fields in a document. Similarly, the advanced search tool on Google enables the Examiner to locate documents with particular words, or particular strings of words, or documents without a particular word or words. However, in each case, the search engine does not allow the Examiner to locate documents on the basis of meaning. So, for example, if there is a relevant reference that teaches essentially the same idea, but uses completely different words (e.g., a synonym, or worse yet, a synonymous phrase) than those in the query, the reference, even though perhaps anticipating, may well not be discovered. Even if the Examiner could spare the time to imagine and search every possible synonym, or even synonymous phrase to the key words critical to the invention, it could still overlook references because sometimes the same idea can be expressed without using any of the same words at all, and sometimes the synonymous idea is not neatly compressed into a phrase, but distributed over several sentences or paragraphs.
The reason for this is that words do not denote or connote meaning one to one as, for example, numerals tend to do. Put differently, certain meanings can be denoted or connoted by several different words or an essentially infinite combination of words, and, conversely, certain words or combinations of words can denote or connote several different meanings. Despite this infinite many-to-many network of possibilities human beings can isolate (because of context, experience, reasoning, inference, deduction, judgment, learning and the like) isolate probable meanings, at least tolerably effectively most of the time. The current prior art computer- automated search tools (e.g. the PTO website, or Google, or Lexis), cannot. The presently preferred embodiment of my invention bridges this gap considerably because it can search on the basis of meaning.
For example, using the some of the search functions of the preferred embodiment of the present invention, the Examiner could conduct a search, and with no additional effort or time as presently invested, obtain search results relevant to patentability even if they did not contain a single word in common with the key words chosen by the Examiner. Therefore, the system would obtain results relevant to the Examiner's task that would not ordinarily be located by present systems because it can locate references on the basis of meaning.
Also on the basis of meaning, it can exclude irrelevant references, even if they share a key word or words in common with the search request. In other words, one problem in prior art research is the problem of a false positive; results that the search engine "thought" were relevant merely because they had a key word in common, but that were in fact totally irrelevant because the key word, upon closer inspection in context, actually denoted or connoted an irrelevant idea. Therefore, the Examiner must search for the needle in the haystack, which is a waste of time. hi contrast, using some of the search functions of the preferred embodiment of the present invention, the density of relevant search results increases dramatically, because the system is "intelligent" enough to omit search results that, despite the common key words, are not relevant. Of course, it is not perfect in this respect any more than human beings are perfect in this respect. But, it is much more effective at screening irrelevant results than present systems, and in this respect resembles in function or in practice an intelligent research assistant than a mere keyword based search engine. Thus, using the system, the Examiner can complete a much better search in much less time. The specific mechanics of using the system this way, in one example, would work as follows:
Imagine the Examiner is assigned to examine an application directed to computer software for a more accurate method of interpreting magnetic resonance data and thereby generating more accurate diagnostic images. To search for relevant prior art using the search functions of the preferred embodiment of the present invention, the Examiner would: a. Using the Create Entity wizard, create a "Topic" entity with the relevant categories in the various contexts in which "Magnetic Resonance Imaging" occurs. As an illustration, Figures 1 and 2 show where "Magnetic Resonance Imaging" occurs in a Pharmaceuticals taxonomy. Notice that there are several contexts in which the category occurs. Add the relevant categories to the entity and apply the "OR" operation. Essentially, this amounts to defining the entity "Magnetic Resonance Imaging" (as it relates to YOUR specific task) as being equivalent to all the occurrences of Magnetic Resonance Imaging in the right contexts - based on the patent application being examined. b. Name the new entity "Magnetic Resonance Imaging" and perhaps "imaging" and "diagnostic" or some variations and combinations of the same. c. Drag and drop the "Magnetic Resonance Imaging" Topic entity to the Dossier
(special agent or default knowledge request) icon in the desired profile (the profile is preferably configured to include the "Patent Database" knowledge community). This launches a new
Dossier request/agent that displays each special agent (context template). Each special agent is displayed with the right default predicate as follows:
All Bets on Magnetic Resonance Imaging Best Bets on Magnetic Resonance Imaging Breaking News on Magnetic Resonance Imaging Headlines on Magnetic Resonance Imaging Random Bets on Magnetic Resonance Imaging Experts in Magnetic Resonance Imaging Newsmakers in Magnetic Resonance Imaging Interest Group in Magnetic Resonance Imaging Conversations on Magnetic Resonance Imaging Annotations on Magnetic Resonance imaging Annotated Items on Magnetic Resonance Imaging Upcoming Events on Magnetic Resonance imaging Popular Items on Magnetic Resonance Imaging Classics on Magnetic Resonance Imaging d. Alternatively, the request can be created by using the Create Request Wizard. To do this, select the Dossier context template and select the "Patent Database" knowledge community as the knowledge source for the request. Alternatively, you can configure the profile to include the "Patents Database" knowledge community and simply use the selected profile for the new request. Hit Next - the wizard intelligently suggests a name for the request based on the semantics of the request. The wizard also selects the right default predicates based on the semantics of the "Magnetic Resonance Imaging" "Topic" entity. Because the wizard knows the entity is a "Topic," it selects the right entities that make sense in the right contexts. Hit Finish. The wizard compiles the query, sends the SQML to the KISes in the selected profile, and then displays the results. hi the foregoing example, the results could be drawn, ultimately, from any source. Preferably, some of the results would have originated on the Web, some on the PTO intranet, some on other perhaps proprietary extranets. Regardless of the scope or origin of the original documents, by use of the system they have been automatically processed, and automatically "read" and "understood" by the system, so that when the Examiner's query was initiated, and also "read" and "understood" semantically, and by context, the system locates all relevant, and only relevant results. Again, not perfectly, but radically more accurately than in any prior systems. Note also that the system does not depend on any manual tagging or categorization of the documents in advance. While that would also aid in accuracy, it is so labor intensive as to utterly eclipse the advantages of online research in the first place, and is perfectly impractical given the rate of increase of new documents. hi this scenario, the Examiner may also wish to use additional features of the preferred embodiment of the invention. For example, the Examiner may wish to consult experts within the PTO, or literature by experts outside the PTO, as follows (note that Experts in Magnetic Resonance Imaging would be included in the Dossier on Magnetic Resonance Imaging; however, the examiner might want to create a separate request for Experts in order to track it separately, save it as a "request document," email it to colleagues, etc.). Find all Experts in Magnetic Resonance Imaging: a. Follow steps 1-4 above. b. Drag and drop the "Magnetic Resonance Imaging" entity to the Experts (special agent or default knowledge request) icon in the desired profile. This automatically launches a new request/agent appropriately titled "Experts in Magnetic Resonance Imaging." The semantic browser selects the right default predicate "in" because it "knows" the entity is a "Topic" entity and the context template is a "People" template (Experts). As such, the default predicate is selected based on the intersection of these two arguments ("in") since this is what makes sense.
2. BioTech Company Research Scenario
Biotech companies are research intensive, not only in laboratory research, but in research of the results of research by others, both within and outside of their own companies. Unfortunately, the research tools available to such companies have shortcomings. Proprietary services provide context-sensitive and useful results, but those services themselves have inferior tools, and thus rely heavily on indexing and human effort, and subscriptions to expensive specialized journals, and as consequence are very expensive and not as accurate as the present system. On the other hand, biotech researchers can search inexpensively using Google D, but it shares all the key word based limitations described above. i contrast, using the search features of the preferred embodiment of the present invention, a biotech researcher could more efficiently locate more relevant results. Specifically, the researcher might use the system as follows. For example, if some researchers wanted to Find Headlines on Genomics and Anatomy written by anyone in Marketing or Research, they would do that as follows: a. Using the wizard, launch an information-type request/agent for distribution lists with the keywords "Marketing Research". b. Select the Marketing distribution list result and click "Save as Entity" - this saves the object as a "Team" entity (because the semantic browser "knows" the original object is a distribution list - as such, a "Team" entity makes sense in this context). c. Select the Research distribution list result and click "Save as Entity" - this saves the object as a "Team" entity (because the semantic browser "knows" the original object is a distribution list). d. Using the Create Entity Wizard, create a new "Team" entity and select the "Marketing" and "Research" team entities as members. Name the new entity "Marketing or Research". e. Using the Create Request Wizard, select the Headlines context template, and then select the "Marketing or Research" entity as a filter. Also, select the Genomics category and the Anatomy category. Next, select the "AND" operator. Hit Next - the wizard intelligently suggests a name for the request based on the semantics of the request. The wizard also selects the right default predicates based on the semantics of the "Marketing or Research" team entity ("by anyone in"). Because the wizard knows the entity is a "Team," it selects "by anyone in" by default since this makes sense. Hit Finish. The wizard compiles the query, sends the SQML to the KISes in the selected profile, and then displays the results.
In addition, the researchers may wish to Find all Experts in Marketing or Research: a. Follow steps 1-4 above. b. Drag and drop the "Marketing or Research" entity to the Experts (special agent or default knowledge request) icon in the desired profile. This launches a new request/agent appropriately titled "Experts in Marketing or Research." The semantic browser selects the right default predicate "in" because it "knows" the entity is a "Team" entity and the context template is a "People" template (Experts). As such, the default predicate is selected based on the intersection of these two arguments ("in") since this is what makes sense. If the researchers expect to need to return to this research, or to supplement it, or to later analyze the results, they may wish to Open a Dossier on Marketing or Research, as follows: a. Follow steps 1-4 above. b. Drag and drop the "Marketing or Research" entity to the Dossier (special agent or default knowledge request) icon in the desired profile. This launches a new Dossier request/agent that displays each special agent (context template). Each special agent is displayed with the right default predicate as follows:
All Bets by anyone in Marketing or Research
Best Bets by anyone in Marketing or Research
Breaking News by anyone in Marketing or Research
Headlines by anyone in Marketing or Research
Random Bets by anyone in Marketing or Research
Experts in Marketing or Research
Newsmakers in Marketing or Research
Interest Group in Marketing or Research
Conversations involving anyone in Marketing or Research ° Annotations by anyone in Marketing or Research
° Annotated Items by anyone in Marketing or Research
• Upcoming Events by anyone in Marketing or Research
• Popular Items by anyone in Marketing or Research
• Classics by anyone in Marketing or Research
The researchers may be interested in Finding "Breaking News on my Competitors", and would do so as follows: a. For each competitor, create a new "competitor" entity (under "companies") using the Create Entity Wizard. Select the right filters as needed. For instance, a competitor with a well-known English name - like "Groove" should have an entity that includes categories in which the company does business and also the keyword. b. Using the Create Entity Wizard, create a portfolio (entity collection) and add all the competitor entities you created in step a. Name the entity collection "My Competitors." c. Using the Create Request Wizard, select the Breaking News context template and add the portfolio (entity collection) you created in step b. as a filter. Keep the default predicate selection. Hit "Next" - the wizard intelligently suggests a name for the request using the default predicate ("Breaking News on My Competitors"). Hit Finish. The wizard launches a new request/agent named "Breaking News on My Competitors." hi addition, the researchers may wish to be kept apprised. They could instruct the system to alert them on "Breaking News on our Competitors", as follows: a. Create the "Breaking News on My Competitors" request as described above. b. Add the request to the request watch list. The semantic browser will now display a watch pane (e.g., a ticker) showing "Breaking News on My Competitors." Using the Notification Manager (NM), you can also indicate that the semantic browser send alerts via email, instant messaging, text messaging, etc. when there are new results from the request/agent.
In addition, the researchers may wish to keep records of competitors for future reference, and to have them constantly updated. The system will create and update such records, by the researchers instructing the system to Show a collection of Dossiers on each of our competitors, as follows: a. Create entities for each of your competitors as described in 4a. above. b. For each competitor entity, create a new Dossier on that competitor by dragging the entity to the Dossier icon for the desired profile - this creates a Dossier on the competitor. c. Using the Create Request Wizard, create a new request collection (blender) and add each of the Dossier requests created in step b. above to the collection (you can also drag and drop requests to the collection after it has been created in order to further populate the collection). Hit Next — the wizard intelligently suggests a name for the request collection. Hit Finish. The wizard launches a request collection that contains the individual Dossiers. You can then add the request collection as a favorite and open it everyday to get rich, contextual competitive intelligence.
The researchers may wish to review a particular dossier, and can do so by instructing the system to Show a Dossier on the CEO (e.g., named John Smith): a. Using the wizard, launch an information-type request/agent for People with the keywords "John Smith". b. Select the result and click "Save as Entity" - this saves the object as a "Person" entity (because the semantic browser "knows" the original object is a person - as such, a "Person" entity makes sense in this context). c. Using the Create Request Wizard, select the Dossier context template, and then select the "John Smith" entity as a filter. Hit Next - the wizard intelligently suggests a name for the request based on -the semantics of the request. The wizard also selects the right default predicates based on the semantics of the "John Smith" person entity. Hit Finish. The wizard compiles the query, sends the SQML to the KISes in the selected profile, and then displays the results (as sub-queries/agents) as follows:
All Bets by John Smith Best Bets by John Smith Breaking News by John Smith Headlines by John Smith Random Bets by John Smith ° Experts like John Smith (this returns Experts that have expertise on the same categories as those in which John Smith has expertise)
Newsmakers like John Smith (this returns Newsmakers that have recently "made news" in the same categories as those in which John Smith has recently "made news")
• Interest Group like John Smith (this returns the people that have shown an interest in the same categories as those in which John Smith has shown interest - within a time-window (2-3 months in the preferred embodiment))
° Conversations involving John Smith
• Annotations by John Smith
• Annotated Items by John Smith
• Upcoming Events by John Smith
• Popular Items by John Smith Classics by John Smith
The foregoing scenarios illustrate the operation of the system. The system itself is described in greater detail below.
B. SUBJECT MATTER FOR THE PRESENTLY PREFERRED EMBODIMENT OF THE INFORMATION NERVOUS SYSTEM
Several improvements, enhancements and variations have been developed since the filing of my co-pending parent application and prior provisional applications referenced above. Some of these are improvements on, or only clarifications of, features previously included in the parent application, and some are new features of the system altogether. These are listed and described below. They are not arranged in order of importance, or in any particular order. While the preferred embodiment of the present invention would allow the user to use any or all of these features and improvements described below, alone or in combination, no single feature is necessary to the practice of the invention, nor any particular combination of features.
Also, in this application, reference is made to the same terms as are defined in my parent application Serial No. 10/179,651, and the Description throughout this application is intended to be read in conjunction with the definitions, terminology, nomenclature and Figures of my parent application except where the context of this application clearly indicates to the contrary.
1. Smart Selection Lens Overview
The Smart Selection Lens is similar to the Smart Lens feature of the Information Nervous System information medium, hi this case, the user can select text within the object and the lens will be applied using the selected text as the object (dynamically generating new "images" as the selection changes). This way, the user can "lens" over a configurable subset of the object metadata, as opposed to being constrained to "lens" over either the entire object or nothing at all. This feature is similar to a selection cursor/verb overloaded with context. For example, the user can select a piece of text in the Presenter and hit the "Paste as Lens" icon over the object in which the text appears. The Presenter will then pass the text to the client runtime component (e.g., an ActiveX object) with a method call like: bstrSRML = GetSRMLForText( bstrText );
This call then returns a temporary SRML buffer that encapsulates the argument text. The Presenter will then call a method like: bstrSQML = GetQueryForSmartLensOnObject( bstrSRMLObject );
This method gets the SQML from the clipboard, takes the argument SRML for the object, and dynamically creates new SQML that includes the resource in the SRML as a link in the SQML (with the default predicate "relevant to"). The method then returns the new SQML. The Presenter then calls the method:
ProcessSemanticQuery( bstrSQML);
This method passes the generated lens SQML and then retrieves the number of items in the results and the SRML results, preferably asynchronously. For details on this call, see the specification "Information Nervous System Semantic Runtime OCX." The Presenter then displays a preview window (or the equivalent, based on the current skin) with something like:
[Lens Agent Title] Found 23 items [PREVIEW OBJECT 1 ] [PREVIEW WINDOW CONTROLS ] where the "Lens Agent Title" is the title of the agent on the clipboard. For details of the preview window (and the preview window controls), please refer to my parent application Serial No. 10/179,651. hi the preferred embodiment, the preview window will:
• Disappear after a timer expires (maybe 500ms) - on mouse move, the timer is preferably reset (this will avoid flashing the window when the user moves the mouse around the same area).
° Fade out slowly (eventually).
The preferred embodiment also has the following features:
1. One selection range per obj ect but multiple selections per results-set is the best option. Otherwise, the system would result in a confusing user experience and complex UI to show lens icons per selection per object (as opposed to per object).
2. Outstanding lens query requests (which are regular SQML queries, albeit with SQML dynamically generated consistent with the agent lens) should be cancelled when the Presenter no longer needs them (e.g. if the Presenter is navigating to a new page, or if we are requesting new lens info for an object). In any case, such cancellation is not critical from a performance (or bandwidth) standpoint because lens queries will likely only ask for a few objects at a time. Even if the queries are not cancelled, the Presenter can ignore the results. Regardless, because the Presenter also has to deal with stale results, dropping them on the floor -the Presenter will have to do this anyway (whether or not lens queries are also cancelled). There will be a window of delay between when the Presenter issues a cancel request and when the cancellation actually is complete. Because some results can trickle in during this time, they need to be discarded. Thus, the preferred embodiment has asynchronous cancellation implementations - the software component has been designed to always be prepared to ignore bad or stale results.
3. The Presenter preferably has both icons (indicating the current lens request state) and tool-tips: When the user hovers over or clicks on an object, the Presenter can put up a tooltip with the words, "Requesting Lens Info" (or words to that effect). When the info comes back, hovering will show the "Found 23 Objects" tip and clicking will show the results. This interstitial tool tip can then be transitioned to the preview window if it is still up when the results arrive.
In addition, note that the smart selection lens, like the smart lens, can be applied to objects other than textual metadata. For instance, the Smart Selection Lens can be applied to images, video, a section of an audio stream, or other metadata. In these cases, the Presenter would return the appropriate SRML consistent with the data type and the "selection region." This region could be an area of an image, or video, a time span in an audio stream, etc. The rest of the smart lens functionality would apply as described above, with the appropriate SQML being generated based on the SRML (which in turn is based on the schema for the data type under the lens).
2. Pasting Person Objects Overview
The Information Nervous System (which, again, is one of our current shorthand names for certain aspects of our presently preferred embodiments) also supports the drag and drop or copy and paste of 'Person' objects (People, Users, Customers, etc.). There are at least two scenarios to illustrate the operation of the preferred embodiment in this case: 1. Pasting a Person object on a smart request representing a Knowledge community (or Agency) from whence the Person came. In this case, the server's semantic query processor merely resolves the SQML from the client using the Person as the argument. For instance, if the user pastes (or drags and drops) a person 'Joe' on top of a smart request 'Headlines on Reuters,' the client will create a new smart request using the additional argument. The Reuters Information Nervous System Web service will then resolve this request by returning all Headlines published or annotated by 'Joe.' In this case, the server will essentially apply the proper default predicate ('published or annotated by') - that makes sense for the scenario.
2. Pasting a Person object on a smart request representing a Knowledge community (or Agency) from whence the Person did not come. In this case, because the Person object is not in the semantic network of the destination Knowledge community (on its SMS), the server's semantic query processor would not be able to make sense of the Person argument. As such, the server must resolve the Person argument, in a different way, such as, for example, using the categories on which the person is an expert (in the preferred embodiment) or a newsmaker. For instance, taking the above example, if the user pastes (or drags and drops) a person 'Joe' on top of a smart request 'Headlines on Reuters' and Joe is not a person on the Reuters Knowledge community, the Reuters Web service (in the preferred embodiment) must return Headlines that are "relevant to Joe's expertise." This embodiment would then require that the client take a two- pass approach before sending the SQML to the destination Web service. First, it must ask the Knowledge community that the person belongs to for "representative data (SRML)" that represents the person's expertise. The Web service resolves this request by: a. Querying the Knowledge community (e.g., Reuters) on which the person object is pasted or dropped for that community's semantic domain information which comprises and/or represents that community's specifictaxonomy and ontology. Note that there could be several semantic domains. b. Querying the Knowledge community from whence the person object came for that person object's semantic domain information. c. If the semantic domains are identical or if there is at least one common semantic domain, the client queries the Knowledge community from whence the person came for the person's categories of expertise. The client then constructs SQML with these categories as arguments and passes this SQML to the Knowledge community on which the person was pasted or dropped.
If the semantic domains are not identical or there is not least one common semantic domain, the client queries the Knowledge community from whence the person came for several objects that belong to categories on which the person is an expert, hi the preferred embodiment, the implementation should pick a high enough number of objects that accurately represent the categories of expertise (this number is preferably picked based on experimentation). The reason for picking objects in this case is that the destination Web service will not understand the categories of the Knowledge community from whence the person came and as such will not be able to map them to its own categories. Alternatively, a category mapper can be employed (via a centralized Web service on the Internet) that maps categories between different Knowledge Communities, i this case, the destination Knowledge community will always be passed categories as part of the SQML, even though it does not understand those categories - the Knowledge community will then map these categories to internal categories using the category mapper Web service. The category mapper Web service will have methods for resolving categories as well as methods for publishing category mappings.
3. Saving and Sharing Smart Requests Overview
Users of the Information Nervous System semantic browser (the Information Agent or Librarian) will also be able to save smart requests to disk, email them as an attachment, or share them via Instant Messenger (also as an attachment) or other means. The client application will expose methods to save a smart request as a sharable document. The client application will also expose methods to share a smart request document as an attachment in email or Instant Messenger. A sharable smart request document is a binary document that encapsulates SQML (via a secure stream in the binary format). It provides a safe, serialized representation of a semantic query that, among other features, can protect the integrity and help protect the intellectual property of the specification. For example, the query itself may embody trade secrets of the researcher's employer, which, if exposed, could enable a competitor to reverse engineer critical competitive information to the detriment of the company. The protection can be accomplished in several ways, including by strongly encrypting the XML version of the semantic query (the SQML) or via a strong one-way hash. The sharable document has an extension (.REQ) that represents the request. An extension handler on the client operating system is installed to represent this extension. When a document with the extension is opened, the extension handler is invoked to open the document. The extension handler opens the document by extracting the SQML from the secure stream, and then creating a smart request in the semantic namespace with the SQML. The handler then opens the smart request in the semantic namespace.
When a smart request in the semantic namespace is saved or if the user wants to send it as an email attachment, the client serializes the SQML representing the smart request in the binary .REQ format and saves it at the requested directory path or opens the email client with the .REQ document as an attachment.
Figure 3 shows the binary document format that encapsulates the SQML buffer with the smart request and also illustrates how the extension handler opens the document. A similar model can also be employed for sharing results (via SRML). hi this case, a binary document encapsulates the SRML, rather than the SQML as in the case above.
Figure 4A and 4B shows an illustration of two .REQ documents (titled 'Headlines on Reuters Related to My Research Report (Live)' and 'Headlines on Reuters (as of January 21 2003, 08 17AM)' on the far right) with a registered association in the Windows shell. The first request document is 'live' and the second one is a snapshot at a particular time (they are both time-sensitive requests). Notice that the operating system has associated the semantic browser application (Nervana Librarian) with the document. When the document is opened, the semantic query gets opened in the application.
• Saving and sharing entities - the same process applies as above except with a .ENT extension to represent an entity. When an entity document is invoked, the Nervana Librarian opens the entity SQML in the browser.
• Extension Property Sheet - this will create a temporary smart request or entity (depending on the kind of document) in the semantic environment and display the property sheet for a smart request or entity.
Extension Tool tips - this will display a helpful tool tip when the user hovers over a librarian document (a request, .REQ or an entity, .ENT).
4. Saving and Sharing Smart Snapshots Overview
The information Nervous System also supports the sharing of what the inventor calls "Smart Snapshots." A smart snapshot is a smart request frozen in time. This will enable a scenario where the user wants to share a smart request but not have it be "live." For instance, by default, if the user shares the smart request "Breaking News on Reuters related to this document" with a colleague, the colleague will see the live results of the smart request (based on the "current time"). However, if the user wants to share "[Current] Breaking News on Reuters related to this document," a smart snapshot will be employed.
A smart snapshot is the same as a smart request (it is also represented by an SQML query document) except that the "attributes" section of the SQML document contains attributes marking it as a snapshot (the flag QUERYATTRIBUTES_SNAPSHOT). The creation date/time of the SQML document is also stored in the SQML (as before - the SQML schema contains a field for the creation date/time). When the user indicates that he/she wants to share the smart request, the user interface (the semantic browser, Information Agent, or Librarian) prompts him/her whether he/she wants to share the smart request (live) or a smart snapshot. If the user indicates s smart request, the process described above (in Part 3) is employed. If the user indicates a smart snapshot, the binary document is populated with the edited SQML (containing the snapshot attribute) and the remainder the process is followed as above.
When the recipient of the binary document receives it (by email, instant messaging, etc.), and opens it, the extension handler opens the document and adds an entry into the semantic namespace as a smart request (as described above). When the recipient opens the smart request, the client's semantic query processor will send the processed SQML to the server's XML web service (as previously described). The server's semantic query processor then processes the SQML and honors the snapshot attribute by invoking the semantic query relative to the SQML creation date/time. As such, results will be relative to the original date/time, thereby honoring the intent of the sender.
5. Virtual Knowledge Communities
Virtual Knowledge Communities (agencies) refer to a feature of the Information Nervous System that allows the publisher of a knowledge community to publish a group of servers to appear as though they were one server. For instance, Reuters could have per-industry Reuters Knowledge Communities (for pharmaceuticals, oil and gas, manufacturing, financial services, etc.) but might also choose to expose one 'Reuters' knowledge community. To do this, Reuters will publish and announce the SQML for the virtual knowledge community (rather than the URL to the WSDL of the XML Web Service). The SQML will contain a blender (or collection) of the WSDLs of the actual Knowledge Communities. The semantic browser will then pick up the SQML and display an icon for the knowledge community (as though it were a single server). Any action on the knowledge community will be propagated to each server in the SQML. If the user does not have access for the action, the Web service call will fail accordingly, else the action will be performed (no different from if the user had manually created a blender containing the Knowledge Communities). 6. Implementing Time-Sensitive Semantic Queries
Semantic queries that are time-sensitive are preferably implemented in an intelligent fashion to account for the rate of knowledge generation at the knowledge community (agency) in question. For instance, 'Breaking News' on a server that receives 10 documents per second is not the same as 'Breaking News' on a server that receives 10 documents per month. As such, the server-side semantic query processor would preferably adjust its time-sensitive semantic query handling according to the rate at which information accumulates at the server. To implement this, general rules of thumb could be used, for instance:
• The most recent N objects where N is adjusted based on the number of new objects per minute.
All objects received in the last N minutes with a cap on the number of objects (i.e., min (cap, all objects received in the last N minutes)).
N can also be adjusted based on whether the query is a Headline or Breaking News. In the preferred embodiment, newsmaker queries is preferably implemented with the same time- sensitivity parameters as Headlines.
7. Text-To-Speech Skins Overview
Text-to-speech is implemented at the object level and at the request level. At the object level, the object skin runs a script to take the SRML of the object, interprets the SRML, and then passes select pieces of text (in the SRML fields) to a text-to-speech engine (e.g., using the Microsoft Windows Speech SDK) that generates voice output.
Figure 5 shows a diagram illustrating text-to-speech object skin. When executed, the pipeline shown in Figure 5 results in the following voice output:
1. Reading Email Message
2. Appropriate Delay
3. Message From Nosa Omoigui
4. Appropriate Delay
5. Message Sent to John Smith
6. Appropriate Delay
7. Message Copied To Joe Somebody
8. Appropriate Delay 9. Message Subject Is Web services are software building blocks used for distributed computing
10. Appropriate Delay
11. Message Summary is Web services
12. Appropriate D elay
13. [Optional] Message Body is Web services are software building blocks used for distributed computing
This example assumes a voice skin template as follows:
1. Reading Email Message
2. Appropriate Delay
3. Message From <message author name>
4. Appropriate Delay
5. Message Sent to <message to: recipient name>
6. Appropriate Delay
7. Message Copied To <message cc: recipient name>
8. Appropriate Delay
9. Message Subject Is <message subject text>
10. Appropriate Delay
11. Message Summary is <message body summary>
12. Appropriate Delay
13. [Optional] Message Body is <message body>
Other templates can also be used to render voice that is easily understandable and which conveys the semantics of the object type being rendered. Like the example shown above (which is for email), the implementation should use appropriate text-to-speech templates for all information object types, in order to capture the semantics of the object type.
At the request level, the semantic browser's presentation engine (the Presenter) loads a skin that takes the SRML for all the current objects being rendered (based on the user-selected cursor position) and then invokes the text-to-speech object skin for each object. This essentially repeats the text-to-speech action for each XML object being rendered, one after another.
Email Object (SRML)
Object Interpretation Engine (Object Skin)
Text-to-Speech Engine
From: Nosa Omoigui
To: John Smith
Cc: Joe Somebody
Subject: Web services
Summary: Web services are software building blocks used for distributed computing
Body: Web services... Voice Output
Reading Email Message
Delay
Voice Output
Message From Nosa Omoigui
Delay
Voice Output
Message Sent To John Smith
Delay
Voice Output
Message Copied To Joe Somebody
Delay
Message Subject is Web services are software building blocks used for distributed computing
Voice Output
Delay
Voice Output
Message Summary is Web services
Delay
Voice Output
Message Summary is Web services
Figure 6 shows an illustration of several email objects being presented in the semantic browser via a request skin.
From: Nosa Omoigui
To: John Smith
Cc: Joe Somebody
Subject: Web services
Summary: Web services are software building blocks used for distributed computing
Body: Web services...
Email Object 1
Object Skin (Object 1)
Email Object 2
Email Object 3
Email Object N
8. Language Translation Skins
Language translation skins are implemented similar to text-to-speech skins except that the transform is on the language axis. The XSLT skin (smart style) can invoke a software engine to automatically perform language translation in real-time and then generate XML that is encoded in Unicode (16 bits per character) in order to account for the universe of languages. The XSLT transform that generates the final presentation output then will render the output using the proper character set given the contents of the translated XML.
Language agnostic semantic queries
Semantic queries can also be invoked in a language-agnostic fashion. This is implemented by having a translation layer (the SQML language translator) that translates the SQML that is generated by the semantic browser to a form that is suitable for interpretation by the KDS (or KBS) which in turn has a knowledge domain ontology seeded for one or more languages. The SQML language translator translates the objects referred to by the predicates (e.g., keywords, text, concepts, categories, etc.) and then sends that to the server-side semantic query processor for interpretation. The results are then translated back to the original language by the language translation skin.
9. Categories as First Class Objects in the User Experience
This refers to a feature by which categories of a knowledge community are exposed to the end user. The end user will be able to issue a query for a category as an information type - e.g., 'Web services.' The metadata will then be displayed in the semantic browser, as would be the case for any first-class information object type. Visualizations, dynamic links, context palettes, etc. will also be available using the category object as a pivot. This feature is useful in cases where the user wants to start with the category and then use that as a pivot for dynamic navigation, as opposed to starting off with a smart request (smart agent) that has the category as a parameter.
10. Categorized Annotations
Categorized annotations follow from categqries being first-class objects. Users will be able to annotate a category directly - thereby simulating an email list that is mapped to a category. However, for cases where there are many categories (for instance, in pharmaceuticals), this is not recommended because information can belong to many categories and the user should not have to think about which category to annotate - the user should publish the annotation directly to the knowledge community (agency) where it will be automatically categorized or annotate an object like a document or email message that is more contextual than a category. 11. Additional Context Templates
1. Experts - The Experts feature was indicated as a special agent in my parent application Serial No. 10/179,651. As should have also been understood from that application, the Experts feature can also operate in conjunction with the context templates section. Experts are a context template and as the name implies indicate people that have expertise on one or more subject matters or contexts (indicated by the PREDICATETYPEID_EXPERTON predicate).
2. Interest Group - this refers to a context template which as the name implies indicate people that have interest (but not necessarily expertise) on one or more subject matters or contexts (indicated by the PREDICATETYPEJDJNTERESTIN predicate). This context template returns People that have shown interest in any semantic category in the semantic network. A very real-world scenario will have Experts returning people that have answers and interest Group returning results of people that have questions (or answers), hi the preferred embodiment, this is implemented by returning results of people who have authored information that in turn has been categorized in the semantic network, with the knowledge domains configured for the KIS. Essentially, this context template presents the user with dynamic, semantic communities of interest. It is a very powerful context template. Currently, most organizations use email distribution lists (or the like) to indicate communities of interest. However, these lists are hard to maintain and require that the administrator manually track (or guess) which people in the organization preferably belong to the list(s). With the Interest Group context template, however, the "lists" now become intelligent and semantic (akin to "smart distribution lists"). They are also contextual, a feature that manual email distribution lists lack.
Like with other context templates, the Interest Group context predicate in turn is interpreted by the server-side semantic query processor. This allows powerful queries like "Interest Group on XML" or "Interest Group on Bioinformatics." Similarly, this would allow queries (via drag, and drop and/or smart copy and paste) like "Interest Group on My Local Document" and "interest Group on My Competitor (an entity)." The Interest Group context template also becomes a part of the Dossier (or Guide) context template (which displays all special agents for each context templates and loads them as sub-queries of the main agent/request).
In the preferred embodiment, the context template should have a time-limit for which it detects "areas of interest." An example of this would be three months. The logic here is that if the user has not authored any information (most typically email) that is semantically relevant to the SQML filter (if available) in three months, the user either has no interest in that category (or categories) or had an interest but doesn't any longer.
3. Annotations of My Items - this is a context template that is a variant of Annotations but is further filtered with items that were published by the calling user. This will allow the user to monitor feedback specifically on items that he/she posted or annotated.
12. Importing and Exporting User State
The semantic browser will support the importation and exportation of user state. The user will be able to save his/her personal state to a document and export it to another machine or vice- versa. This state will include information (and metadata) on:
• Default user state (e.g., computer sophistication level, default areas of interest, default job role, default smart styles, etc.) Profiles
Entities (per profile) Smart requests (per profile) Local Requests (per profile) Subscribed Knowledge Communities (per profile)
The semantic browser will show UI (likely a wizard) that will allow the user to select which of the user state types to import or export. The UI will also ask the user whether to include identity/logon information. When the UI is invoked, the semantic browser will serialize the user state into an XML document that has fields corresponding to the metadata of all the user state types. When the XML document is imported, the semantic browser will navigate the XML document nodes and add or set the user state types in the client environment corresponding to the nodes in the XML document.
13. Local Smart Requests
Local smart requests would allow the user to browse local information using categories from an knowledge community (agency), i the case of categorized local requests, the semantic client crawls the local hard drives, email stores, etc. extracts the metadata (including summaries) and stores the metadata in a local version of the semantic metadata store (SMS). The client sends the XML metadata (per object) to an knowledge community for categorization (via its XML Web Service). The knowledge community then responds with the category assignment metadata. The client then updates the local semantic network (via the local SMS) and responds to semantic queries just like the server would. Essentially, this feature can provide functionality equivalent to a local server without the need for one.
14. Integrated Navigation integrated Navigation allows the user to dynamically navigate from within the Presenter (in the main results pane on the right) and have the navigation be integrated with the shell extension navigation on the left. Essentially, this merges both stacks, hi the preferred embodiment, this is accomplished via event signaling. When the Presenter wants to dynamically navigate to a new request, it sets some state off the GUID that identifies the current browser view. The GULD maps to a key in the registry that also has a field called 'Navigation Event,' 'Next Namespace Object ID' and 'Next Path.' The 'Navigation Event' field holds a DWORD value that points to an event handle that gets created by the current browser view when it is loaded. When the Presenter wants to navigate to a new request, it creates the request in the semantic environment and caches the returned ID of the request. It then dynamically gets the appropriate namespace path of the request (depending on the information/context type of the request) and caches that too. It then sets the two fields ('Next Namespace Object ID' and 'Next Path' with these two values). Next, it sets the 'Navigation Event' (in Windows, this is done by calling a Win32 API named 'SetEvent').
To catch the navigation event, the browser view starts a worker thread when it first starts. This thread waits on the navigation event (and also simultaneously waits on a shutdown event that gets signaled when the browser view is being terminated - in Windows, it does this via a Win32 API named 'WaitForMultipleObjects'). If the navigation event is signaled, the 'Wait' API returns indicating that the navigation event was signaled. The worker thread then looks up the registry to retrieve the navigation state (the object id and the path). It then calls the shell browser to navigate to this object id and path (in Windows, this is done by retrieving a 'PIDL' and then calling IShellBrowser::BrowseTo off the shell view instance that implements IShellView).
15. Hints for Visited Results
The Nervana semantic browser empowers the user to dynamically navigate a knowledge space at the speed of thought. The user could navigate along context, information or time axes. However, as the user navigates, he/she might be presented with redundant information. For instance, the user can navigate from a local document to 'Breaking News' and then from one of the 'Breaking News' result objects to 'Headlines.' However, semantically, some of the Headlines might overlap with the breaking news (especially if not enough time has elapsed). This is equivalent to browsing the Web and hitting the same pages over and over again from different 'angles.'
The Nervana semantic browser handles this redundancy problem by having a local cache of recently presented results. The Presenter then indicates redundant results to the user by showing the results in a different color or some other UI mechanism. The local cache is aged (preferably after several hours or the measured time of a typical 'browsing experience'). Old entries are purged and the cache is eventually reset after enough time might have elapsed. Alternately, at the users option, the redundant results can be discarded and not presented at all. Specifically, the semantic browser will also handle duplicate results by removing duplicates before rendering them in the Presenter - for instance if objects with the same metadata appear on different Knowledge Communities (agencies). The semantic browser will detect this by performing metadata comparisons. For unstructured data like documents, email, etc., the semantic browser will compare the summaries - if the summaries are identical the documents are very likely to be identical (albeit this is not absolutely guaranteed, especially for very long documents).
16. Knowledge Federation
Client-Side Knowledge Federation
Client-side Knowledge Federation which allows the user to federate knowledge communities and operate on results as though they came from one place (this federation feature was described in my parent Application Serial No. 10/179,651). In the preferred embodiment, such Client-side Knowledge Federation is accomplished by the semantic browser merging SRML results as they arrive from different (federated) KISes.
Seiner-Side Knowledge Federation
Server-Side Knowledge Federation is technology that allows external knowledge to be federated within the confines of a knowledge community. For instance, many companies rely on external content providers like Reuters to provide them with information. However, in the Information Nervous System, security and privacy issues arise - relating to annotations, personal publications, etc. Many enterprise customers will not want sensitive annotations to be stored on remote servers hosted and managed by external content providers.
To address this, external content providers will provide their content on a KIS metadata cache, which will be hosted and managed by the company. For instance, Reuters will provide their content to a customer like Intel but Intel will host and manage the KIS. The Intel KIS would crawl the Reuters KIS (thereby chaining KIS servers) or the Reuters DSA. This way, sensitive Intel annotations can be published as 'Post-Its' using Reuters content as context while Intel will still maintain control over its sensitive data.
Federated Annotations
Federated annotations is a very powerful feature that allows the user to annotate an object that comes from one agency/server (KIS) and annotate the object with comments (and/or attachment(s)) - like "Post-Its" on another server. For example, a server (call it Server A) might not support annotations (this is configurable by the administrator and might be the common case for Internet-based servers that don't have a domain of trust and verifiable identity). A user might get a document (or any other semantic result) from Server A but might want to annotate that object on one or more agencies (KISes) that do support annotations (more typically Intranet or Extranet-based agencies that do have a domain of trust and verifiable identity). In such a case, the annotation email message would include the URI of the object to be annotated (the email message and its attachment(s) would contain the annotation itself). When the server crawls its System Inbox and picks up the email annotation, it scans the annotation's encoded To or Subject field and extracts the URI for the object to be annotated. If the URI refers to a different server, the server then invokes an XML Web Service call (if it has access) to that server to get the SRML metadata for the object. The server then adds the SRML metadata to its Semantic Metadata Store (SMS) and adds the appropriate semantic links from the email annotation to the SRML object. This is very powerful because it implies that users of the agency would then view the annotation and also be able to semantically navigate to the annotated object even though that object came from a different server.
If the destination server (for the annotation) does not have access to the server on which the object to be annotated resides, the destination server informs the client of this and the client then has to get the SRML from the server (on which the object resides) and send the complete SRML back to the destination server (for the annotation). This embodiment essentially implies that the client must first "de-reference" the URI and send the SRML to the destination server, rather than having the destination server attempt to "de-reference" the URI itself. This approach might also be superior for performance reasons as it spreads the CPU and I/O load across its clients (since they have to do the downloading and "de-referencing" of the URI to SRML).
Semantic Alerts for Federated Annotations hi the same manner that semantic browser would poll each KIS in the currently viewed user profile for "Breaking News" relevant to each currently viewed object on a regular basis (e.g., every minute), the same will be performed for annotations. Essentially, this resembles polling whether each object that is currently displayed "was just annotated." For annotations that are not federated (i.e., annotations that have strong semantic links to the objects they annotate), this is a straightforward SQML call back to the KIS from whence the annotated object came. However, for federated annotations, the process is a bit more complicated because it is possible that a copy of object has been annotated on a different KIS even though the KIS from whence the object came doesn't support annotations or contain an annotation for the specific object.
In this case, for each object being displayed, the semantic browser would poll each KIS in the selected profile and pass the URI of the object to "ask" the KIS whether that object has been annotated on it. This way, semantic alerts will be generated even for federated annotations.
Annotation Hints
This refers to a feature where the KIS returns a context attribute indicating that an object has been annotated. This can be cached when the KIS detects an annotation (typically from the System Inbox) and is updating the semantic network. This context attribute then becomes a perfonnance optimizer because for those objects with the attribute set, the client wouldn't have to query the KIS again to check if the object has been annotated. This amounts to caching the state of the object to avoid an extra (and unnecessary) roundtrip call to the KIS.
Another Perspective on Annotations
An interesting way to think of the Simple and Semantic Annotations feature of the Information Nervous System is that now every object/item/result in a user's knowledge universe will have its own contextual inbox. That way, if a user views the object, the inbox that is associated with the object's context is always available for viewing, hi other words,
Category Naming and Identification (URIs) for Federated Knowledge Communities
This refers to how categories will be named on federated knowledge communities. For instance, a Reuters knowledge community (agency) deployed at Intel will be named Reuters@Intel with categories named like 'Reuters@hιtel/Information
Technology/Wireless/802i . hi the preferred embodiment, every category will be qualified with at least the following properties:
• Knowledge Domain ID - this is a globally unique identifier that uniquely identifies the knowledge domain from whence the category came
• Name - this is the name of the category
• Path - this is the full taxonomy path of the category
The preferred embodiment, the categories knowledge domain id (and not the name) is preferably used in the category URI, because the category could be renamed as the knowledge domain evolves (but the identifier should remain the same). An example of a category URI in the preferred embodiment is: nerv://c9554bce-aedf-4564-81f7-48432bf8e5a0?type=category&path= Information
Technology/Wireless/80211 hi this example, the knowledge domain id is c9554bce-aedf-4564-81f7-48432bf8e5a0, the URI type is "category" and the category path is "Information Technology/Wireless/80211".
17. Anonymous Annotations and Publications
The semantic browser will also allow users to anonymously annotate and publish to an knowledge community (agency). In this mode, the metadata is completely stored (with the user identity) but is flagged indicating that the publisher wishes to remain anonymous. This way, the inference Engine can infer using the complete metadata but requests for the publisher will not reveal his/her identity. Alternately, the administrator will also be able to configure the knowledge community (agency) such that the inference engine cannot infer using anonymous annotations or publications.
18. Offline Support in the Semantic Browser
The semantic browser will also have offline support. The browser will have a cache for every remote call. The cache will contain entries to XML data. This could be SRML or could be any other data that gets returned from a call to the XML Web Service. Each call is given a unique signature by the semantic browser and this signature is used to hash into the XML data. For instance, a semantic query is hashed by its SQML. Other remote calls are hashed using a combination of the method name, the argument names and types, and the argument data.
For every call to the XML Web Service, the semantic runtime client will extract the signature of the call and then map this to an entry in the local cache. If the browser (or the system) is currently offline, the client will return the XML data in the cache (if it exists). If it does not exist, the client will return an error to the caller (likely the Presenter). If the browser is online, the client will retrieve the XML data from the XML Web Service and update the cache by overwriting the previous contents of the file entry with a file path indicated by the signature hash. This assumes that the remote call actually goes through - it might not even if the system/browser is online, due to network traffic and other conditions, hi such a case, the cache does not get overwritten (it only gets overwritten when there is new data; it does not get cleared first).
19. Guaranteed Cross-Platform Support in the Semantic Browser
Overview
As discussed in my parent application (Serial No. 10/179,651), the information Nervous System can be implemented in a cross-platform manner. Standard protocols are preferably employed where possible and the Web service layer should use interoperable Web service standards and avoid proprietary implementations. Essentially, the test is that the semantic browser does not have to "know" whether the Knowledge community (or agency) Web service it is talking to is running on a particular platform over another. For example, the semantic browser need not know whether the Web service it is talking to is ranning on Microsoft's .NET™ platform or Sun's J2EE platform (to take 2 examples of proprietary application servers), a Linux or any other "open source" server. The Knowledge community Web service and the client- server protocol should employ Web service standards that are commonly supported by different Web service implementations like .NET™ and J2EE™.
In an ideal world, there will be a common set of standards that would be endorsed and properly implemented across Web service vendor implementations. However, this might not be the case in the real world, at least not yet. To handle a case where the semantic browser must handle unique functionality in different Web service implementations, the Knowledge community schema is preferably extended to include a field that indicates the Web service platform implementation. For instance, a .NET™ implementation of the Knowledge community is preferably published with a field that indicates that the platform is .NET™. The same applies to J2EE™. The semantic browser will then have access to this field when it retrieves the metadata for the Knowledge community (either directly via the WSDL URL to the Knowledge community, or by receiving announcements via multicast, the enterprise directory (e.g., LDAP), the Global Knowledge community Directory, etc.). The semantic browser can then issue platform-specific calls depending on the platform that the Knowledge community is running on. This is not a recommended approach but if it is absolutely necessary to make platform-specific calls, this model is preferably employed in the preferred embodiment.
20. Knowledge Modeling
Knowledge Modeling refers to the recommended way enterprises will deploy an Information Nervous System. This involves deploying several KIS servers (per high-level knowledge domain) and one (or at most few) KDS (formerly KBS) servers that host the relevant ontology and taxonomy. KIS servers are preferably deployed per domain to strike a balance between being too narrow such that there is not enough knowledge sharing possibility of navigation and inference in the network and being too high that scalability (in storage and CPU horsepower needed by the database and/or the inference engine) becomes a problem. Of course, the specific point of balance will shift over time as the hardware and software technologies evolve, and the preferred embodiment does not depend on the particular balance struck, hi addition, KIS servers are preferably deployed where access control becomes necessary at the server level (for higher-level security) as opposed to imposing access control at the group level with multiple groups sharing the same KIS. For instance, a large pharmaceutical company could have a knowledge community KIS for oncology for the entire company and another KIS for researchers working on cutting-edge R&D and applying for strategic patents. These two KIS' might crawl the same sources of information but the latter KIS would be more secure because it would provide access only to users from the R&D group. Also, optionally, these researchers' publications and annotations will not be viewable on the corporate KIS.
Figure 7 illustrates an example of a possible knowledge architecture for a pharmaceuticals company. As shown in Figure 7, the KDS can serve several subsidiary KIS', as follows:
Client
Knowledge integration Server 1 (Oncology) Knowledge Integration Server 2 (Pharmacology) Knowledge Integration Server 3 (Biotechnology) Knowledge Integration Server 4 (Cardiology) Knowledge Domain Server (Pharmaceuticals)
21. KIS Housekeeping Rules
The Knowledge Integration Server (KIS) will allow the admin to set up 'housekeeping' rales to purge old or stale metadata. This will prevent the SMS on the KIS from growing infinitely large. These rules could be as simple as purging any metadata older than a certain age (between 2-5 years depending on the company's policies for keeping old data) and which does not have any annotations and that is not marked as a favorite (or rated).
22. Client Component Integration & Interaction Workflow
The client components of the system can be integrated in several different steps or sequences, as can the workflow interaction or usage patterns. In the presently preferred embodiment, the workflow and component integration would be as follows:
1) Shell: User implicitly creates a SQML query (i.e. an agent) via UI navigation or a wizard.
2) Shell: User opens an agent (via tree or folder view).
3) The query buffer is saved as a file, and a registry entry created is created for the agent. a) Registry entry contains: Agent Name, Creation date, Agent (Request)- GULO, SQML path, Comments, Namespace object type (agency, agent, blender, etc), and attributes
4) Shell: The request is handed off to the presenter: a) A registry request GULD entry is created containing (namespace path that generated the request, and SQML file URL). b) Browser is initialized and opened with command line http://PresenterPage.html#RequestGUID http://presenterpage.html/. The Presenter loads default Chrome contained in the page. c) Presenter page loads presenter binary behavior and Semantic Runtime OCX.
5) Presenter: Loads SQML and issues requests via the query manager. a) Resolves request GULD to get SQML file path. b) Loads SQML file into buffer, creates resource handler requests, passes them to resource handlers, waits for and gathers results. Summarization of local resources happens here. All summarization follows one of two paths: Summarize the doc indicated by this file path, or summarize this text (extracted from clipboard, Outlook, Exchange, etc.). Both paths produce a summary in the same form, suitable for inclusion in a request to the semantic server XML Web service. c) Compiles SQML file into individual server request buffers, including any resource summary from above. d) Initiates Server Requests by calling semantic runtime client Query Manager.
6) Query Manager: Monitors server requests and makes callback on data. It also signals an event on request completion or timeout. The callback is into the Presenter, which mean inter-process messaging to pass the XML.
7) Presenter: receives data and loads appropriate skin: a) Receives SRML data in buffer; this will happen incrementally. b) Determines if there is a preferred skin (smart style) associated with this agent, otherwise chooses default skin. c) Transforms SRML into preferred skin format via XSLT. This is multistage, for the tree of results (root is list, then objects, then Deep/Lens/BN info) as results come in. d) Display results in target DIV in page. The target is an argument to the behavior itself and is defined by the root page.
8) Presenter: Calls Semantic Runtime to fill context panels (per context template), deep info, smart copy and paste, and other semantic commands. The Presenter also loads the smart style, which then loads semantic images, motion, etc. consistent with the semantics of the request.
Figure 8 illustrates the presently preferred client component integration and interaction workflow described above.
23. Categories Dialog Box User Interface Specification a. Overview
The Categories Dialog Box allows the user to select one or more categories from a category folder (or taxonomy) belonging to a knowledge domain. While more or fewer can be deployed in certain situations, in the preferred embodiment, the dialog box has all of the following user interface controls:
1. Profile - this allows the user to select a profile with which to filter the category folders (or taxonomies) based on configured areas of interest. For instance, if a profile has areas of interest set to "Health and Medicine," selecting that profile will display only those category folders that belong to the "Health and Medicine" area of interest (for instance, Pharmaceuticals, Healthcare, and Genes). This control allows the user to focus on the taxonomies that are relevant to his/her knowledge domain, without having to see taxonomies from other domains. 2. Area of Interest - this allows the user to select a specific area of interest. By default, this combo box is set to "My Areas of Interest" and the profile combo box is set to "All Profiles." This way, the dialog box will display category folders for all areas of interest for all profiles. However, by using the "Area of interest" combo box, the user can directly specify an area of interest with which to filter the category folders, regardless of the areas of interest in his/her pro file(s).
3. Publisher Domain Zone/Name - this allows the user to select the domain zone and name of the taxonomy publisher. This is advantageous to distinguish publishers that might have name collisions. In the preferred embodiment, the Publisher Domain Name uses the DNS naming scheme (for instance, IEEE.org, Reuters.com). The domain zone allows the user to select the scope of the domain name. In the preferred embodiment, the options are Internet, Intranet, and Extranet. The zone selection further distinguishes the published category folder (or taxonomy). A fairly common case would be where a department in a large enterprise has its own internal taxonomy. In this case, the department will be assigned the Intranet domain zone and will have its own domain name - for instance, Intranet\Marketing or IntranefλSales.
4. Category Folder - this allows the user to select a category folder or taxonomy. When this selection is made, the categories for the selected category folder are displayed in the categories tree view.
5. Search categories - this allows the user to enter one or more keywords with which to filter the currently displayed categories. For instance, a Pharmaceuticals researcher could select the Pharmaceuticals taxonomy but then enter the keyword "anatomy" to display only the entries in the taxonomy that contain the keyword "anatomy."
6. "Remember" check box - this allows the user to specify whether the dialog box should "remember" the last search when it exits. This is very helpful in cases where the user might want to perform many similar category-based searches/requests from the same category folder and with the same keyword filter(s). 7. Search Options - these controls allow the user to specify how the dialog box should interpret the keywords. The options allow the user to select whether the keywords should apply to the entire hierarchy of each entry in the taxonomy tree, or whether the keywords should apply to only the [end] names of the entries. For instance, the taxonomy entry "Anatomy\Cells\Chromaffin Cells" will be included in a hierarchy filter because the hierarchy includes the word "Anatomy." However, it will be excluded from a names filter because the end-name ("Chromaffm Cells") does not include the word "Anatomy."
Also, the search options allow the user to select whether the dialog box should check for all keywords, for any keyword, or for the exact phrase.
8. Categories Tree View - the tree view displays the taxonomy hierarchy and allows the user to select one or more items to add to the Create Request Wizard or to open as a new Dossier (Guide) request/agent. The user interface breaks the category hierarchy into "category pages" - for performance reasons. The UI allows the user to navigate the pages via buttons and a slide control. There is also a "Deselect All" button that deselects all the currently selected taxonomy items.
9. Explore Button - this is the main invocation button of the dialog box. When the dialog box is launched from the Create Request Wizard, this button is renamed to "Add" and adds the selected items to the wizard "filters" property page. When the dialog box is launched directly from the application, the button is titled "Explore" and when clicked launches a Dossier request on the selected categories. If the user has multiple profiles or if multiple taxonomy categories are selected, the dialog box launches another dialog box, the "Explore Categories Options" dialog box that prompts the user to select the profile with which to launch the Dossier and/or the operator to use in applying the categories as filters to the Dossier (AND or OR).
The features described above are illustrated in Figures 9 - 11, which show three different views of the Explore Categories dialog box. 24. Client-Assisted Server Data Consistency Checking
As the server (KIS) crawls knowledge sources, there will be times when the server's metadata cache is out of sync with the sources themselves. For instance, a web crawler on the KIS that periodically crawls the Web might add entries into the semantic metadata store (SMS) that become out of date, hi this case, the client would get a 404 error when it tries to invoke the source URI. For data source adapters (DSAs) that have monitoring capabilities (for instance, for file-shares that can be monitored for changes), this wouldn't be much of an issue because the KIS is likely to be in sync with the knowledge source(s). However, for sources such as Web sites that don't have monitoring/change-notification services, this may present an issue of concern.
My parent application (Serial No. 10/179,651) described how the KIS can use a consistency checker (CC) to periodically purge stale entries from the SMS. However, in some situations this approach might impair perfonnance because the CC would have to periodically scan the entire SMS and confirm whether the indexed objects still exist. An alternative embodiment of this feature of the invention is to have the client (the semantic browser) notify the server if it gets a 404 error. To do this, the semantic browser would have to track when it gets a 404 error for each result that the user "opens." For Web documents, the client can poll for the HTTP headers when it displays the results, even before the user opens the results. In this case, if the source web server reports a 404 error (object not found), the client should report this to the KIS.
When the KIS gets a "404 report" from the client, it then intelligently decides whether this means the object is no longer available. The KIS cannot arbitrarily delete the object because it is possible that the 404 error was due to an intermittent Web server failure (for instance, the directory on the Web server could have been temporarily disabled). The KIS should itself then attempt to asynchronously download the object (or at the very least, the HTTP headers in the case of a Web object) several times (e.g., 5 times). If each attempt fails, the KIS can then conclude that the object is no longer available and remove it from the SMS. If another client reports the 404 error for the same object while the KIS is processing the download, the KIS should ignore that report (since it is redundant).
This alternate technique could be roughly characterized as lazy consistency checking, h some situations, it may be advantageous and preferred.
25. Client-Side Duplicate Detection
The server (KIS) performs duplicate detection by checking the source URIs before adding new objects into the semantic metadata store (SMS). However, for performance reasons, it is sometimes advantageous if the server does not perform strict duplicate-detection. In such cases, duplicate detection is best performed at the client. Furthermore, because the client federates results from several KISes, it is possible for the client to get duplicates from different KISes. As such, it is advantageous if the client also performs duplicate detection. hi the preferred embodiment, the client removes objects that are definitely duplicates and flags objects that are likely duplicates. Definite duplicates are objects that have the same URI, last modified time stamp, summary/concepts, and size. Likely duplicates are objects that have the same summary/concepts, but have different URIs, last modified times, or sizes. For objects for which summary extraction is difficult, it is recommended that the title also be used to check for likely duplicates (i.e., objects that have the same summary but different titles are not considered likely duplicates because the summary might not be a reliable indicator of the contents of the object). Also, if summary/concept extraction is difficult (in order to detect semantic overlap/redundancy), the semantic browser can limit the file-size check to plus or minus N % (e.g., 5%) - for instance, an object with the same summary/concepts and different URIs, last-modified times, and sizes might be disqualified as a likely duplicate if the file-size is within 5% of the file-size of the object it is being compared to for redundancy checking.
26. Client-Side Virtual Results Cursor
The client (semantic browser) also provides the user with a seamless user experience when there are multiple knowledge communities (agencies) subscribed to a user profile. The semantic browser preferably presents the results as though they came from one source. Similarly, the browser preferably presents the user with one navigation cursor - as the user scrolls, the semantic browser re-queries the KISes to get more results. In the preferred embodiment, the semantic browser keeps a results cache big enough to prevent frequent re-querying - for instance, the cache can be initialized to handle enough results for between 5-10 scrolls (pages). The cache size are preferably capped based on memory considerations. As the cursor is advanced (or retreated), the browser checks if the current page generates a cache hit or miss. If it generates a cache hit, the browser presents the results from the cache, else if re-queries the KISes for additional results which it then adds to the cache.
The cache can be implemented to grow indefinitely or to be a sliding window. The former option has the advantage of simplicity of implementation with the disadvantage of potentially high memory consumption. The latter option, which is the preferred embodiment, has the advantage of lower memory consumption and higher cache consistency but with the cost of a more complex implementation. With the sliding window, the semantic browser will purge results from pages that do not fall within the window (e.g., the last N - e.g., 5-10 - pages as opposed to all pages as with the other embodiment).
27. Virtual Single Sign-On
The client (semantic browser) also provides the user with a seamless user experience when authenticating the user to his/her subscribed knowledge communities (agencies). It does this via what the inventor calls "virtual single sign-on." This model involves the semantic browser authenticating the user to knowledge communities without the user having to enter his/her username and password per knowledge community. Typically, the user will have a few usernames and passwords but might have many knowledge communities of which he/she is a member (especially within a company based on departmental or group access, and on Internet- based knowledge communities). As such, the ratio of the number of knowledge communities to the number of authentication credentials (per user) is likely to be very high. With virtual single sign-on, the user specifies his/her logon credentials to the semantic browser in a server (knowledge community)-independent fashion. The semantic browser stores the credentials in a Credential Cache Table (CCT). The CCT has columns as illustrated below:
Account Name User Name Password Knowledge Community Entry List
• Account Name - this is a friendly name for the account
• User Name - this is the logon user name (e.g., an email address)
• Password - this is the password, stored encrypted with a secure private key
• Knowledge Community Entry List (KCEL) - this is a list of knowledge communities that authenticate the user using the credentials for this account
When the user first attempts to subscribe to a knowledge community (or access the knowledge community in some other way - for instance, to get the properties of the community), the semantic browser prompts the user for his/her password and then tries to logon to the server using the supplied credentials. If a logon is successful, the semantic browser creates a new CCT entry (CCTE) with the supplied credentials and adds the KC to the Knowledge Community Entry List (KCEL) for the new CCT entry.
For each subsequent subscription attempt, the semantic browser checks the CCT to see if the KC the user is about to subscribe to is in the KCEL for any CCTE. If it is, the semantic browser retrieves the credentials for the CCTE and logs the user on with those credentials. This way, the user does not have to redundantly enter his/her logon credentials.
Note that the semantic browser also supports pass-through authentication when the operating system is already logged on to a domain. For instance, if a Windows machine is already logged on to an NT (or Active Directory) domain, the client-side Web service proxy also includes the default credentials to attempt to logon to a KC. hi the preferred embodiment, the additional credentials supplied by the user are preferably passed via SOAP security headers (via Web Services Security (WS-Security) or a similar scheme). For details of WS-Security and passing authentication-information in SOAP headers, see http://www.oasis- open.org/committees/download.php/3281/WSS-SOAPMessageSecurity-17-082703-merged.pdf The semantic browser exposes a property to allow the user to indicate whether the credentials for a CCTE are preferably purged when the KCEL for the CCTE is empty or whether the credentials should be saved, hi the preferred embodiment, the credentials are preferably saved by default unless the user indicates otherwise. If the user wants the credentials purged, the semantic browser should remove a KC from a CCTE in which it exists when that KC is no longer subscribed to any profile in the browser. If after removing the KC from the CCTE's KCEL, the CCTE becomes empty, the CCTE is preferably deleted from the CCT.
The virtual single sign-on feature, like many of the features in this application, could be used in applications other than with my Information Nervous System or the Virtual Librarian. For example, it could be adapted for use by any computer user who must log into more than one domain.
28. Namespace Object Action Matrix
The table below shows the actions that the semantic browser invokes when namespace objects are copied and pasted onto other namespace objects.
Figure imgf000050_0001
Figure imgf000051_0001
29. Dynamic End-to-End Ontology/Taxonomy Updating and Synchronization
The Information Nervous System™ will support dynamic updates of ontologies and taxonomies. Knowledge domain plug-ins that are published by Nervana (or that are provided to Nervana by third-party ontology publishers) will be hosted on a central Web service (an ontology depot) on the Nervana Web domain (Nervana.com). Each KDS will then periodically poll the central Web service via a Web service call (for each of its knowledge domain plug-ins, referenced by the URI or a globally unique identifier of the plug-in) and will "ask" the Web service if the plug-in has been updated. The Web service will use the last-modified timestamp of the ontology file to determine whether the plug-in has been updated. If the plug-in has been updated, the Web service will return the new ontology file to the calling KDS. The KDS then replaces its ontology file.
If the KDS is running during the update, it will ordinarily temporarily stop the service before replacing the file, unless it supports file-change notifications and reloads the ontology (which is the recommended implementation).
Each KIS also has to poll each KDS it is connected to in order to "ask" the KDS if its ontology has changed. In the preferred embodiment, the KIS should poll the KDS and not the central Web service in case the KDS has a different version of the ontology. The KDS also uses the last modified time stamp of the knowledge domain plug-in (the ontology) to determine if the ontology has changed. It then indicates this to the KIS. If the ontology has changed, the KIS needs to update the semantic network accordingly. In the preferred embodiment, it does this by removing semantic links that refer to categories that are not in the new version of the ontology and adding/modifying semantic links based on the new version of the ontology. In an alternative embodiment, it purges the semantic network and re-indexes it. The client then polls each KIS it is subscribed to in order to determine if the taxonomies it is subscribed to (directly via the central Web service or via the KISes) have changed. The KIS exposes a method via the XML Web service via which the client determines if the taxonomy has changed (via the last modified time stamp of the taxonomy/ontology plug-in file). If the taxonomy has changed, the client needs to update the Categories Dialog user interface (and other Ul-based taxonomy dependents) to show the new taxonomy.
For taxonomies that are centrally published (e.g., via Nervana), the client should poll the central Web service to update the taxonomies.
With this model, the client, KIS, KDS, and central taxonomy/ontology depot will be kept synchronized.
30. Invoking Dossier (Guide) Queries
Dossier Semantic Query Processing
Dossier (Guide) queries are preferably invoked by the client-side semantic query processor by parsing the SQML of the request/agent and replacing the Dossier context predicate with each special agent (context template) context predicate - e.g., All Bets, Best Bets, Breaking News, Headlines, Random Bets, Newsmakers, etc. Each query (per context template) is then invoked via the query processor - just like an individual query. This way, the user operates at the level of the Dossier but the semantic browser maps the dossier to individual queries behind the scenes.
For example, the SQML for "Dossier on Category C" is parsed and new SQML queries are generated as follows:
All Bets on Category C Best Bets on Category C Breaking News on Category C Headlines on Category C Random Bets on Category C Newsmakers on Category C Etc. The client-side semantic query processor retains every other predicate except the context predicate. This way, the filters remain consistent as illustrated by the example above.
Dossier Smart Lens
Like other requests/agents in the Information Nervous System™, dossiers (guides) can be used as a Smart Lens (just like how they can be targets for drag and drop, smart copy and paste, etc.). In this case, the smart lens displays a "Dossier Preview Window" with sections/tabs/frames for each context template (special agent). Sample screenshots of the Dossier showing the UI of the Dossier Smart Lens are included in Figures 12 and 13.
Dossier Screenshots
31. Knowledge Community (Agency) Semantics
The following describe the semantics of a knowledge community (agency) within the context of the semantic namespace/environment in the semantic browser:
1. Selecting a knowledge community - this opens a dossier request from that KC. Essentially, the Dossier becomes the equivalent of the KC's "home page."
2. Drag and drop (document, text, entity, keywords, etc.) to a KC - this opens a Dossier request/agent on the object (using the default predicate) from the KC
3. Copy KC to the clipboard - this selects KC as the Smart Lens. When the user hovers over a result or entity, the semantic browser displays the Smart Lens by showing the KC name and the KC's profile name under the cursor and then opens a Dossier from the KC on the object underneath the lens in the lens preview pane
4. Subscribing to a KC - when a KC is subscribed for the first time, the semantic browser adds the KC's email address to the local email contacts (e.g., in Microsoft Outlook or Outlook Express). This makes it easy for the user to publish knowledge to the KC by sending it email (via the integrated contacts list). Similarly, when the KC is unsubscribed from all profiles, the semantic browser prompts the user whether it should remove the KC from the local email contacts list. 32. Dynamic Ontology and Taxonomy Mapping
One of the challenges of using taxonomies and ontologies is how to map the semantics of one taxonomy/ontology onto another. The Information Nervous System™ accomplishes this by the following algorithm:
Each KDS will be responsible for ontology mapping (via an Ontology Mapper (OM)) and will periodically update the central Web service (the ontology depot) with an Ontology Mapping Table (OMT). The updates are bi-directional: the KDS will periodically update its ontologies and taxonomies from the central Web service and send updates of the OMT to the central Web service. Each OMT will be different but the central ontology depot will consolidate all OMTs into a Master OMT. The ontology mapper will create a consistent user experience because the user wouldn't have to select all items in the umbrella taxonomy that are relevant but overlapping. The semantic browser will automatically handle this. The KIS wouldn't have any concept of the mapper but will get mapped results from the KDS which it will then use to update the semantic network.
The KDS and KIS administrators would still be responsible for selecting the right KDS ontology plug-ins, however - based on the quality of each ontology/taxonomy (the ontology mapping doesn't improve ontologies; it merely maps them).
33. Semantic Alerts Optimisations
Semantic Alerts in the semantic browser can be optimized by employing the following rule (in order):
For a given filter (e.g., result, document, text, keywords, entity):
1. Check for Headlines first.
2. If there are Headlines, check for Breaking News and Newsmakers.
This is because in the preferred embodiment, Headlines are implemented similar to Breaking News except with a larger time window. As a consequence, if there are no Headlines (in the preferred embodiment), there is no Breaking News. Also, in the preferred embodiment, Newsmakers are implemented by returning the authors of Headlines. As such, if there are no Headlines, there are no Newsmakers.
34. Semantic "News" Images
Both Corbis (http://www.corbis.com) and Getty Images (http://www.gettyimages.com) have "News" images that are constantly kept fresh. The Information Nervous System™ can use these kinds of images for semantic images that are not only context-sensitive but also "fresh." This can be advantageous in terms of keeping the user interface interesting and "new." For instance, "Breaking News on SARS" can show not only pharmaceutical images but images showing doctors responding to recent SARS outbreaks, etc.
35. Dynamically Choosing Semantic Images
Semantic images can be dynamically and intelligently selected using the following rules:
1. If the currently displayed namespace object is a request, parse the SQML of the object for categories. If there are categories, send the categories to the central Web service (that hosts the semantic image cache) to get images that are relevant to the categories. Also, send the request type (e.g., knowledge types like All Bets and Headlines, or information types like Presentations) to the central Web sendee to return images consistent with the request type
2. If the namespace object is not a request, send the areas of interest for the current profile (if available) to the central Web service. The Web service then returns semantic images consistent with the profile's areas of interest. If the profile does not have configured areas of interest, send the areas of interest for the application (the semantic browser). If the application does not have configured areas of interest, send an empty string to the central Web service - in this case, the central Web service returns generic images (e.g., branded images).
36. Dynamic Knowledge Community (Agency) Contacts Membership Knowledge communities (agencies) have members (users that have read, write, or read- write access to the community) and contacts. Contacts are users that are relevant to the community but are not necessarily members. For example, a departmental knowledge community (KC) in a large enterprise would likely have the members of the department as members of the KC but would likely have all the employees of the enterprise as contacts. Contacts are advantageous because they allow members of the KC to navigate users that are semantically relevant to the KC but might not be members. The KC might semantically index
/ sent by contacts - the index in this case would include the contacts even though the contacts are not members of the KC.
Another way to think of this is that communities of knowledge in the real world tend to have core members and peripheral members. Core members are users that are very active in the community while peripheral members include "other" users such as knowledge hobbyists, occasional contributors, potential recruits, and even members of other relevant communities.
With dynamic KC contacts membership in the Information Nervous System™, the KIS will add users to its Contacts table in the semantic metadata store (SMS) and to the semantic network "when and as it sees them" (in other words, as it indexes email messages that have new users that are not members). This allows the community to dynamically expand its contacts, but in a way that distinguishes between Members and mere Contacts, and "understands" the importance of the distinction semantically when operating the system (e.g., executing searches and the like).
37. Integrated Full-Text Keyword and Phrase Indexing
The KIS also indexes concepts (key phrases) and keywords as first-class members of the semantic network. This can be done in a domain-independent fashion as follows:
For each new object (e.g., documents) to be added to the semantic network:
1. Extract concepts (key phrases) from the body of the object.
2. For each concept, add the concept to the semantic network with the object type id OBJECTTYPEJJD_CONCEPT. Add a semantic link with the predicate PREDICATETYPEΓD_CONTALNSCONCEPT to the "Semantic Links" table with the new object as subject and the new concept object as the subject.
3. For the current concept, extract the keywords from the concept key phrase and add each keyword to the semantic network with the object type id OBJECTTYPEH>_KEYWORD. Also, add a semantic link with the predicate PREDICATETYPELD_CONTAΓNSKEYWORD to the "Semantic Links" table with the new object as subject and the new keyword object as the subject.
Repeat the steps above for the title of the object and other meta-tags as appropriate for the schema of the object.
While some embodiments do not require integrated full-text indexing, it is included in the presently preferred embodiment because it provides several useful advantages:
1. It allows a consistent model for implementing semantic filters (in SQML). The user can add categories, documents, entities, and keywords as filters and the filters are applied consistently to the semantic network (as sub-queries).
2. In particular, it supports the semantic query processing of entities. Entities can be defined with categories and can be further narrowed with keywords (to disambiguate the keywords in the case where the keywords could mean different things in different contexts). Integrated full-text indexing allows the KIS semantic query processor (SQP) to interpret entities seamlessly - by applying the necessary sub-queries with categories and keywords/concepts to the semantic network.
3. In general, integrated full-text indexing results in a seamless and consistent data and query model.
38. Semantic "Mark Object as Read55
In some cases, the KIS might not have the resources to store semantic links between People and objects on a per-object basis. In addition, semantic-based redundancy is not the same as per-object redundancy — as in email. To take an example, email clients allow users to select an email message as read or unread - this is typically implemented as a flag stored on the mail server with the email message. However, because email is not a semantic system, a semantically similar or identical message on the server would not be flagged as such - the user has to flag each message separately regardless of semantic redundancy.
In the Information Nervous System™, the user is able to flag an object as read not unlike in email. However, in this case, the semantic browser extracts the concepts from the object and informs all the KISes in the request profile that the "concepts" have been read. The KIS then dynamically maps the concepts to categories via the KDSes it is configured with and adds a flag to the objects belonging to those categories (in the preferred embodiment) and/or adds a flag to the semantic network with a semantic link with the predicate PREDICATETYPEID_VIEWEDCATEGORY between the categories corresponding to the concepts and all the objects that are linked to the categories. In the preferred embodiment, the KIS should only flag those categories over a link-strength threshold (for the source concepts). This ensures that only those objects (in the preferred embodiment) and/or categories that are semantically close to the original object will be flagged.
When the semantic browser flags the object via the KISes, the KISes should return a flag indicating whether the network was updated (it is possible that no changes would be made in the event that the object does not have any "strong" categories or if there are no other objects that share the same "strong" categories). If at least one KIS in the request profile indicates that the network was updated, the semantic browser should refresh the request/agent. The semantic browser can expose a property to allow the user to indicate whether he/she wants the KISes to return only unread objects or all objects (read or unread), in which case the browser should display unread objects differently (like how email clients display unread messages in a bold font). The presentation layer in the semantic browser should then display the read and unread objects with an appropriate font and/or color to provide a clear visual distinction.
39. Multi-Select Object Lens
Multi-select object lens is an alternative implementation of the object lens that was described in my parent application, hi that embodiment, the object lens was invoked via smart copy and paste - pasting an object over another object would invoke the object lens with the appropriate default predicate. This has the benefit of allowing the user to copy objects across instances of the semantic browser, across profiles, and from other environments (like the file- system, word processors, email clients, etc.). In the currently preferred embodiment, the object lens is a Dossier Lens (the context predicate is a Dossier, the filters are the source and target objects, and the profile is the profile in which the source object was displayed).
Multi-selection can also be used instead of copy and paste to invoke an object lens. The semantic browser will allow the user to select multiple objects (results). The user can then hit a button (or alternative user-interface object) to invoke the object lens on the selected objects. In this case, a Dossier Lens will be displayed (in a preview pane) with a Dossier context predicate, with the filters as the selected objects, and the current profile as the request profile.
40. Ontology-Based Filtering and Spam Management
The KIS (in the preferred embodiment) would only add objects to the Semantic Metadata Store (SMS) if those objects belong to at least one category from at least one of the knowledge domains the KIS is configured with (via one or more KDSes). This essentially means the KIS will not index objects it "does not understand." The exception to this is that the KIS will index all objects from its System Inbox - because this contains at-times personal community-specific publications and annotations that might be relevant but not always semantically relevant.
A side-effect of this ontology-based filtering model is spam management - ontology- based indexing would be effective in preventing spam from being indexed and stored. If users use the semantic browser to access email, as opposed to their inboxes, only email that has been semantically filtered will get through.
41. Results Refinement
The results of a request/agent can be further refined via additional filters and predicates. For example, the request/agent Headlines on Bioinformatics could be further refined with keywords specific to certain areas of Bioinformatics. This way, the end-user can further narrow the result set using the request/agent as a base. In addition, for time-sensitive requests, the user can specify a time-window to override the default time-window. For example, the default Breaking News time-request could be set to 3 hours. The user should be able to override this for a specific request/agent (in addition to changing the defaults on a per-profile or application-wide basis) with an appropriate UI mechanism (e.g., a slider control that ranges from 1 hour to 24 hours). The same applies to Headlines and Newsmakers (e.g., a slider control that ranges from 1 day to 1 week).
When the user specifies a filter-override, the semantic browser invokes the XML Web Service call for each of the KISes in the request profile and passes the override arguments as part of the call. If override arguments are present, the Web service uses those values instead of the default filter values. The same applies to additional filters (e.g., keywords) - these will be passed as additional arguments to the Web service and the Web service will apply additional sub-queries appropriately to further filter the query that is specified in the agent/request SQML (in other words, the SQML is passed as always, but in addition, the filter overrides and additional filters are also passed).
A good case for filter-overrides will be for Best Bets. The default semantic relevance strength for Best Bets could be set to 90% (in the preferred embodiment). However, for a given request/agent, the user might want to see "bets" across a semantic relevance range. Exposing a relevance UI control (e.g., a slider control that ranges from 0% to 100%) will allow this. This essentially allows the user to change the Best Bets on the fly from "All Bets" (0%) all the way to "Perfect Bets" (100%).
A hybrid model should also be employed for embodiments of context template (special agent) implementations that involve multiple axes of filtering. For instance, Breaking News could also impose a relevance filter of 25% and Headlines and Newsmakers could impose a relevance filter of 50% (Breaking News has a lower relevance threshold because it has a higher time-sensitivity threshold; as such, the relevance threshold can be relaxed), i this case, the semantic browser should expose UI controls to allow the user to refine the special agents across both axes (a slider control for time-sensitivity and another slider control for relevance).
With dossiers, the semantic browser can display UI controls for each special agent displayed in the Dossier - the main Dossier pane can show all the UI controls (changing any UI control would then refresh the Dossier sub-request for that special agent). Also, if the Dossier has tabs for each special agent, each tab can have a UI control specific to the special agent for the tab.
42. Semantic Management of Information Stores
The Information Nervous System™ can also be used to manage information stores such as personal email inboxes, personal contact lists, personal event calendars, a desktop file-system (e.g., the Microsoft Windows Explorer file-management system for local and network-based files), and also other stores like file-shares, content management systems, and web sites.
For client-based stores (such as email inboxes and file-systems), the client runtime of the semantic browser should periodically poll the store via a programmatic interface to check for items that have become redundant, stale, or meaningless. This would address the problem today where email inboxes keep growing and growing with stale messages that might have "lost their meaning and relevance." However, due to the sheer volume of information users are having to cope with, many computer users are losing the ability to manage their email inboxes themselves, resulting in a junk-heap of old and perhaps irrelevant messages that take up storage space and make it more difficult to find relevant messages and items.
The client runtime should enumerate the items in the user's information stores, extract the concepts from the items (e.g., from the body of email messages and from local documents) and send the concepts to the KISes in the user's profiles. In an alternative embodiment, only the default profile should be used. The client then essentially "asks" the user's subscribed KISes whether the items mean anything to them. In the preferred embodiment, the client should employ the following heuristics:
1. First, check for redundancy - by flagging (or deleting) duplicate email items, duplicate documents that share concepts and summaries (but perhaps with different titles or file- sizes). The client should either delete the duplicate items (user-configurable) or flag the items by moving them into a special folder (user-configurable) in the email client or desktop.
2. Next, for non-duplicate items, the client should check for meaninglessness or irrelevance. First, the client should only check items that are "older" than N days (e.g., 30 days) by examining the last-modified time of the email item, document, or other object. For items that qualify, extract the concepts and call the XML Web Service for each KIS in all the user's profiles (or the default profile in an alternative embodiment).
3. For very old items (e.g., older than 180 days), the client should specify a very low threshold of meaning to the XML Web Service (e.g., 25%) for preservation. Essentially, this is akin to deleting (or flagging) those items that are very old and weak in meaning.
4. For fairly old items (e.g., older than 90 days old but younger than 180 days old), the client should specify a very low threshold (e.g., 10%) for preservation. This is akin to deleting (or flagging) those items that are fairly old and very weak in meaning.
5. For old items (but not too old - e.g., older than 1 day old but younger than 30 days old), the client should specify a very low threshold (e.g., 0%) for preservation. This is akin to deleting (or flagging) those items that are old (but not too old) but are meaningless, based on the user's profile(s).
Essentially, the model for this aspect or feature of the preferred embodiment balances semantic sensitivity with time-sensitivity by imposing a higher semantic threshold on younger items (thereby preserving items that might be largely - albeit not totally - meaningless if they are fairly young. For example, fairly recent email threads might be very weak in meaning - the client should preserve them anyway because their "youth" is also a sign of relevance. As they "age," however, the client can safely delete them (or flag them for deletion).
This model can also be applied to manage documents on local file-systems. The model can be extended to content-management systems, document repositories, etc. by configuring an Information Store Monitor (ISM) to monitor these systems (via calls to the Information Nervous System™ XML Web Services) and configuring the ISM with KISes that are configured with KDSes that have ontologies consistent with the domain of the repositories to be semantically managed. This feature will save storage space and storage/maintenance costs by semantically managing content management systems and ensuring that only relevant items get preserved on those systems over time.
43. Slide-Rule Filter User Interface
The refinement pane in the semantic browser allows the user to "search within results." The user will be able to add additional keywords, specify date ranges, etc. The date-range control can be implemented like a slide-rule. Shifting one panel in the slide-rule would shift the lower date boundary while moving the other panel will shift the upper date boundary. Other panels can then be added for time boundaries - shifting both time and date panels will impose both date and time constraints. Panels can also be added for other filter axes.
C. SERVER-SIDE SEMANTIC QUERY PROCESSOR SPECIFICATION
1. Overview
This section describes a currently preferred embodiment of how the server-side semantic query processor (SQP) resolves SQML queries. On a given server, queries can be broken into several components: a. Context (documents, keywords, entities, portfolios (or entity collections)). b. Context/Knowledge Template (or Special Agent) or Information Template - this describes whether the request if for a knowledge type (e.g., Breaking News, Conversations, Newsmakers, or Popular Items) or for a particular information type (e.g., Documents, Email).
On the client, a semantic query is made up of the triangulation of context, request (or Agent) type, and the knowledge communities (or Agencies). The client sends the SQML that represents the semantic query to all the knowledge communities in the profile in which the request lives. The client asks for a few results at a time and then aggregates the results from one or more servers.
The server-side semantic query processor subdivides semantic queries into several sub- queries, which it then applies (via SQL inner joins or sub-queries in the preferred embodiment). These sub-queries are:
1. Request type sub-query — this represents a sub-query (semantic or non-semantic) depending on the request type. Examples are context (knowledge) types (e.g., All Bets, Best Bets, Headlines, Experts, etc.) and information types (like General Documents, Presentations, Web Pages, Spreadsheets, etc.).
2. Semantic context sub-query - this represents a semantic sub-query derived from the context (filter) passed from the client (an example of this is categories sent from the client or mapped from keywords/text via semantic stemming).
3. Non-semantic context sub-query - this represents a non-semantic sub-query derived from the context (filter) passed from the client (examples are keywords without semantic stemming - mapping to ontology-based categories).
4. Access-control sub-query - this represents a sub-query that filters out those items in the semantic metadata store (SMS) that the calling user does not have access to. For details, see the "Security" specification. The foregoing steps are illustrated in Figure 14 (Server-Side Semantic Query Processor Components). Figure 14 shows how the server-side semantic query processor processes incoming semantic queries (represented as SQML).
2. Semantic Relevance Score
The semantic relevance score defines the normalized score that the concept extraction engine returns. It maps a given term of "blob" of text to one or more categories for a given ontology. The score is added to the semantic network (in the "LinkStrength" field of the "SemanticLinks" table) when items are added to the Semantic Network.
3. Semantic Relevance Filter
The relevance filter is different from the relevance score (indeed, both will typically be combined). The relevance filter indicates how the SQP will semantically interpret context (note: in the currently preferred embodiment, the filtering is always semantic in this case). There are two relevance filters: High and Low. With the High relevance filter, the SQP will include a sub- query that is the intersection of categories and terms. For instance, context for the keyword "XML" will be interpreted as: Items that share the same categories as XML and also include the keyword "XML." This is the highest level of ontology-based semantic filtering that can occur. However, it could lead to information loss in cases where there are objects in the Semantic Network (or Semantic Metadata Store (SMS)) that are semantically equivalent to the context but that do not share its keywords or terms. For instance, the query described above would miss items that share the same categories as XML but which include the term "Extensible Markup Language" instead. A Low relevance filter will only include objects that share the same categories as the context but unlike the High relevance filter, would not include the additional constraint of keyword equivalence.
For this reason, the relevance filter is preferably used only to create sub-query "buckets" that are then used for ordering results. For instance, the SQP might decide to prioritize a High relevance filter ahead of a Low relevance filter when filtering the semantic network but would still return both (with duplicates removed) in order to help guarantee that synonyms don't get rejected during the final semantic filtering process.
4. Time-Sensitivity Filter
The time-sensitivity filter determines how time-critical the semantic sub-query is. There are two levels: High and Low. A High filter is meant to be extremely time-critical. Default is 3 hours (this accounts for lunch breaks, time away from the office/desk, etc.). A Low filter is meant to be moderately time-critical. The default is 12 hours.
5. Knowledge Type Semantic Query Implementations
Throughout this application certain specific knowledge types are referred to by apt shorthand names, some of which the applicant uses or may use as trademarks. This section explains the nature and function of some of these in greater detail. a. All Bets
For "All Bets" queries, the server simply returns all the items in the semantic metadata store. If the SQML has filters, the filters are imposed via an inner sub-query with no semantic link strength threshold. For instance, All Bets on Topic A will return all items that have anything (strongly or barely) to do with Topic A. b. Random Bets the preferred embodiment, for "Random Bets" queries, the server simply returns all the items in the semantic metadata store (like in the case of "All Bets" queries) but orders the results randomly. If the SQML has filters, the filters are imposed via an inner sub-query with no semantic link strength threshold. For instance, Random Bets on Topic A will return all items (ordered randomly) that have anything (strongly or barely) to do with Topic A. c. Breaking News
If the server has user-state, Breaking News can be implemented in a very intelligent way. The table below illustrates the currently preferred ranking and prioritization for Breaking News when the server tracks what items (and/or categories) the user has read:
Figure imgf000066_0001
hi the preferred embodiment, the server processes SQML for Breaking News (via the Breaking News context predicate) as follows:
1. All breaking news is filtered with a sub-query that the returned news must be "younger" than N hours (or days, or months, configurable) - this imposes the key time- sensitivity constraint.
2. Breaking News is always semantic.
3. In the preferred embodiment, the Semantic Network Manager (SNM) should update the semantic network to indicate the "last read time" for each user to each category. This is then used in the sub-query to check whether news has been "read" or not (per category or per object - per category is the preferred embodiment because the latter will not scale).
4. Priority is given to news items that the user has not "read" (this is implemented by comparing the last read time in the SemanticLinks table with the semantic link type that links "User" to "Category").
5. The implication of the semantic prioritization scheme is that the user could get "older" breaking news first because the news is more semantically relevant and "younger" breaking news "later" because the news is less semantically relevant. This results in a hybrid relevance-time sensitivity prioritization scheme.
6. The primary ordering axis (Creation Time) guarantees that results are filtered by freshness. The secondary ordering axis (Relevance Score) acts as a tiebreaker and guarantees that equally fresh results are distinguished primary based on relevance.
7. Breaking News Intrinsic Alerts can be implemented on the client by limiting the Brealdng News priority to Priority 2 and by changing the Priority 1 and Priority time-sensitivity filters to high. This way, only very fresh Breaking Unread Semantic News (of both High and Low semantic relevance filters) will be returned. This is advantageous because the alert should have a higher disruption threshold than the Breaking News Request (or agent) - since it is implicit rather than explicit.
8. Unread Breaking News is higher priority than Read Breaking News because users are likely to be more interested in stuff they haven't seen yet.
9. Unread Breaking News has a lower time-sensitivity filter than Read Breaking News because users are likely to be more tolerant of older news that is new to them than younger news that is not. hi some cases, the server might not have user-state (and "read" information). In this case, a simple implementation of Breaking News is shown below:
1. By default (no filter), Breaking News should return only items younger than N hours (default is 3 hours).
2. If there is at least one filter in the SQML, Breaking News should apply the time- sensitivity filter (3 hours) to the outer sub-query and also apply a moderately strong relevance filter to the inner sub-query (off the SemanticLinks table). In the preferred embodiment, this should correspond to a relevance score (and link strength) of 50%. For instance, Breaking News on Topic A should return those items that have been posted in the last 3 hours and which belong to the category (or categories) represented by Topic A with at least a relevance score of 50%. This will avoid false positives like Breaking News items which are barely relevant to Topic A. d. Headlines
Ditto with Breaking News (except that time-sensitivity constraints are more relaxed - e.g., the High filter is 12 hours instead of 3 hours and the low filter is 1 day instead of 12 hours). In the simple implementation, the time-sensitivity constraint is 1 day. This can also be made 3- days on Mondays to dynamically handle weekends (making the number of days the "number of working days"). e. Newsmakers
Newsmakers are handled the same way as Headlines, except that the SQP returns the authors of the Headline items rather than the items themselves. f. Best Bets
As described in my parent application (Serial No. 10/179,651), Best Bets are implemented by imposing a filter on the strength of the semantic link with the "Belongs to Category" predicate. The preferred default is 90%, although the client (at the option of the user) can change this on the fly via an argument passed via the XML Web Service. Best Bets are implemented with a SQL inner join between the Objects table and the SemanticLinks table and joining only those rows in the SemanticLinks table that have the "Belongs to Category" predicate and a LinkStrength greater than 90% (default). When the SQML that is being processed contains filters (e.g., keywords, text, entities, etc.), the server-side semantic query processor must also invoke a sub-query, which is a SQL inner join that maps to the desired filters, i the preferred embodiment, this sub-query should also include a "Best Bets" filter.
In the preferred embodiment, it is advantageous and probably preferable for most users for the outer sub-query to be a Best Bet, and for the inner sub-query. To illustrate this, "Best Bets on Topic A" is semantically different from "Best Bets that are also relevant to Topic A." In the first example, only Best Bets, which are Best Bets "ON" Topic A, will be returned (via applying the "Best Bets" semantic filter on the inner sub-query). In contrast, the second example will return Best Bets on anything that might have anything to do with Topic A. As such, the second example might return false positives because for example, a document, which is a Best Bet on Topic B but a "weak bet" on Topic B, will be returned and that is not consistent with the semantics of the query or the presumably desired results. Extending the "Best Bets" filter to not only the outer sub-query but also all inner sub-queries will prevent this from happening. Other query implementations can also follow this rule (with the right sub-queries applied based on the semantics of the main query) if the SQML contains filters. g. Query Implementation for Other Knowledge Types
Other knowledge types are implemented in a similar fashion as above (via the right predicates). Several examples are described below.
Information Type Semantic Query Implementations
All information type semantic query implementations can follow, and preferably (but not necessarily) follow, the same pattern: the SQP returns only those objects that have the object type id that corresponds to the requested information type. An example is "Information TypeXPresentations." When the SQP parses the SQML received from the client, it extracts this attribute from the SQML and maps it to an object type id. It then invokes a SQL query with an added filter for the object type id. For special information types that could span several individual information types (such as "Information Type\All Documents"), the SQP maps the request to a set of object type ids and invokes a SQL query with this added filter.
Context Semantic Query Implementations
When the client sends SQML that contains concepts (extracted on the client from text or documents), the server-side SQP has to first semantically interpret the context before generating sub-queries that correspond to it. To do this, the server sends the concepts to all KDS'es (KBS'es) it is configured with (for the desired knowledge community or agency) for semantic categorization. When the server gets the categories back, it preferably determines which of those categories are "strong" enough to be used as filters before generating the appropriate sub-queries.
This "filter-strength" determination is advantageous because if the context is, for example, a fairly long document, that document could contain thousands of concepts and categories. As a result, the "representative semantics" of the document might be contained in only a subset of all the concepts/categories in the document. Mapping all the categories to sub- queries will return results that might be confusing to the user - the user would likely have a "sense" of what the document contains and if he/she sees results that are relevant to some weak concepts in the document, the user might not be able to reconcile the results with the document context. Therefore, in the preferred embodiment, the server-side SQP preferably chooses only "strong categories" to apply to the sub-queries. It is recommended that these be categories with a semantic strength of at least 50%. That way, only those categories that register strongly in the semantic context would be applied to the sub-query. The implementation of the sub-query would then follow the rales described above depending on whether the query contains a context predicate, is based on a knowledge type, information type, etc.
Semantic Stemming Implementation
As described in my parent application, the server-side semantic query processor performs semantic stemming to map keywords, text, and concepts to categories based on one or more domain ontologies. One way it does this by invoking an XML Web Service call to the KDS/KBS (or KDSes/KBSes) it is configured with in order to obtain the categories. It then maps the categories to its semantic network. This form of stemming is superior to regular stemming that is based on keyword variations (such as singular and plural variations, tense variations, etc.) because it also involves domain-specific semantic mapping that stems based on meaning rather than merely stemming based on keyword forms.
In the currently preferred embodiment, the KIS calls the KDS/KBS each time it receives SQML that requires further semantic interpretation. However, this could result in delays if the KDS/KBS resides on a different server, if the network connection is not fast, or if the KDS/KBS is busy processing many requests, hi this case, the KIS can also implement a Semantic Stemming Cache. This cache maps keywords and concepts to categories that are fully qualified with URIs (making them globally unique). When the server-side semantic query processor receives SQML that contains keywords, text, or concepts (extracted from, say, documents on the client by the client-side semantic query processor), it first checks the cache to see if the keywords have already been semantically stemmed. If there is a cache hit, the SQP simply retrieves the categories from the cache and maps those categories to the semantic network via SQL queries. If there is a cache miss (i.e., if the context is not in the cache), it then calls the KDSes/KBSes to perform semantic categorization. It then takes the results, maps them to unique category URIs, and adds the entry to the cache (with the context as the hash code). Note that even if the context does not map to any category, the "lack of a category" is preferably cached. In other words, the context is added as a cache entry with no categories. This way, the server can also quickly determine that a given context does not have any categories, without having to call the KDSes/KBSes each time to find out.
Cache Management
The SQP can also manage the semantic stemming cache. It has to do this for two reasons: first, to keep the cache from growing uncontrollably and consuming too much system resources (particularly memory with a heap-based hash table); and, second, if the KIS configuration is changed (e.g., if knowledge domains are added/removed), the cache is preferably purged because the entries might now be stale. The first scenario can be handled by assigning a maximum number of entries to the cache, the preferred embodiment, the SQP caches the current amount of memory consumed by the cache and the cache limit is dictated by memory usage. For example, the administrator might set the maximum cache size to 64MB. To simplify the implementation, this can be mapped to an approximate count of items (e.g., by dividing the maximum memory usage by an estimate of the size of each cache entry).
For each new entry, if the cache limit has not been reached, the SQP simply adds the entry to the cache. However, if the cache limit has been reached, the SQP (in the preferred embodiment) should purge the least recently added items from the cache. In the preferred embodiment, this can be implemented by keeping a queue of items that is kept in sync with a hash table that implements the cache itself (for quick lookups using the context as a key). When the SQP needs to purge items from the cache to free up space, it de-queues an item from the least-recently-added queue and also removes the corresponding item from the hash table (using the context as key). This way, fresh items are more likely to result in a cache hit than older items. This will result in a faster user experience on the client because context for saved agents/requests/queries will end up being cached with quick-lookups each time the user opens the agent/request/query. The same goes for Dossier (Guide) queries which will have the same context (but with different knowledge types) - the client will request for each knowledge type for the same context and since the context will be cached, each sub-query will execute faster.
D. EXTENSIBLE CLIENT-SIDE USER PROFILES SPECIFICATION FOR THE INFORMATION NERVOUS SYSTEM
Overview
Extensible client-side user profiles allow the user of a semantic browser to have a different state for different job roles, knowledge sources, identities, personas, work styles, etc. This essentially allows the user to create different "knowledge worlds" for different scenarios. For instance, a Pharmaceuticals researcher might have a default profile that includes all sources of knowledge that are relevant to his/her work. As described in my parent application Serial No. 10/179,651, the SRML from each of these sources will be merged on the client thereby allowing the user to seamlessly go through results as though they were coming from one source. However, the researcher might want to track patents separate from everything else. In such a case, the researcher would be able to create a separate "Patents" profile and also include those knowledge communities (agencies) that have to do with patents (e.g.,. the US Patent Office Database, the EU Patent Database, etc.)
To take another example, for instance, the user might create a profile for 'Work' and one for 'Home.' Many investment analysts track companies across a variety of industries. With the semantic browser, they would create profiles for each industry they track. Consultants move from project to project (and from industry to industry) and might want to save requests and entities created with each project. Profiles will be used to handle this scenario as well. Profiles contain the following user state:
• Name/Description - the descriptive name of the profile.
• One or more knowledge commumties (agencies) that indicate the source of knowledge (running on a KIS) at which requests (agents) will be invoked.
• Identity Information - the user name (currently tagged with the user's email address) and password.
• Areas of Interest or Favorite Categories - this is used to suggest information communities (agencies) to the user (by comparing against information communities with identical or similar categories) and as a default query filter for requests created with the profile.
• Smart styles - the smart styles to be used by default for requests and entities created with the profile.
• Default Flag - this indicates whether the profile is the default profile. The default profile is initiated by default when the user wishes to create requests and entities, browse information communities, etc. Unless the user explicitly selects a different profile, the default profile gets used.
Profiles can be created, deleted, modified, and renamed. However, in the preferred embodiment the default profile cannot be deleted because there has to be at least one profile in the system at all times. In alternate embodiments, a minimum profile would not be required.
Preferably, all objects in the semantic browser are opened within the context of a profile. For instance, a smart request is created in a profile and at runtime, the client semantic query processor will use the properties of the profile (specifically the subscribed knowledge communities (agencies) in that profile) to invoke the lequest. This allows a user to correlate or scope a request to a specific profile based on the knowledge characteristics of the request (more typically the sources of knowledge the user wants to use for the request).
Figure 15 illustrates the semantic browser showing two profiles (the default profile named "My Profile" and 15Aand a profile named "Patents" 15B). Observe how the user is able to navigate his/her knowledge worlds via both profiles without interference.
Figures 16A-C illustrate how a user would configure a profile (to create a profile, the user will use the "Create Profile Wizard" and the profile can then be modified via a property sheet as shown).
Figure 17 shows how a user would select a profile when creating a request with the "Create Request Wizard." E. SMART STYLES SPECIFICATION FOR THE INFORMATION NERVOUS SYSTEM
1. Smart Styles Overview
A color theme and animation theme applied to a style theme yields a "smart style". "Smart" in this context means the style is adaptive or responsive to the mood of its request, context panes, preview mode, handheld mode, live mode, slideshow mode, Screensaver mode, ' blender/collection mode, accessibility, user settings recognition, and possibly other variables within the system (see below). There is an infinite number and kind or "Classes" of possible styles. The preferred embodiment comprises at least the following style Classes:
1. Subtle - for task-oriented productivity.
2. Moderate - for task-oriented productivity with some presentation effects.
3. Exciting - exciting effects (good for both primary and secondary machines, and for inactive Nervana windows - e.g., Nervana client windows in the background or docked on the taskbar).
4. Super-exciting (great for smart screensavers with productivity - e.g., secondary machines - when the user is using his/her primary machine).
5 Sci-Fi (for Matrix fans, great for smart screensavers without specific need for productivity - e.g., when the user is away from his/her desk).
Style, Color & Animation Themes - Variable, unlimited - created by Nervana, and perhaps users and/or third party skin authors
2. Implicit and Dynamic Smart Style Properties a. Mood - the smart style must convey the mood of the request (i.e., the request is a parameter passed to the smart style). This will involve semantic images, semantic motions, Visualizations, etc. that convey the semantically informed or semantically determined properties of the smart request (the context template or information type, the categories, whether there are filters (e.g., local documents), the information types of those filters, etc.) b. Context panes - e.g., deep info pane (per object), dockable preview panes, dockable contextual PU? watch groups/panes, etc. c. Preview Mode - each smart style must be able to display its results for preview (in a small window). d. Handheld Mode - each smart style must be able to display its results optimized for a handheld device. e. Live mode - each smart style must have a "live" mode during which it would display real-time semantic Visualizations (per object). This can be toggled on or off (e.g. if the user does not want real-time semantic Visualizations, or to save bandwidth that results from real-time Web service calls per object). f. Slideshow mode - preferably, each smart style must be able to "play" the results of the request - like a live stream. g. Screensaver mode - preferably, each smart style must be able to "play" the results of the request as a screensaver. This is a variant of slideshow mode, except in fullscreen theater mode. h. Blender/collection mode - preferably, each smart style must change its UI appropriately if the request it is displaying is a blender/collection. i. Accessibility - preferably, each smart style must support accessibility. j. User settings recognition - the Nervana Librarian will allow users to indicate whether they are beginners, moderate users, or power-users, and their respective job function s) (R&D, sales, marketing, executive, etc.). Preferably, each smart style considers (or is influenced by) these functions where appropriate.
• Preferably, each smart style is responsible, consistent with the semantics of the request, for recognizing (or discerning or perceiving) and then Visualizing (or presenting or depicting or illustrating, consistent with what should deserve the user's attention): the Mood of the Current Request (including semantic images, motion, chrome, etc. a Change in the number of Items in the Current Request the Mood of each object (intrinsically) the Mood of each object's context (headlines, breaking news, experts, etc.)
Binary/Absolute issues or characteristics (e.g., is there breaking news, OR NOT? how many experts are there? how many headlines?) as distinct from issues that are matters of degree, or on a gradient or continuum
If the characteristic is on a gradient or continuum, perceiving the relative placement along it (e.g., how breaking is breaking news?, how critical are the headlines? what is the level of expertise for the experts?, etc.) a change in each object's context (there is new breaking news, there are new annotations, etc.) the RELATIVE criticality of each object being displayed (different sized view ports, different fonts, different chrome, etc.) a request navigation and "loading" status (interstitials that INTRODUCE the mood of the new request being loaded)
• all properties of any individual PIP windows (animated with an animation control)
• the addition of a new PLP window (to a PIP window palette)
• any Resizing/Moving/Docking PLP Windows
• any preview windows (for context palettes, "Visualization UI" on each object, timelines, etc.)
• Sounds consistent with all of the foregoing Visualizations of mood and notifications (across the board)
Figure 18 shows a screenshot with the 'Smart Styles' Dialog Box illustrating some of the foregoing operations and features. As can be seen, the Dialog Box allows the user to browse smart styles by pivoting across style classes, style themes, color themes, and animation themes.
A preview window shows the user a preview of the currently selected smart style.
F. SMART REQUEST WATCH SPECIFICATION FOR THE INFORMATION NERVOUS SYSTEM
1. Overview
Smart Request Watch refers to a feature of the information Nervous System that allows users of the semantic browser (the Infoπnation Agent or the Librarian) to monitor (or "watch") smart requests in parallel. This is a very advantageous feature in that it enhances productivity by allowing users to track several requests at the same time.
- The feature is implemented in the client-side semantic runtime, the semantic browser, and skins that allow a configurable way of watching smart requests (via a mechanism similar to "Picture-fri-Picture" (PIP) functionality in television sets). Preferably, one or more of the following software components are used:
1. The Request Watch List (RWL)
2. Request Watch Groups
3. The Notification Manager (NM)
4. Watch Group Monitors (WLM)
5. The Watch Pane
6. The Watch Window 2. Request Watch Lists (RWLs) and Groups (RWGs)
The Request Watch List is a list of smart requests (or smart agents) that the client runtime manages. This list essentially comprises the smart requests the user wishes to monitor. The Request Watch List comprises a list of entries, the Request Watch List Entry (RWLE) with the following data structure:
Figure imgf000077_0001
The Request Watch List (RWL) contains an array or vector of RWLE structures. The Request Watch List Manager manages the RWL. The semantic browser provides a user interface that allows the user to add smart requests to the RWL - the UI talks to the RWLM to add and remove RWLEs to/from the RWL. The RWL is stored (and persisted) centrally by the client-side semantic runtime (either as an XML file-based representation or in a store like the Windows registry).
The RWL can also be populated by means of Request Watch Groups (RWGs). A Request Watch Group provides a means for the user to monitor a collection of smart requests. It also provides a simple way for users to have the semantic browser automatically populate the RWL based on configurable criteria. There are at least two types of RWGs: Auto Request Watch Groups and the Manual Request Watch Group. Auto Request Watch Groups are groups that are dynamically populated by the semantic browser depending on the selected profile, the profile of the currently displayed request, etc. The Manual Request Watch Group allows the user to manually populate a group of smart requests (regular smart requests or blenders) to monitor as a collection. The Manual Request Watch Group also allows the user to add support context types (e.g., documents, categories, text, keywords, entities, etc.) - in this case, the system will dynamically generate the semantic query (SQML) from the filter(s) and add the resulting query to the Manual Request Watch Group. This saves the user from having to first create a time- sensitive request based on one or more filters before adding the filters to the Watch Group - the user can simply focus on the filters and the system will do the rest.
Users will be able to add the following types of Auto-RWGs (for one or more configurable profiles, including "All Profiles" as shown in the Smart Request Watch Dialog Box in Figure 19):
1. Breaking News - this tells the semantic browser to automatically add a Breaking News smart request to the RWL (for the selected profile(s)).
2. Headlines - this tells the semantic browser to automatically add a Headlines smart request to the RWL (for the selected profile(s)).
3. Newsmakers - this tells the semantic browser to automatically add a Newsmakers smart request to the RWL (for the selected profile(s)).
4. Categorized Breaking News - this tells the semantic browser to automatically add Categorized Breaking News smart requests to the RWL (for the contextual profile). The semantic browser will dynamically add smart requests with category filters corresponding to each subcategory of the currently displayed smart request (and for the contextual or current profile) - if the currently displayed smart request has categories. For example, if the smart request "Breaking News" about Technology" is currently being displayed in a semantic browser instance, and if the category "Technology" has 5 sub-categories (e.g., Wireless, Semiconductors, Nanotechnology, Software, and Electronics), the following smart requests will be dynamically added to the RWL when the current smart request is loaded:
Breaking News about Technology. Wireless [<Contextual Profile Name>] Breaking News about Technology.Semiconductors [<Contextual Profile Name>] Breaking News about Technology .Nanotechnology [<Contextual Profile Name>] Breaking News about Technology. Software [<Contextual Profile Name>] Breaking News about Technology.Electronics [<Contextual Profile Name>]
Also, the RWLEs for these entries will be initialized with the RequestViewInstancelD of the current semantic browser instance. If the user navigates to a new smart request, the categorized Breaking News for the previously loaded smart request will be removed from the
RWL and a new list of categorized Breaking News will be added for the new smart request (if it has any categories) - and initialized with a new RequestViewInstancelD corresponding to the new smart request view. This creates a smart user experience wherein relevant categorized breaking news (for subcategories) will be dynamically displayed based on the currently displayed request. The user will then be able to monitor Categorized Breaking News smart requests as a watch group or collection.
5. Categorized Headlines - this tells the semantic browser to automatically add Categorized Headlines smart requests to the RWL (for the contextual profile). This is similar to Categorized Breaking News, except that Headlines are used in this case. The user will then be able to monitor Categorized Headlines smart requests as a watch group or collection.
6. Categorized Newsmakers - this tells the semantic browser to automatically add Categorized Newsmakers smart requests to the RWL (for the contextual profile). This is similar to Categorized Breaking News, except that Newsmakers are used in this case. The user will then be able to monitor Categorized Newsmakers smart requests as a watch group or collection.
7. My Favorite Requests - this tells the semantic browser to automatically add all favorite smart requests to the RWL (for the selected profile(s)). This allows the user to watch or monitor all his/her favorite smart requests as a group. 8. My Favorite Breaking News - this tells the semantic browser to automatically add all favorite breaking news smart requests to the RWL (for the selected profile(s)). This allows the user to watch or monitor all his/her favorite breaking news smart requests as a group.
9. My Favorite Headlines - this tells the semantic browser to automatically add all favorite headlines smart requests to the RWL (for the selected profile(s)). This allows the user to watch or monitor all his/her favorite headlines smart requests as a group.
10. My Favorite Newsmakers - this tells the semantic browser to automatically add all favorite newsmakers smart requests to the RWL (for the selected profile(s)). This allows the user to watch or monitor all his/her favorite newsmakers smart requests as a group.
Request Watch Group Manager User Interface
Figure 19 illustrates the "Smart Request Watch" Dialog Box in the semantic browser of the preferred embodiment. The top half of the dialog is used to add auto-watch groups. The user can select auto-watch group types and profile types ("All Profiles," "Contextual Profile," and the actual profile names) and add them to the auto-watch-group list. The user can also remove auto- watch-groups. The bottom half of the dialog box is used to add/remove smart requests to/from the manual watch group.
3. The Notification Manager (NM)
In the preferred embodiment the Notification Manager (NM) is a component of the semantic runtime client that monitors smart requests in the RWL. The NM has a thread that periodically invokes each smart request in the RWL (via the client semantic query processor) and updates the RWLE with the "results count" and the "last update time." i the preferred embodiment the NM preferably invokes the smart requests every 5-30 seconds. The NM can intelligently adjust the periodicity or frequency of request checks depending on the size of the RWL (in order to minimize bandwidth usage and the scalability impact on the Web service).
For time-sensitive smart requests (like Breaking News, Headlines, and Newsmakers), the NM preferably invokes the smart request without any additional time filter. However, for non time-sensitive requests (like for information as opposed to context types or for non time-sensitive context templates like Favorites and Recommendations), the NM preferably invokes the query for the smart request with a time filter (e.g., the last 10 minutes).
4. Watch Group Monitors
In the preferred embodiment, the semantic runtime client manages what the inventor calls Watch Group Monitors (WGM). For each watch group the user has added to the watch group list, the client creates a watch group monitor. A watch group monitor tracks the number of new results in each request in its watch group. The watch group monitor creates a queue for the RWLEs in the watch group that have new results. The WGM manages the queue in order to maximize the freshness of the results. The WGM periodically polls the NM to see whether there are new results for each request in its watch group. If there are, it adds the request to the queue depending on the 'last result time' of the request. It does this in order to prioritize requests with the freshest results first. The currently displayed visual style (skin) running in the Presenter would then call the semantic runtime OCX to dequeue the requests in the WGM queue. This way, the request watch user interface will be consistent with the existence of new results and the freshness of the results. Once there are no more new results in the currently displayed request, the smart style will dequeue the next request from the WGM queue.
5. The Watch Pane
The Watch Pane (WP) refers to a panel that gets displayed in the Presenter (alongside the main results pane) and which holds visual representations of the user's watch groups. The WP allows the user to glance at each watch group to see whether there are new results in its requests. The WP also allows the user to change the current view with which each watch group's real-time status gets displayed. The following views are currently defined:
Tiled View - this displays the title of the watch group along with the total number of new results in all its smart requests.
• Ticker View — this displays the total number of new results in all the watch group's smart requests but also shows an animation that sequentially displays the number of new results in each smart request (as a ticker). • Preview View - this is similar to the ticker view except that the most recent result per smart request is also displayed alongside the number of new results in the ticker.
• Deep View - in this view, the WP displays the total number of new results in all the watch group's smart requests along with a ticker that shows the number of new results in each smart request and a slide-show of all the new results per smart request.
6. The Watch Window
The WP also allows the user to watch a watch group. The user will do this by selecting one of the watch groups in the WP and dragging it into the main results pane (or by a similar technique). This forms a Watch Window (WW). This WW resembles or can be analogized to TV's picture-in-picture functionality in appearance or layout, but differs in several ways, most noticeably in that in this case the displayed content is comprised of semantic requests and results as opposed to television channels are being "watched." Of course, the underlying technology generating the content is also quite different. The WW can be displayed in any of the aforementioned views. When the WW is in Deep View however, the WW's view controls are displayed. The following controls are currently defined:
• Pinning Requests - this allows the user to pin a particular request in the watch group. The WW will keep displaying the new results for only the pinned requests (in a cycle) and will not advance to other requests in the watch group for as long as the current request remains pinned.
° Swapping Requests - this allows the user to swap the currently displayed request with the main request being shown in the semantic browser. The smart style will invoke a method on the OCX to create a temporary request with the swapped request (hashed by its SQML buffer) and then navigate to that request while also informing the Presenter to now display the main request in its place (in the WW).
Stop, Play, Seek, FF, RW, Speedup - these allow the user to stop, play, seek, fast- forward, rewind or speedup the "watch group request stream." For instance, a fast-forward will advance to several requests ahead of the currently displayed one.
• Results controls - this allows the user to control the results in each request in the watch group. Essentially, the results are a stream within a stream and this will also allow the user to control the results in the current request in the current watch group.
• Auto-Display Mode - this will automatically hide the WW when there are no results to display and fade it in when there are new results. This way, the user can maximize the utility of his/her real estate on the screen knowing that watch windows will fade in when there are new semantic results. This feature also allows the user to manage his/her attention during information interaction in a personal and semantic way.
Docking, Closing, Minimizing, Maximizing - these features, as the names imply, allow the user to dock, close, minimize or maximize watch windows. Figure 20 illustrates a Watch Window displaying Filtered Smart Requests (e.g.,. Headlines on Wireless). Figure 20 is an Illustration of the Watch Window with a Current Smart Request Title (e.g., "Breaking News").
7. Watch List Addendum
In the User Interface, the Watch List can be named "News Watch." The user will be asked to add/remove requests, objects, keywords, text, entities, etc. to/from the "News Watch." The "News Watch" can be viewed with a Newsstand watch pane. This will provide a spatially- oriented view of the user's requests and dynamically-created requests (via objects added to the Watch List, and created dynamically by the runtime using those objects as filters) - not unlike the view of a news-magazine rack when one walks into a Library or Bookstore. G. ENTITIES SPECIFICATION FOR THE INFORMATION NERVOUS SYSTEM
1. Introduction
Entities are a very powerful feature of the preferred embodiment of the Information Nervous System. Entities allow the user to create a contextual definition that maps to how they work on a regular basis. Examples of entities include:
Figure imgf000083_0001
There are also industry-specific entities. For instance, in pharmaceuticals, entities could include drags, drag interaction issues, patents, FDA clinical trials, etc. Essentially, an entity is a semantic envelope that is a smart contextual object. An entity can be dragged and dropped like any other smart object. However, an entity is represented by SQML and not SRML (i.e., it is a query-object because it has much richer semantics). An entity can be included as a parameter to a smart request. The user creates entities based on his/her tasks. Entities in the preferred embodiment contain at least the following information (in alternate embodiments they could contain more or less information):
1. Name/Description - a friendly descriptive name for the entity.
2. The categories of the entity - based on standard cross-industry taxonomies or vertical/company-specific taxonomies.
3. Contextual resources - these could include keywords, local documents, Internet documents, or smart objects (such as people).
An entity can be opened in the semantic browser, can be used as a pivot for navigation, as a parameter for a smart request (e.g., Headlines on My Project), can be dragged and dropped, can be copied and pasted, can be used with the smart lens, can be visualized with a smart style, can be used as the basis for an intrinsic alert, can be saved as a .ENT document, can be emailed, shared, etc. In other words, an entity is a first-class smart object.
The semantic runtime client dynamically creates SQML by appending the rich metadata of the entity to the subject of the relational request to create a new rich SQML that refers to the entity.
Entities preferably also have other powerful characteristics:
1. Regarding topics, entities allow the user to create his/her private taxonomy (without being at the mercy of or restricted exclusively to a public taxonomy that is strictly defined and as such, might not map exactly to the user's specific context for a request). The problem with taxonomies is that no taxonomy can ever fit everybody's needs - even in the same organization. Context is very personal and entities allow the user to create a personal taxonomy. For instance, take the example of a dog (of the boxer breed) named Kashmir owned by a dog- owner Steve. To everyone else (but Steve), Kashmir can be expressed (taxonomically) as:
Living Things Animals
Mammals
Dogs
Boxers
Kashmir But to Steve, Kashmir is also:
My Loved Ones My Pets
Kashmir
To Steve's veterinary doctor, however, Kashmir is:
My Clients
My Dogs
My Dogs in Good Health Kashmir
If taxonomies (standalone) were used to "define" Kashmir, none of the three taxonomies would satisfy the general public, Steve, and Steve's veterinary doctor. With entities on the other hand, Steve could create a "Kashmir" entity based on "what Kashmir means to him." Everyone else could then do the same. And so can Steve's veterinary doctor. Entities therefore empower the user with the ability to create private topics that might be extensions of broad taxonomies.
To take another example, a Pharmaceuticals researcher in a large Pharmaceutical company might be working on a new top-secret project (named "Gene Project") on Genomics. Because "Gene Project" is an internal project, it would likely not exist in a public taxonomy which could be used with the semantic browser of this the preferred embodiment of my invention. However, the researcher could create an entity named "Gene Project", typed as a Project, and could then initialize the entity by scoping it to Genomics (which exists in broad taxonomies) and then also qualifying it with the keyword-phrase "Gene Project" (using the AND operator). Essentially, this is akin to defining "Gene Project" as anything on Genomics that has the phrase "Gene Project." This will impose much stricter context than merely using the keywords "Gene Project" (which might return results that contain the word "Project" but have nothing to do with Genomics). By defining a personal topic, "Gene Project" that is scoped to Genomics but also extends "Gene Project" with a specific qualifier, the researcher now has much more precise and personal context. The entity can then be dragged and dropped, copied and pasted, etc. to create requests (e.g., "Experts on Gene Project." At runtime, the server-side semantic query processor will interpret this (by mapping the SQML to the semantic network) as "Experts on any information that belongs to the category Genomics AND which also includes the phrase "Gene Project."
2. Entities also allow the user to create a dynamic taxonomy - public taxonomies are very static and are not updated regularly. With entities, the user can "extend" his/her private taxonomy dynamically and at the speed of thought. Knowledge is transferred at the speed of thought. Entities allow the user to create context with the same speed and dynamism as his/her mind or thought flow. This is very significant. For instance, the user can create an entity for a newly scheduled meeting, a just-discovered conference, a new customer, a newly discovered competitor, etc. - ALL AT THE SPEED OF THOUGHT. Taxonomies don't allow this.
3. Taxonomies assume that topics are the only source of context. With entities, a user can create abstract contextual definitions that include - but are not limited to - topics. Examples include people, teams, events, companies, etc. Entities might eventually "evolve" into topics in a taxonomy (over time and as those entities gain "fame" or "notoriety") but in the "short-term," entities allow the user to create context that has not yet evolved (or might never evolve) into a full-blown taxonomic entry. For instance, Nervana (our company) was initially an entity (known only to itself and its few employees) but as we have grown and attracted public attention, as an entity we are evolving into a topic in a public taxonomy. With entities, users don't have to wait for context (like Nervana) to "eventually become" topics.
4. Entities allow the user to create what the inventor calls "compound context." An example of this is a meeting. A meeting typically involves several participants with documents, presentation slides, and/or handouts relevant to the topic of discussion. With entities in the Information Nervous System, a user can create a "meeting" context that captures the semantics of the meeting. Using the Create Entity Wizard, the user can specify that the entity is a meeting, and then specify the semantic filters. Consider an example of a project meeting with five participants and 2 handed out documents, and one presentation slide. The Presenter of the meeting might want to create an entity in order to track knowledge specifically relevant to the meeting. For instance, he/she might want to do this to determine when to schedule a follow-up meeting or to track specific action items relating to the meeting. To create the entity, the user would add the email addresses of the participants, the handed out documents, and also the presentation to the entity filter definition. The user then saves the entity which is then created in the semantic namespace/environment. The user can then edit the entity with new or removed filters (and/or a new name/description) at a later date/time - for instance, if he/she has discovered new documents that would have been relevant to the meeting. When the user drags and drops the entity or includes it in a request/agent, the semantic browser then compiles the entity and includes it in a master SQML with the sub-queries also passed to the XML Web Service for interpretation. The server-side semantic query processor then processes the compound SQML by constructing a series of SQL sub-queries (or an equivalent) and by joining these queries with the entity sub-queries which in turn are generated using SQL sub-queries.
The user can use an AND or OR (or other) operator to indicate how the entity filters should be applied. For instance, the user can indicate that the meeting (semantically) is the participants of the meeting AND the documents/slides handed out during the meeting. When the entity is compiled at the client and the server, the SQML equivalent is used to interpret the entity (with the desired operator). This is very powerful. It means that the user can define an entity named "Project Meeting" and drag and drop that entity to the special agent named "Breaking News." This then creates a request named "Breaking News on Project Meeting" (with the appropriate SQML referring to the identifier of the entity- which will then be compiled into sub- SQML before it is passed to the server(s) for interpretation. The server then applies default predicates to the entries in the entity (based on what "makes sense" for the object). In this particular example, because of the definition of the entity, the server will then only return:
Breaking News BY ALL the participants AND which is ALSO semantically relevant TO ALL the documents/slides
For instance, this will only return conversations/threads that involve all the participants of the meeting and which are semantically relevant to all the handouts given out during the meeting. This is precisely what the user desired (in this case) and the semantic browser would have empowered the user to essentially construct a rather complex query.
Even more complex queries are possible. Entities can include other entities to allow for compound entities. For instance, if an entire team of people were involved in the meeting, the Presenter might want to create an entity that includes an email distribution list of those people, hi this case, the user might search the Information Nervous System for the distribution list and then save the result as an entity. The browser will allow the user to save results as entities and based on the result type, it will automatically create an entity with a default entity type that "makes sense." For instance, if the user saves a document result as an entity, the semantic browser it will create a "Topic" entity. If the user saves a Person result as an entity, the semantic browser will create a "Person" entity. If the user saves an email distribution list as an entity, the semantic browser will create a "Team" entity.
In this example, the user can save a Person result as a Person entity and then drag and drop that entity into the Project Meeting entity. The Team entity that maps to the email distribution list of the meeting participants can be dragged and dropped to the Project Meeting entity. The user can then create a request called "Headlines on Project Meeting" that includes the entity. The semantic query processor will then return Headlines BY anyone in the email distribution list (using the right default predicate) and which is semantically relevant to ALL the handouts given out during the meeting. Similarly, a Dossier (Guide) on the Project Meeting will return All Bets on the meeting, Best Bets on meeting, Experts on the meeting, etc.
Note that such a compound entity that includes other entities gets checked by the client- side semantic consistency checker for referential integrity, hi other words, if Entity A refers to Entity B and the user attempts to delete Entity B, the semantic browser will detect this and flag the user that Entity B has an outstanding reference. If the user deletes Entity B anyway, the reference in Entity A (and any other references to Entity B) will get removed. Alternately, in some embodiments, the user could be prohibited (whether informed or not) from deleting Entity B in the same situation, based on permissions of others within an organization associated with the entity. For example, employers could monitor activities of employees for risk management purposes, like as is done with email in some companies, only much potentially much more powerfully (Of course, appropriate policies and privacy considerations would have to be addressed). The same process applies to Request Collections (Blenders), Portfolios (Entity Collections - see below), and other compound items in the semantic namespace/environment (items that could refer to other items in the namespace/environment).
5. Popular entities can also be shared amongst members of a knowledge community. Like other items in the semantic browser (like requests or knowledge communities (agencies), entities can be saved as files (so the user can later open them or email them to colleagues, or save them on a central file share, etc.). A common scenario would be that the corporate Librarians at businesses would create entities that map to internal projects, meetings, seminars, tasks, and other important corporate knowledge items of interest. These entities would then be saved on a file-share or other sharing mechanism (like a portal or web-site) or on a knowledge community (agency). The knowledge workers in the organization would then be able to use the entities. As the entities get updated, in the preferred embodiment the Librarians can and will automatically edit their context and users will be able refresh or synchronize to the new entities. Entities could also and alternately be shared on a peer-to-peer basis by individual users. This is akin to a legal peer-to-peer file sharing for music, but instead of music, what is shared is context to facilitate meaning, or more meaningful communication.
2. Portfolios (or Entity Collections)
Portfolios are a special type of entity that contains a collection of entities. In the preferred embodiment, to minimize complexity and confusion (at least of nomenclature or terminology), while an entity can be of any size or composition, and portfolio can contain any kind or number of entities, a portfolio would not contain other portfolios. A portfolio allows the user to manage a group of entities as one unit. A portfolio is a first-class entity and as such has all the aforementioned features of an entity. When a portfolio is used as a parameter in a smart request, the OR qualifier is applied (by default) to its containing entities, hi other words, if Portfolio P contains entities El and E2, a smart request titled 'Headlines on P' will be processed as 'Headlines on El or E2.' The user can change this setting on individual smart requests (to an AND qualifier).
3. Sample Scenarios
Again, in reviewing the scenarios below, it is helpful to recall that, conceptually, the system can gather more relevant information in part because it "knows" who is asking for it, and "understands" who that person or group is, and the kinds of information they are probably interested in. Of course, strictly speaking, the system is not cognitive or self aware in the full human sensed and the operative verbs in the preceding sentence are conceptual metaphors or similes. Still, in operation and results, it mimics understanding and knowledge to an unprecedented degree in part because of its underlying semantically-informed architecture and execution.
This point can be illustrated by a simplistic contrast: If two very different people entered the exact same search at the exact same time into a search engine such as Google, they would get the exact same results. In contrast, with the preferred embodiment of the present system, if those same two people entered the same request via an Entity, each would get different results tailored to be relevant to each.
To appreciate some of the potential power of this feature, it is useful to note that while the system or Entities "know" who is posing the query, the Entities do not depend for that knowledge on the user informing them and keeping them constantly updated and informed (although user information can be supplied and considered at any time). If that were the case, the system could be too labor intensive to be efficient and useful in many situations; it would just be too much work. Instead, the Entities "know" who the requester is by inference and from semantics from characteristics sometimes supplied by others, sometimes derived or deduced, sometimes collected from other requests and the like, as explained throughout this application and its parent application.
Some example scenarios of Entities in operation:
1. A pharmaceuticals 'patent' entity could include the categories of the patent, relevant keywords, and relevant documents.
2. A CIA agent could create a 'terrorist' entity to track terrorists. This could include categories on terrorism, suspicious wire transfers, suspicious arms sales, classified documents, keywords, and terrorism experts in the information community.
3. Find All Breaking News on Yesterday' s Meeting.
4. Find Headlines on any of my competitors (this is done by creating the competitor entities, and then creating a smart request with the entities as parameters using the OR qualifier with each predicate).
5. Find Experts on my investment portfolio companies (create the individual entities, create a portfolio containing these entities and then create a smart request that has the 'Experts' context template and that uses the portfolio as an argument).
6. Open a Dossier (Guide) on my competitors (create the individual competitor entities, create a portfolio containing these entities and then create a smart request that has the 'Dossier' (or 'Guide') context template and that uses the portfolio as an argument). Figure 21 shows Entity views displayed in the semantic browser (on the left).
H. KNOWLEDGE COMMUNITY BROWSING AND SUBSCRIPTION SPECIFICATION FOR THE INFORMATION NERVOUS SYSTEM
Overview
The Nervana semantic browser will allow the user to subscribe and unsubscribe to/from knowledge communities (agencies) for a given profile. These knowledge communities will be readily available to the user underneath the profile entry in the semantic environment. In addition, these lαiowledge communities will be queried by default for intrinsic alerts, context panels, and etc. whenever results are displayed for any request created using the same profile.
The semantic environment includes state indicating the subscribed knowledge communities for each profile. The client-side semantic query processor (SQP) uses this information for dynamic requests that start from results for requests of a given profile (the SQP will ask the semantic runtime client for the knowledge communities for the profile and then issue XML Web Service calls to those knowledge communities as appropriate). Figures 22A and 22B show the user interface for the knowledge community subscription and un-subscription. The dialog box has combo boxes allowing the user to filter by profile, to view all, new, subscribed, suggested, and un-subscribed communities, by industry and area of interest, by keywords, by publishing point (all publishing points, the local area network, the enterprise directory, and the global knowledge community directory), and by creation time (anytime, today, yesterday, this week, and last week). The semantic runtime client queries the publishing point endpoint listeners (for each publishing point) using the filters. It then gathers the results and displays them in the results pane. The user is also able to view the categories of each knowledge community in the results pane via a combo box. Figure 20B illustrates the bottom portion of the Knowledge Communities Dialog Box.
I. CLIENT-SIDE SEMANTIC QUERY DOCUMENT SPECIFICATION FOR THE
INFORMATION NERVOUS SYSTEM
1. Semantic Query Markup Language (SQML) Overview
In the currently preferred embodiment, the Nervana Semantic DHTML Behavior is an Internet Explorer DHTML Behavior that, from the client's perspective, every thing it understands as a query document. The client opens 'query documents,' in a manner resembling how a word processor opens 'textual and compound documents.5 The Nervana client is primarily responsible for processing a Nervana semantic query document and rendering the results. A Nervana semantic query document is expressed and stored in form of the Nervana Semantic Query Markup Language (SQML). This is akin to a "semantic file format."
In the preferred embodiment, the SQML semantic file format comprises of the following:
• Head - The 'head' tag, like in the case of HTML, includes tags that describe the document.
• Title - The title of the document.
• Comments - The comments of the document.
• UserName - The username of the document creator.
• SystemName - The systemname of the device on which the document was created.
• Subj ect - The subj ect of the document.
• Creator - The creator of the document. Company - The company in which the document was created.
RequestType - This indicates the type of request. It can be "smart request"
(indicating requests to one or more information community web services) or
"dumb request" (indicating requests to one or more local or network resources).
ObjectType - This fully qualifies the type of objects returned by the query.
URI - The location of the document.
CreationTime - The creation time of the document.
LastModifiedTime - The last modified time of the document.
LastAccessedTime - The last accessed time of the document.
Attributes - The attributes of the document, if any.
RevisionNumber - The revision number of the document.
Language - The language of the document.
Version - this indicates the version of the query. This allows the web service's semantic query processor to return results that are versioned. For instance, one version of the browser can use VI of a query, and another version can use V2.
This allows the web service to provide backwards compatibility both at the resource level (e.g., for agents) and at the link level.
Targets - This indicates the names and the URLs of the information community web services that the query document targets.
Type - this indicates the type of targets. This can be "targetentries," in which case the tag includes sub-tags indicating the actual web service targets, or
"allsubscribedtargets," in which case the query processor uses all subscribed information communities.
Categories - This indicates the list of category URLs that the query document refers to. Each "category" entry contains a name attribute and a URI attribute that indicates the URL of the Knowledge Domain Server (KDS) from which the category came.
Type - this indicates the type of categories. This can be either "categoryentri.es," in which case the sub-tag refers to the list of category entries, "allcategories," in which case all categories are requested from the information community web services, or "myfavoritecategories," in which case the query processor gets the user's favorite categories and then generates compiled SQML that contains these categories (this compiled SQML is then sent to the server(s)).
Query - This is the parent tag for all the main query entries of the query document
Resource - The reference to the 'dumb' resource being queried. Examples include file paths, URLs, cache entry identifiers, etc. These will be mapped to actual resource managers components by the interpreter.
Type - The type of resource reference, qualified with the namespace. Examples of defined resource reference types are: nervana:url (this indicates that the resource reference is a well-formed standard Internet URL, or a custom Nervana URL like
'agent://..."), nervana: filepath (this indicates that the resource reference is a path to a file or directory on the file-system), and nervana amespaceref (this indicates that the resource comes from the client semantic namespace). • Uri - This indicates the universal resource identifier of the resource. In the case of paths and Internet URLs, this indicates the URL itself. In the case of namespace entries, this indicates the GUID identifier of the entry.
• Mid - This indicates the metadata identifier, which is used by the SQML interpreter to map the resource to the metadata section of the document. The metadata id is mapped to the same identifier within the metadata section.
• Args - This indicates the arguments of the resource identifier.
• Links - this indicates the reference to the semantic links (for "targets" only)
• Type - this indicates the type of links. This can be "linkentries," indicating the links are explicit entries.
• LinkEntries - this indicates the details of a link entry.
• Predicate - this indicates the type of predicate for the link. For instance, the predicate "nervana:relevantto" indicates that the query is "return all objects from the resource R that are relevant to the object O," where R and O and the specified resource and object, respectively. Other examples of predicates include nervana:reportsto, nervana:teammateof, nervana:from, nervana:to, nervanaxc, nervana:bcc, nervana: attachedto, nervana:sentby, nervana: sentto, nervana:postedon, nervana ontainstext, etc.
• Type - this indicates the type of object reference indicates in the 'Link' tag. Examples include standard XML data types like xmkstring, xmkinteger, Nervana equivalents of same, custom Nervana types like nervana: datetimeref (which could refer to object references like 'today' and 'tomorrow'), and any standard Internet URL (HTTP, FTP, etc.) or Nervana URL (objects://, etc.) that refers to an object that Nervana can process as a semantic XML object.
• Metadata - this contains the references to the metadata entries.
• MetadataEntry - this indicates the details of a metadata entry.
• Mid - this indicates the metadata identifier (GUID). o Value - this indicates the metadata itself.
EXAMPLE: DOCUMENTS (INFORMATION OR CONTEXT-BASED)
<?xml version="1.0" encoding="utf-8"?> <sqml>
<head requesttype="smart request" obj ecttyp e="context\headlines" uri- 'c :\foo ' s\bar.pdf' creationtime- 'foo" lastmodifiedtime- 'foo" lastaccessedtime- 'foo" attributes="0" revisionnumber="0" language- 'foo" version- 'foo" />
<title>foo</title> <comments>foo</comments> <username>foo</username> <systemname>foo</systemname> <subj ect>foo</subj ect> <creator>foo</creator> <company>foo</company> <targets>
<target name- 'Marketing" reftype- 'uri" ref="kisp://marketing/default.wsdι" /> <target name- 'Research" reftype="uri" ref="kisp://research/default.wsdl" /> </targets> <categories>
<category name- 'reuters\pharmaceuticals\biotechnology" reftype- 'uri" ref="kdsp://reuters.com/categories.wsdl?id=45" /> <category name- 'reuters\pharmaceuticals\life_sciences" reftype- 'uri" ref— 'kdsp://reuters.com/categories.wsdl?id=57" /> </categories> /> <resources>
<resource name="foo" type- 'information\documents\general document" reftype- 'nervana: filepath" ref="file://c:\bar.doc" mid="7886e4a0-55d9-45ac-a084-97adc6fffd0f args="" /> <resource name="foo" type="information\all information" reftype="nervana:url" ref="file://c:\bar.doc" mid-"01fc64a3-c068-4339-bc97-17e5ff37e93f args="" /> <resource name- 'foo" type="information\all information" reftype- 'nervana:folderpath" ref="file://c:\" mid="f8cc39c3-e4f0-4a29-be2a-d2faf36eb3a0" args- 'includesubfolders=true" /> <resource name="foo" type- 'information\documents\general document" reftype- 'nervana:url" ref="http://www .bar.com/doc.htm" mid="f8cc39c3-e4f0-4a29-be2a-d2faf36eb3a0" args="" /> <resource name- 'foo" type="information\documents\general document" reftype="nervana:uri" ref="ftp ://gate.com/doc.txt" mid="f8cc39c3-e4f0-4a29-be2a-d2faf36eb3a0" args="" /> <resource name="foo" type— einformation\documents\generaI document" reftype- 'nervana: filepath" ref="file:/Λ\servers\server\file.pdf' mid="lb870a25-4e98-45d8-a444-fl)283a495357" args="" /> <resource name-'foo" type- 'information\documents\text document" reftype- 'nervana:text" ref^"" mid="7886e4a0-55d9-45ac-a084-97adc6fffd0f args="" /> <resource name- 'foo" type="information\documents\general document" " reftype- 'nervana: cacheentry" ref="ef9c90ea-282d-46d6-b355-ac8a4fc2f3e5" mid="" args="" /> <resource name- 'foo" type- 'information\email\email message" reftype- 'nervana:url' ' ref="request://email.all@ibm.com" mid= args= /> <resource name- 'foo" type- 'information\emair\email annotation" reftype- 'nervana:url" ref=' 'obj ects ://rad. com/agency, asp" mid="" args="" /> <resource name- 'foo" type="information\documents\general document" reftype:="nervana:url" ref="objects://rad.com/agency.asp" mid= _<«« args= _"<«£ /> <resource name- 'foo" type="information\documents\general document" reftype- 'nervana:url" ref="obj ects ://rad. com/agency, asp' ' mid-"" args="" /> <resource name- 'foo" type="information\documents\general document" reftype- 'nervana:uri" ref="request://documents. all@intel.com" mid= args= /> </resources> <links> <link operator="and" predicate- 'nervana:relatedto" name- 'foo" type="information\documents\general document" reftype- 'nervana: filepath" ref^"file://c:\foo.doc" mid="7886e4a0-55d9-45ac-a084-97adc6fffd0f args-"" /> <link operator-'and" predicate="nervana: contains" name- 'foo" type="information\documents\general document" reftype="nervana:text" ref="" mid="46ea76cb-1383-4885-af6f-0e0fc6a66896" args="" /> <link operator- 'and" predicate- 'nervana:postedon" name- 'foo" type="types\datetime" reftype="nervana:datetimeref ref="" mid="3fa64c3c-4754-4380-91b5-521299036c62" args— '" /> <link operator- 'and" predicate="nervana:relatedto" name- 'foo" type- 'information\documents\general document" reftype- 'nervana^rl" ref- 'kisp://98@in.com/m.asp" mid="c2649c39-alc3-4ca8-ae8d-c85c04372e9a" args="" /> <link operator-'and" predicate- 'nervanaύsofpriority" name- 'foo" type="types\priority" reftype="nervana:priority" ref="" mid="69bbc048-98c8-4f76-8edf-5a00ce91cl83" args="" /> </links> <metadata>
<metadataentry mid="7886e4a0-55d9-45ac-a084-97adc6fffd0f reftype- 'uri" ref="file://c:\foo\bar.pdf />
<value>
<document>
<title>scenario modelling</title>
<type>text</type>
<format>application/pdf</format>
<filepath>c:\foo\bar.pdf</filepath>
<shortfilename>bar.pdf</shortfilename>
<creationtime>foo</creationtime>
<lastmodifiedtime>foo</lastmodifiedtime>
<lastaccessedtime>foo</lastaccessedtime>
<attributes>0</attributes>
<size>0</size>
<subj ect>foo</subj ect>
<creator>foo</creator>
<manager>foo</manager>
<company>foo</company>
<category>foo</category>
<keywords>foo</keywords>
<comments>foo</comments>
<hlinkbase>foo</hlinkbase>
<template>foo</template>
<lastsavedby>foo</lastsavedby>
<revisionnumber>0</revisionnumber>
<totaleditingtime>foo</totaleditingtime>
<numpages>0</numpages>
<numparagraphs>0</numparagraphs>
<numlines>0</numlines>
<numwords>0</numwords> l
<numcharacters>0</numcharacters>
<numcharacterswithspaces>0</numcharacterswithspaces>
<numbytes>0</numbytes> <language>foo</language> <version>foo</version> <abstract>foo</abstract> </document> </value> /> <metadataentry mid="bfcbl2b4-70bb-473a-847c-ebffel87828f reftype- 'uri" ref^"file://c:\foo\bar.pdf />
<value>
<email>
<title>scenario modelling</title>
<type>text</type>
<format>application/pdf</format>
<filepath>c:\foo\bar.pdf</filepath>
<shortfιlename>bar.pdf</shortfilename>
<creationtime>foo</creationtime>
<lastmodifiedtime>foo</lastmodifiedtime>
<lastaccessedtime>foo</lastaccessedtime>
<attributes>0</attributes>
<size>0</size>
<subj ect>foo</subj ect>
<creator>foo</creator>
<manager>foo</manager>
<comρany>foo</company>
<category>foo</category>
<keywords>foo</keywords>
<comments>foo</comments>
<hlinkbase>foo</hlinkbase>
<temρlate>foo</template>
<lastsavedby>foo</lastsavedby>
<revisionnumber>0</revisionnumber>
<totaleditingtime>foo</totaleditingtime>
<numpages>0</numpages>
<numparagraphs>0</numparagraphs>
<numlines>0</numlines>
<numwords>0</numwords>
<numcharacters>0</numcharacters>
<numcharacterswithspaces>0</numcharacterswithspaces>
<numbytes>0</numbytes> <language>foo</language> <version>foo</version> <abstract>foo</abstract> </email> </value> /> </metadata> </sqml> 2. SQML Generation
Preferably, SQML is generated in any one or more of several possible ways:
By creating a smart request By creating a local request By creating an entity
By opening one or more local documents in the semantic browser By the client (dynamically) - in response to a drag and drop, smart copy and paste, intrinsic alert, context panel/link invocation, etc.
3. SQML Parsing hi some embodiments in some situations, SQML that gets created on the client might not be ready (in real-time) for remote consumption - by the server's XML web service or at another machine site. This is especially likely to be the case when the SQML refers to local context such as documents, Entities, or Smart Requests (that are identified by unique identifiers in the semantic environment).1 In the preferred embodiment, the client generally creates SQML that is ready for remote consumption. Preferably, it does this by caching the metadata for all references in the metadata section of the document. This is preferable because in some cases, the resource or object to which the reference points might no longer exist when the query is invoked. For instance, a user might drag and drop a document from the Internet to a smart request in order to generate a new relational request. The client extracts the metadata (including the summary) from the link and inserts the metadata into the SQML. Because the resolution of the query uses only the metadata, the query is ready for consumption once the metadata is inserted into the SQML document. However, the link that the object refers to might not exist the day after the user found it. In such a case, even if the user invokes the relational request after the link might have ceased to exist, the request will still work because the metadata would already have been cached in the SQML.
The client SQML parser performs "lazy" updating of metadata in the SQML. When the request is invoked, it attempts to update the metadata of all parameters (resources, etc.) in the SQML to handle the case where the objects might have changed since they were used to create
Blenders (or collections) contain references to smart requests. the relational request. If the object does not exist, the client uses the metadata it already has. Otherwise, it updates it and uses the updated metadata. That way, even if the object has been deleted, the user experience is not interrupted until the user actually tries to open the object from whence the metadata came.
J. SEMANTIC CLIENT-SIDE RUNTIME CONTROL API SPECIFICATION FOR
THE INFORMATION NERVOUS SYSTEM
1. Introducing the Nervana Semantic Runtime Control - Overview
In the preferred embodiment, the Nervana Semantic Runtime Control is an ActiveX control that exposes properties and methods for use in displaying semantic data using the Nervana semantic user experience. The control will be primarily called from XSLT skins that take XML data (using the SRML schema) and generate DHTML+TLME or SVG output, consistent with the requirements of the Nervana semantic user experience. Essentially, in this embodiment, the Nervana control encapsulates the "SDK" on top of which the XSLT skins sit in order to produce a semantic content-driven user experience. The APIs listed below illustrate the functionality that will be exposed or made available by the final API set in the preferred embodiment.
2. The Nervana Semantic Runtime Control API a. EnumObjectsInNamespacePath
INTRODUCTION
The EnumObjectsInNamespacePath method returns the objects in a namespace path.
USAGE SCENARIO
A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to open a namespace path in order for the user to navigate the namespace from within the semantic browser.
PROTOTYPE
SCODE
EnumObj ectsInNamespacePath(
[in] BSTR Path, [in] LONG QueryMask,
[out] BSTR *ρQueryRequestGuid ); b. CompileSemanticQueryFromBuffer
INTRODUCTION
The CompileSemanticQueryFromBuffer method opens an SQML buffer and compiles it into one or more execution-ready SQML buffers. For instance, an SQML file containing a blender will be compiled into SQML buffers representing each blender entry. If the blender contains blenders, the blenders will be unwrapped and an SQML buffer will be returned for each contained blender. A compiled or "execution-ready" SQML buffer is one that can be semantically processed by an agency. The implication is that a blender that has agents from multiple agencies will have its SQML compiled to buffers with the appropriate SQML from each agency.
Note: If the buffer is already compiled, the method returns S_FALSE and the return arguments are ignored.
USAGE SCENARIO
A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to compile an SQML buffer and retrieve generated "compiled code" that is ready for execution. In typical scenarios, the application or skin will compile an SQML buffer and then prepare frame windows where it wants each individual SQML query to sit. It can then issue individual SQML semantic calls by calling OpenSemanticQueryFromBuffer and then have the results displayed in the individual frames.
PROTOTYPE
SCODE
Compiles emanticQueryFromBuffer(
[in] BSTR SQMLBuffer,
[in] DWORD Flags,
[out] DWORD *pdwNumCompiledBuffers,
[out] BSTR *pbstrCompiledBuffers ); c. OpenSemanticQueryFromBuffer
INTRODUCTION
The OpenSemanticQueryFromBuffer method opens an SQML buffer and asynchronously fires the XML results (in SRML) onto the DOM, from whence a Nervana skin can sink the event. Note that in this embodiment the SQML has to be "compiled" and ready for execution. If the SQML is not ready for execution, the call will fail. To compile an SQML buffer, call CompileSemanticQueryFromBuffer.
USAGE SCENARIO
A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to open a compiled SQML buffer.
PROTOTYPE SCODE
OpenSemanticQueryFromBuffer( [in] BSTR SQMLBuffer, [in] DWORD Flags, [out] GUID *pQueryID ); d. GetSemanticQueryBufferFromFile
INTRODUCTION
The GetSemanticQueryBufferFromFile method opens an SQML file, and returns the buffer contents. The buffer can then be compiled and/or opened.
USAGE SCENARIO
A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to convert an SQML file into a buffer before processing it.
PROTOTYPE
SCODE
GetSemanticQueryBufferFromFile (
[in] BSTR SQMLFilePath,
[in] DWORD FileOpenFlags,
[out] BSTR *pbstrSQMLBuffer ); e. GetSemanticQueryBufferFromNamespace
INTRODUCTION
The GetSemanticQueryBufferFromNamespace method opens a namespace object, and retrieves its SQML buffer.
USAGE SCENARIO
A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to open an SQML buffer when it already has access to the id and path of the namespace object.
PROTOTYPE SCODE
GetSemanticQueryBufferFromNamespace(
[in] GUID ObjectLD,
[in] BSTR Path,
[out] BSTR *ρbstrSQMLBuffer ); f. GetSemanticQueryBufferFromURL
INTRODUCTION
The GetSemanticQueryBufferFromURL method wraps the URL in an SQML buffer, and returns the buffer.
USAGE SCENARIO A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to convert an URL of any type to SQML. This can include file paths, HTTP URLs, FTP URLs, Nervana agency object URLs (prefixed by "wsobject://") or Nervana agency URLs (prefixed by "wsagency ://").
PROTOTYPE
SCODE
GetS emanticQueryBufferFromURL(
[in] BSTR URL,
[out] BSTR *pBuffer ); g. GetSemanticQueryBufferFromClipboard
INTRODUCTION
The GetSemanticQueryBufferFromClipboard method converts the clipboard contents to SQML, and returns the buffer.
USAGE SCENARIO
A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to get a semantic query from the clipboard. The application can then load the query buffer.
PROTOTYPE
SCODE GetSemanticQueryBufferFromCliρboard( [out] BSTR *pBuffer ); h. Stop
INTRODUCTION
The Stop method stops current open request.
USAGE SCENARIO A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to stop a load request is just issued.
PROTOTYPE
SCODE Stop( [in] GUID QueryLD ); i. Refresh
INTRODUCTION
The Refresh method refreshes the current open request.
USAGE SCENARIO
A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to refresh the currently loaded request.
PROTOTYPE
SCODE Refresh( [in] GULD QueryLD ); j. CreateNamespaceObject
INTRODUCTION
The CreateNamespaceObject method creates a namespace object and returns its GULD.
USAGE SCENARIO
A Nervana client application (for instance, the semantic browser) or a Nervana skin will typically call this method to create a temporary namespace object when a new query document has been opened.
PROTOTYPE SCODE
CreateNamespaceObj ect(
[in] BSTR Name,
[in] BSTR Description,
[in] BSTR QueryBuffer,
[in] LONG AgentObjectType,
[in] LONG Attributes,
[in] LONGNamespaceObjectType,
[out] GUID *pObjectID ); k. DeleteNamespaceObject
INTRODUCTION
The DeleteNamespaceObject method deletes a namespace object.
USAGE SCENARIO
A Nervana client application (for instance, the semantic browser) or a Nervana skin will typically call this method to delete a temporary namespace object.
PROTOTYPE
SCODE DeleteNamespaceObject [in] GUID ObjectLD );
1. CopyObject
INTRODUCTION
The CopyObject method copies the semantic object to the clipboard as an SQML buffer using a proprietary SQML clipboard format. The object can then be "pasted" onto agents for relational semantic queries, or used as a lens over other objects or agents. USAGE SCENARIO
A Nervana skin will typically call the CopyObject method when the user clicks on the
"Copy" menu option - off a popup menu on the object.
PROTOTYPE
SCODE CopyObject( [in] BSTR ObjectSRML ); m. CanObjectBeAnnotated
INTRODUCTION
The CanObjectBeAnnotated method checks whether the given object can be annotated.
USAGE SCENARIO
A Nervana skin will typically call the CanObjectBeAnnotated method to determine whether to show UI indicating the "Annotate" command.
PROTOTYPE
SCODE CanObjectBeAnnotated( [in] BSTR bstrObjectSRML ); n. AnnotateObject
INTRODUCTION
The AnnotateObject method invokes the currently installed email client and initializes it to send an email annotation of the object to the email agent of the agency from whence the object came.
USAGE SCENARIO
A Nervana skin will typically call the AnnotateObject method when the user clicks on the "Annotate" menu option - off a popup menu on the object.
PROTOTYPE
SCODE AnnotateObject( [in] BSTR bstrObjectSRML ); o. CanObjectBePublished
INTRODUCTION
The CanObjectBePublished method checks whether the given object can be published. USAGE SCENARIO
A Nervana skin will typically call the CanObjectBePublished method to determine whether to show UI indicating the "Publish" command.
PROTOTYPE
SCODE CanObjectBePublished ( [in] BSTR bstrObjectSRML ); p. PublishObject
INTRODUCTION
The PublishObject method invokes the currently installed email client and initializes it to send an email publication of the object to the email agent of the agency from whence the object came.
USAGE SCENARIO
A Nervana skin will typically call the PublishObject method when the user clicks on the "Publish" menu option - off a popup menu on the object.
PROTOTYPE
SCODE AnnotateObject( [in] BSTR bstrObjectSRML ); q. OpenObjectContents
INTRODUCTION The OpenObjectContents method opens the object using an appropriate viewer. For instance, an email object will be opened in the email client, a document will be opened in the browser, etc..
USAGE SCENARIO
A Nervana skin will typically call the OpenObjectContents method when the user clicks on the "Open" menu option - off a popup menu on the object.
PROTOTYPE
SCODE OpenObjectContents ( [in] BSTR ObjectSRML ); r. SendEmailToPersonObject
INTRODUCTION
The SendEmailToObject method is called to send email to a person or customer object. The method opens the email client and initializes it with the email address of the person or customer object.
USAGE SCENARIO
A Nervana skin will typically call the SendEmailToObject method when the user clicks on the "Send Email" menu option - off a popup menu on a person or customer object.
PROTOTYPE
SCODE SendEmailToObject( [in] BSTR ObjectSRML ); s. GetObj ect Annotations
INTRODUCTION
The GetObj ect Annotations method is called to get the annotations an object has on the agency from whence it came.
USAGE SCENARIO
A Nervana skin will typically call the GetObj ectAnnotations method when it wants to display the titles of the annotations an object has - for instance, in a popup menu or when it wants to display the annotations metadata in a window.
PROTOTYPE
SCODE
GetObj ect Annotations(
[in] BSTR ObjectSRML,
[in] LONG QueryMask,
[out] BSTR *pQueryRequestGuid ); t. IsObjectMarkedAsFavorite
INTRODUCTION
The IsObjectMarkedAsFavorite method is called to check whether an object is marked as a favorite on the agency from whence it came. USAGE SCENARIO
A Nervana skin will typically call the IsObjectMarkedAsFavorite method to determine what UI to show - either the "Mark as Favorite" or the "Unmark as Favorite" command. If the object cannot be marked as a favorite (for instance, if it did not originate on an agency), the error code E_LNVALIDARG is returned.
PROTOTYPE
SCODE
IsObjectMarkedAsFavorite( in] BSTR ObjectSRML ); u. MarkObj ect AsFavo rite
INTRODUCTION
The MarkObj ect AsFavorite method is called to mark the object as a favorite on the agency from whence it came.
USAGE SCENARIO
A Nervana skin will typically call the MarkObj ect AsFavorite method when the user clicks on the "Mark as Favorite" command.
PROTOTYPE SCODE
MarkAsFavorite( in] BSTR ObjectSRML ); v. UnmarkObjectAsFavorite
INTRODUCTION The UnmarkObjectAsFavorite method is called to unmark the object as a favorite on the agency from whence it came.
USAGE SCENARIO A Nervana skin will typically call the UnmarkObjectAsFavorite method when the user clicks on the "Unmark as Favorite" command. PROTOTYPE
SCODE
UnmarkAsFavorite( in] BSTR ObjectSRML ); w. IsSmartAgentOnClipboard
INTRODUCTION
The IsSmartAgentOnClipboard method is called to check whether a smart agent has been copied to the clipboard.
USAGE SCENARIO
A Nervana skin will typically call the IsSmartAgentOnClipboard method when it wants to toggle the user interface to display the "Paste" icon or when the "Paste" command is invoked.
PROTOTYPE
SCODE
IsSmartAgentOnClipboard(); x. GetSmartLensQueryBuffer
INTRODUCTION
The GetSmartLensQueryBuffer method is called to get the query buffer of the smart lens.
This returns the SQML of the query that represents the objects on the smart agent that is on the clipboard, and which are semantically relevant to a given object.
USAGE SCENARIO
A Nervana skin will typically call the GetSmartLensQueryBuffer method when the user hits "Paste as Smart Lens" to invoke the smart lens off the smart agent that is on the clipboard.
PROTOTYPE SCODE
GetSmartLensQueryBuffer(
[in] BSTR ObjectSRML,
[in] LONG QueryMask,
[out] BSTR *pQueryRequestGuid );
ill y. OpenObjectContents
INTRODUCTION
The OpenObjectContents method opens the object using an appropriate viewer. For instance, an email object will be opened in the email client, a document will be opened in the browser, etc.
USAGE SCENARIO
A Nervana skin will typically call the OpenObjectContents method when the user clicks on the "Open" menu option - off a popup menu on the object.
PROTOTYPE
SCODE OpenObjectContents( [in] BSTR ObjectSRML );
Part
3. Email Control APIs a. Email_GetFromLink bjects
INTRODUCTION
The Email_GetFromLinkObjects method is called to get the metadata for the "From" links on an email object from the agency from whence it came.
USAGE SCENARIO
A Nervana skin will typically call the Email_GetFromLinkObjects method when it wants to navigate to the "From" list from an email object, or to display a popup menu with the name of the person in the "From" list.
PROTOTYPE SCODE
Email_GetFromLinkObj ects(
[in] BSTR EmailObjectSRML,
[in] LONG QueryMask,
[out] BSTR *pQueryRequestGuid ); b. Email GetToLinkObjects
INTRODUCTION
The Email_GetFromLinkObjects method is called to get the metadata for the "To" links on an email object from the agency from whence it came.
USAGE SCENARIO
A Nervana skin will typically call the Email_GetToLinkObjects method when it wants to navigate to the "To" list from an email object, or to display a popup menu with the name of the person in the "To" list.
PROTOTYPE
SCODE
Email_GetToLinkObjects( .
[in] BSTR EmailObjectSRML,
[in] LONG QueryMask,
[out] BSTR *pQueryRequestGuid ); c. Email GrβtCcLinkObj ects
INTRODUCTION
The Email_GetCcLinkObjects method is called to get the metadata for the "CC" links on an email object from the agency from whence it came.
USAGE SCENARIO
A Nervana skin will typically call the Email_GetCcLinkObjects method when it wants to navigate to the "CC" list from an email object, or to display a popup menu with the name of the person in the "CC" list.
PROTOTYPE
SCODE
Email_GetCcLinkObj ects(
[in] BSTR EmailObjectSRML,
[in] LONG QueryMask,
[out] BSTR *pQueryRequestGuid ); d. Email GetBccLinkObjects
INTRODUCTION
The Email_GetBccLinkObjects method is called to get the metadata for the "BCC" links on an email object from the agency from whence it came.
USAGE SCENARIO
A Nervana skin will typically call the Email_GetBccLinkObjects method when it wants to navigate to the "BCC" list from an email object, or to display a popup menu with the name of the person in the "BCC" list.
PROTOTYPE
SCODE
Email_GetBccLinkObj ects(
[in] BSTR EmailObjectSRML,
[in] LONG QueryMask,
[out] BSTR *pQueryRequestGuid ); e. EmaiI_Get AttachmentLinkObj ects
INTRODUCTION
The Email_Get AttachmentLinkObj ects method is called to get the metadata for the
"Attachment" links on an email object from the agency from whence it came.
USAGE SCENARIO
A Nervana skin will typically call the Email_Get AttachmentLinkObj ects method when it wants to navigate to the "Attachments" link from an email object, or to display a popup menu with the titles of the attachments in the "Attachments" list.
PROTOTYPE
SCODE
Email_Get AttachmentLinkObj ects(
[in] BSTR EmailObjectSRML,
[in] LONG QueryMask,
[out] BSTR *pQueryRequestGuid ); 4. Person Control APIs a. Person GetDirectReports
INTRODUCTION
The Person_GetDirectReports method is called to get the metadata for the "Direct Reports" links on a person object from the agency from whence it came.
USAGE SCENARIO
A Nervana skin will typically call the Person_GetDirectReports method when it wants to navigate to the "Direct Reports" link from a person object, or to display a popup menu with the names of the direct reports in the "Direct Reports" list.
PROTOTYPE
SCODE
Person_GetDirectReports(
[in] BSTR EmailObjectSRML,
[in] LONG QueryMask,
[out] BSTR *pQueryRequestGuid ); b. Person GetDistributionLists
INTRODUCTION
The Person_GetDistributionLists method is called to get the metadata for the "Member of Distribution Lists" links on a person object from the agency from whence it came.
USAGE SCENARIO
A Nervana skin will typically call the Person_GetDistributionLists method when it wants to navigate to the "Member of Distribution Lists" link from a person object, or to display a popup menu with the names of the distribution lists of which the person is a member.
PROTOTYPE
SCODE
Person_GetDistributionLists(
[in] BSTR PersonObjectSRML,
[in] LONG QueryMask,
[out] BSTR *pQueryRequestGuid ); c. Person GetlnfoAuthored
INTRODUCTION
The Person_GetInfoAuthored method is called to get the metadata for the "Info Authored by Person" links on a person object from the agency from whence it came.
USAGE SCENARIO
A Nervana skin will typically call the Person_GetInfoAuthored method when it wants to navigate to the "Info Authored by Person" link from a person object, or to display a preview window with time-critical or recent information that the person authored.
PROTOTYPE
SCODE
Person_GetInfoAuthored(
[in] BSTR PersonObjectSRML,
[in] BOOL SemanticQuery,
[in] LONG QueryMask,
[out] BSTR *pQueryRequestGuid ); d. Person GetlnfoAnnotated
INTRODUCTION
The Person_GetInfoAnnotated method is called to get the metadata for the "Info Annotated by Person" links on a person object from the agency from whence it came.
USAGE SCENARIO
A Nervana skin will typically call the Person_GetInfoAnnotated method when it wants to navigate to the "hifo Annotated by Person" link from a person object, or to display a preview window with time-critical or recent information that the person annotated.
PROTOTYPE
SCODE
Person_GetInfoAnnotated(
[in] BSTR PersonObjectSRML,
[in] LONG QueryMask,
[out] BSTR *pQueryRequestGuid); e. Person_GetAnnotationsPosted
INTRODUCTION
The Person_GetAnnotationsPosted method is called to get the metadata for the "Annotations Posted by Person" links on a person object from the agency from whence it came.
USAGE SCENARIO
A Nervana skin will typically call the Person_GetAnnotationsPosted method when it wants to navigate to the "Annotations Posted by Person" link from a person object, or to display a preview window with time-critical or recent annotations that the person posted.
PROTOTYPE
SCODE
Person_GetAnnotationsPosted(
[in] BSTR PersonObjectSRML,
[in] LONG QueryMask,
[out] BSTR *pQueryRequestGuid ); f. Person sendEmailTo
INTRODUCTION The Person_SendEmailTo method is called to send email to a person or customer object. The method opens the email client and initializes it with the email address of the person or customer object.
USAGE SCENARIO
A Nervana skin will typically call the Person_SendEmailTo method when the user clicks on the "Send Email" menu option - off a popup menu on a person or customer object.
PROTOTYPE
SCODE Person_SendEmailTo( [in] BSTR ObjectSRML );
5. System Control Events a. Event: OnBeforeQuery
INTRODUCTION
The OnBeforeQuery event is fired before the control issues a query to resources consistent with the current semantic request.
USAGE SCENARIO
A Nervana client application (for instance, the semantic browser) or a Nervana skin will sink this event if it wants to cancel a query or cache state before the query is issued.
PROTOTYPE
VOLD
OnB eforeQuery(
[in] GUID QueryLD,
[in] BSTR QueryBuffer,
[in] DWORD QueryMask,
[in] DWORD Flags,
[out] BOOL *Cancel ); b. Event: OnQueryBegin
INTRODUCTION
The OnQueryBegin event is fired when the control issues the first query to a resource consistent with the current semantic request.
USAGE SCENARIO
A Nervana client application (for instance, the semantic browser) or a Nervana skin will sink this event if it wants to cache state or display status information when the query is in progress.
PROTOTYPE
VOLD
OnQueryBegin( [in] GULD ObjectlD ); c. Event: OnQueryComplete
INTRODUCTION
The OnQueryComplete event is fired before the control issues a query to resources consistent with the current semantic request. USAGE SCENARIO
A Nervana client application (for instance, the semantic browser) or a Nervana skin will sink this event if it wants to cancel a query or cache state before the query is issued.
PROTOTYPE
VOLD
OnQueryComρlete( [in] GULD QueryLD ); d. Event: OnQueryResultsAvailable
INTRODUCTION
The OnQueryResultsAvailable event is fired when there are available results of an asynchronous method call. The event indicates the request GULD, via which the caller can uniquely identify the specific method call that generated the response.
USAGE SCENARIO
A Nervana client application (for instance, the semantic browser) or a Nervana skin will sink this event to get responses to method calls on the control.
PROTOTYPE
VOID
OnQueryResultsAvailable(
[in] GULD QuerylD,
[in] SCODE QueryResult,
[in] BSTR Results,
[in] DWORD NumResults,
[in] DWORD QueryMask,
[in] VARIANT ResultsParam ); e. Appendix A
QUERY MASK VALUES
#defme QM_RESULTS 0x01
#defme QM_RESULTCOUNT 0x02
#defme QM_NEWRESULTS 0x04 #defme QM_NEWRESULTCOUNT 0x08
#defme QMJDEFAULT ( QM_RESULTS )
Example:
Person_GetInfoAuthored( PersonObjectSRML,
QM_RESULTS | QM_RESULTCOUNT,
&QueryRequestGuid );
K. SECURITY SPECIFICATION FOR THE INFORMATION NERVOUS SYSTEM
1. Authorization
INTRODUCTION
The 'People' DSA will be initialized with an LDAP Directory URL and Group Name. The 'Users' DSA will also be initialized with an LDAP Directory URL and Group Name. Typically, the 'Users' will be a subset of 'People.' For instance, a pharmaceuticals corporation might install a KIS for different large pharmaceutical categories (e.g., Biotechnology, Life Sciences, Pharmacology, etc). Each of these will have a group of users that are knowledgeable or interested in that category. However, the KIS will also have the 'People' group populated with all employees of the corporation. This will enable users of the KIS to navigate to members of the entire employee population even though those members are not users of the KIS. hi addition, the inference engine will be able to infer expertise with semantic links off people that are in the corporation, not necessarily just users of the KIS.
This is also advantageous for access control at the KIS level - this complements or supplements access control provided by the application server at the Web service layer. The Users group will contain people that have access to the KIS knowledge. However, the People group will contain people that are relevant to the KIS knowledge, even though those people don't have access to the KIS.
Both People and Users DSA populate the People table in the Semantic Metadata Store (SMS) and indicate the object type id appropriately. Note that preferably the passwords are NOT stored in the People table in the SMS.
The Users DSA also populates the User Authentication Table (UAT). This is an in- memory hash table that maps the user names to passwords. The server's Web service will implement the LPasswordProvider interface or an equivalent. The implementation of the PasswordProvider object will return the password that maps to a particular user name. The C# example below illustrates this: namespace WSDK_Security
{ public class PasswordProvider : Microsoft.WSDKSecurity.IpasswordProvider
{ public string GetPassword( string username )
{ return "opensezme";
} }
The following C# code shows how the Web service can retrieve the user information after the user has been authenticated: using System; using System.Collections; using System.ComponentModel; using System.Data; using System.Diagnostics; using System.Web; using System.Web. Services; using Microsoft. WSDK. Security; using Microsoft. WSDK; namespace WSDK_Security
{ public class Servicel : System. Web. Services. Web Service
{ [WebMethod] public string PersonalHello()
{ string response = "";
SoapContext requestContext = HttpSoapContext.RequestContext; if (requestContext == null)
{ throw new ApplicationException("Non-SOAP request.");
} foreach (SecurityToken tok in requestContext.Security. Tokens)
{ if (tok is UsernameToken)
{ response += "Hello " + ((UsernameToken)tok).Username;
} } return response; } } }
The Nervana Web service can then go ahead and call the Server Semantic Runtime with the calling user name. The runtime then maps this to SQL and uses the appropriate filters to issue the semantic query.
For the Nervana ASP.NET application, the following entry is added as a child of the parent configuration element in the Web.config file:
<microsoft.wsdk> <security> <p asswordPro vider type="WSDK_Security.PasswordProvider, WSDK-Security" /> </security> </microsoft.wsdk> a. Client-Side Authorization Request
In order to create a UsernameToken for the request, the Nervana client has to pass the username and password as part of the SOAP request. The Nervana client can pass multiple tokens as part of the request - this is preferable for cases where the user's identity is federated across multiple authentication providers. The Nervana client will gather all the user account information the user has supplied (including user name and password information), convert these to WS-Security tokens, and then issue the SOAP request. The client code will look like the following (reference: http://www.msdn.microsoft.com): localhost.Servicel proxy = new localhost.Servicel(); UsernameToken clearTextToken = new UsernameToken("Joe",
"opensezme",
Pass wordOption. S endHashed) ; proxy.RequestSoapContext.Security.Tokens.Add(clearTextToken); labell .Text = proxy.PersonalHello(); b. Validating the UsernameToken on the Server
(http://msdn.microsoft.com/library/default.asp?url=/library/en- us/dnwssecur/html/wssecwithwsdk. asp) Although the WSDK verifies the Security header syntax and checks the password hash against the password from the Password Provider, there is some extra verification that is preferably be perfonned on the request. For instance, the WSDK will not call the Password Provider if a UsernameToken is received that does not include a password element. If there is no password to check, there is no reason to call the password provider. This means we need to verify the format of the UsernameToken ourselves.
Another possibility is that there is more than one UsernameToken element included with the request. WS-Security provides support for including any number of tokens with a request that may be used for different purposes.
The code above can be modified for the Nervana Web method to verify that the
UsernameToken includes a hashed password and to only accept incoming requests with a single
UsernameToken. The modified code is listed below. [WebMethod] public string ProcessSemanticQuery( string Query )
{ SoapContext requestContext = HttpSoapContext.RequestContext; if (requestContext = null)
{ throw new ApplicationException("Non-SOAP request."); ι i if (requestContext. Security. Tokens. Count =— 1)
{ foreach (SecurityToken tok in requestContext. Security. Tokens)
{ if (tok is UsernameToken)
{ UsernameToken UserToken = (UsernameToken)tok; if (UserToken.PasswordOption = PasswordOption.SendHashed)
{ return ProcessSemanticQueryInternal( Query, UserTokemUsername );
} else
{ throw new SoapException(
"Invalid UsernameToken password type.",
SoapException. ClientFaultCode) ; } } else
{ throw new SoapException( "UsernameToken security token required.", S oapException.ClientFaultCode) ;
}
}
} else
{ throw new SoapException(
"Request must have exactly one security token.",
SoapException.ClientFaultCode);
} return null;
}
2. People Groups
The KIS will include metadata for people groups. These are not unlike user groups in modern operating systems. The People Group will be a Nervana first-class object (i.e., it will inherit from the Object class). In addition, the People Group schema will be as follows:
Figure imgf000125_0001
In most cases, people groups will map to user groups in directory systems (like LDAP). For instance, the KIS server admin will have the KIS crawl a configurable set of user groups. There will be a People DSA that will crawl the user groups and populate the People Groups and Users tables in the SMS. The People DSA will perform the following actions:
Create the group (if it doesn't exist in the SMS) or update the metadata of the Group (if it exists).
• Enumerate all the users in the group (at the source - an LDAP directory in the preferred embodiment). • For all the users in the group, create People objects (or update the metadata if the objects already exist in the SMS).
• Update the semantic network (via the 'SemanticLinks' table in the SMS) by mapping the people objects to the group objects (using the BELONGS_TO_GROUP semantic link type). This ensures that the SMS has semantic links that capture group membership information (in addition to the groups and users themselves).
3. Identity Metadata Federation
Identity Metadata Federation (LMF) refers to a feature wherein an Information Community (agency) is deployed over the Internet but is used to service corporate or personal customers. For instance, Reuters could set up an information community for all its corporate customers that depend on its proprietary content. In such a case where multiple customers share an information community (likely in the same industry), Reuters will have a group on the SMS for each customer. However, each of these customers would have to have its corporate directory mirrored on Reuters in order for people metadata to be available. This would cause problems, particularly from a security and privacy standpoint. Corporations will probably not be comfortable with having external content providers obtaining access to the metadata of their employees. LMF addresses this problem by having the friternet-hosted information community (agency) host only enough metadata for authentication of the user. For instance, Reuters will store only the logon information for the users of its corporate customers in its SMS. When the semantic browser receives SRML containing such incomplete metadata, the client will then issue another query to the enterprise directory (via LDAP access or via UDDI if the enterprise directory metadata is made available through a Web services directory) to fetch the complete metadata of the user. This is possible because the externally stored metadata will have the identity information with which the remaining metadata can be fetched. Since the client fetches the remaining metadata within the firewall of the enterprise, the sensitive corporate metadata is not shared with the outside world. 4. Access Control a. Access Control Policy
In the preferred embodiment, the KIS will include and enforce access control semantics. The KIS employs a policy of "default access." Default access here means that the KIS will grant access to the calling user to any metadata in the SMS, except in cases where access is denied. As such, the system can be extended to provide new forms of denial, as opposed to new forms of access. In addition, this implies that if there is no basis for denial, the user is granted access (this leads to a simpler and cleaner access control model).
The KIS will have an Access Control Manager (ACM). The ACM is primary responsible for generating a Denial Semantic Query (DSQ) which the SQP will append to its query for a given semantic request from the client. The ACM will expose the following method (C# sample):
String GetDenialSemanticQuery( String CallingUserName )
Preferably, the method takes in the calling user name and returns a SQL query (or equivalent) that encapsulates exception objects. These are objects that must not be returned to the calling user by the SQP (i.e., objects for which the user does not have access).
The SQP then builds a final raw query that includes the denial query as follows:
Aggregate Raw Query AND NOT IN (Denial Query)
For example, if the aggregate raw query is:
SELECT OBJECTED FROM OBJECTS WHERE OBJECTTYPEID = 5, and the denial query is:
SELECT OBJECTLD FROM OBJECTS WHERE OWNERUSERNAME <> 'JOHNDOE',
The final raw query (which is that the SQP will finally execute and serialize to SRML to return to the calling user) will be:
SELECT OBJECTJX) FROM OBJECTS WHERE OBJECTTYPEID = 5 AND NOT IN (SELECT OBJECTED FROM OBJECTS WHERE OWNERUSERNAME <> 'JOHNDOE')
Semantically, this is probably equivalent to:
"Select all objects that have an object type id of 5 but that are not in an object list not owned by John Doe."
This in turn is probably semantically equivalent to:
"Select all objects that have an object type id of 5 that are owned by John Doe." b. General Access Control Rules
Each semantic query processed by the semantic query processor (SQP) will contain an access control check. This will guarantee that the calling user only receives metadata that he/she has access to. The SQP will employ the following access control rules when processing a semantic query:
1. Preferably, if the query is for 'People' objects (people, users, customers, experts, newsmakers, etc.), the returned 'People' objects must either:
• Include the calling user, or
• Include people that share at least one people group with the calling user, and be owned by the calling user or the system
Preferably, the corresponding denial query maps to the following rale: The returned objects must satisfy the following:
• Is not the calling user +
Is not owned by the calling user or the system +
• Has people that do not share any people group with the calling user Sample Denial Query SQL
The SQL below illustrates the access control denial query that will be generated by the ACM and appended by the SQP to enforce the access control policy. In this example, the name of the calling user is 'JOHNDOE.' SELECT OBJECTID FROM OBJECTS WHERE
OWNERUSERNAME o 'JOHNDOE' OR
OWNERUSERNAME o 'SYSTEM' OR
WHERE OBJECTID NOT IN (SELECT OBJECTLD FROM PEOPLE WHERE NAME='JOHNDOE') OR
WHERE OBJECTID NOT LN
(SELECT OBJECTID FROM SEMANTICLINKS WHERE
OBJECTTYPEID = "PERSON AND
PREDICATETYPEΓD='BELONGS_TO_GROUP' AND SUBJECTID IN (SELECT SUBJECTID FROM SEMANTICLINKS WHERE OBJECTID IN (SELECT OBJECTID FROM PEOPLE WHERE NAME=' JOHNDOE' ) )
2. Preferably, if the query is for non-People objects (documents, email, events, etc.), the returned objects must:
• Be owned by the calling user or the system user, and
• Be the subject of a semantic link with the calling user as the object, or
• Be the object of a semantic link with the calling user as the subject, or
• Be the subject of a semantic link with the object being a person that shares at least one people group with the calling user, or
• Be the object of a semantic link with the subject being a person that shares at least one people group with the calling user
Preferably, the corresponding denial query maps to the following rule: The returned objects must satisfy the following:
• Is not owned by the calling user + 0 Is not owned by the system user +
0 Is not the subject of a semantic link with the calling user as the object +
° Is not the object of a semantic link with the calling user as the subj ect +
• Is not the subject of a semantic link with the object being a person that shares at least one people group with the calling user +
• Is not the object of a semantic link with the subject being a person that shares at least one people group with the calling user
Sample Denial Query SQL
The SQL below illustrates the access control denial query that will be generated by the ACM and appended by the SQP to enforce the access control policy. In this example, the name of the calling user is 'JOHNDOE.'
SELECT OBJECTID FROM OBJECTS WHERE OWNERUSERNAME <> 'JOHNDOE' OR
OWNERUSERNAME o 'SYSTEM' OR
OBJECTID NOT LN (SELECT OBJECTID FROM SEMANTICLINKS WHERE OBJECTTYPEID = "PERSON' AND OBJECTID LN (SELECT OBJECTED FROM PEOPLE WHERE NAME-'JOHNDOE') OR
WHERE OBJECTED NOT IN (SELECT OBJECTID FROM SEMANTICLINKS INNER JOLN PEOPLE WHERE SEMANTICLLNKS.SUBJECTTYPELD='PERSON' AND SEMANTICLΓNKS.SUBJECTED = PEOPLE.OBJECTED) OR
OBJECTED NOT IN (SELECT OBJECTED FROM SEMANTICLINKS WHERE OBJECTTYPEED='PERSON' AND PREDICATETYPEED='BELONGS_TO_GROUP' AND SUBJECTED IN (SELECT SUBJECTED FROM SEMANTICLINKS WHERE OBJECTED IN (SELECT OBJECTED FROM PEOPLE WHERE NAME='JOHNDOE')) OR
OBJECTED NOT IN (SELECT OBJECTED FROM SEMANTICLINKS WHERE OBJECTTYPEED='PERSON' AND PREDICATETYPEED='BELONGS_TO_GROUP' AND OBJECTED IN (SELECT OBJECTED FROM PEOPLE WHERE NAME=' JOHNDOE'))
Sample Merged Denial Query SQL
By merging these two rales, the ACM returns the following merged query to the SQP for access denial:
SELECT OBJECTID FROM OBJECTS WHERE
OWNERUSERNAME <> 'JOHNDOE' OR
OWNERUSERNAME <> 'SYSTEM' OR
OBJECTED NOT IN (SELECT OBJECTED FROM PEOPLE WHERE NAME='JOHNDOE') OR
OBJECTED NOT IN (SELECT OBJECTED FROM SEMANTICLINKS WHERE
OBJECTTYPEID = "PERSON AND
PREDICATETYPEED='BELONGS_TO_GROUP' AND SUBJECTID IN (SELECT SUBJECTED FROM SEMANTICLINKS WHERE OBJECTID IN (SELECT OBJECTED FROM PEOPLE WHERE NAME=' JOHNDOE')) OR
OBJECTED NOT IN (SELECT OBJECTED FROM SEMANTICLINKS WHERE OBJECTTYPEED = "PERSON' AND OBJECTED IN (SELECT OBJECTED FROM PEOPLE WHERE NAME=' JOHNDOE') OR
OBJECTED NOT IN (SELECT OBJECTID FROM SEMANTICLINKS INNER JOIN PEOPLE ON SEMANTICLLNKS.SUBJECTTYPEED='PERSON' AND
SEMANTICLLNKS.SUBJECTED = PEOPLE.OBJECTED) OR
OBJECTED NOT IN (SELECT OBJECTED FROM SEMANTICLINKS WHERE OBJECTTYPEED='PERSON' AND PREDICATETYPEED='BELONGS_TO_GROUP' AND SUBJECTED IN (SELECT SUBJECTID FROM SEMANTICLINKS WHERE OBJECTED IN (SELECT OBJECTED FROM PEOPLE WHERE NAME=' JOHNDOE')) OR
OBJECTED NOT IN (SELECT OBJECTED FROM SEMANTICLINKS WHERE OBJECTTYPEED='PERSON' AND PREDICATETYPEED='BELONGS_TO_GROUP' AND OBJECTED IN (SELECT OBJECTED FROM PEOPLE WHERE NAME=' JOHNDOE'))
Example Scenario
For instance, A Reuters agency (KIS) might have people groups for each enterprise customer that Reuters serves. The agency will have a common information base (Reuters content) but will have people groups per enterprise customer. These groups might include competitors. As such, it is preferable to ensure that the knowledge flow, generation, and inference do not cross competitor boundaries. For instance, an employee of Firm A must not derive knowledge directly from an employee of Firm B that competes with Firm A, not must he or she derive knowledge indirectly (via inference). An employee of Firm A must not be able to get recommendations for items annotated by employees of Firm B. Or an employee of Firm A must not be able to find experts that work for Firm B. Of course, this assumes that Firm A and Firm B are not partners in some fashion (in which case, they might want to share knowledge). In the case of knowledge partners, Reuters would create a people group (likely via LDAP) that includes the people groups of Firm A and Firm B. The Reuters KIS will then have the following people groups: Firm A, Firm B, and Finns A&B. The SMS will also include metadata that indicates that the people in Firms A and Firms B belong to these groups (via the "belongs to group" semantic link type). With this process in place, the aforementioned rales will guarantee that knowledge gets shared between Firms A and B. c. Access Control Rules for Annotations In the case of annotations, the calling user will be editing the semantic network, as opposed to querying it. hi this case, the following rales would apply:
1. Preferably, if the object being annotated is a Person object, the object must either be:
• The calling user, or
• A person that shares at least one people group with the calling user, and be owned by the calling user or the system
2. Preferably, if the object being annotated is a non-Person object (e.g., a document, email, event, etc.), the object must either be:
• Owned by the calling user Owned by the system Sample Denial Query SQL
The SQL below illustrates the access control denial query that will be generated by the ACM (for checking access control for annotations) and appended by the SQP to enforce the access control policy, hi this example, the name of the calling user is 'JOHNDOE.'
SELECT OBJECTID FROM OBJECTS WHERE
OWNERUSERNAME o 'JOHNDOE' OR
OWNERUSERNAME o 'SYSTEM' OR
OBJECTED NOT IN (SELECT OBJECTID FROM PEOPLE WHERE NAME^'JOHNDOE') OR
OBJECTED NOT IN (SELECT OBJECTED FROM SEMANTICLINKS WHERE OBJECTTYPEED='PERSON' AND PREDlCATETYPEED='BELONGS_TO_GROUP' AND OBJECTED IN (SELECT OBJECTED FROM SEMANTICLINKS WHERE OBJECTED IN (SELECT OBJECTED FROM PEOPLE WHERE NAME=' JOHNDOE'))
Access Control Enforcement
The ACM enforces access control for annotations and other write operations on the KIS. The KIS XML Web Service exposes an annotation method as follows (C# sample):
AnnotateObject( String CallingUserName, String ObjectED );
This method calls the ACM to get the denial query. It then creates a final query as follows:
Annotation Object Query AND NOT IN (Denial Query)
In the preferred embodiment, the annotation object query is always of the form:
SELECT OBJECTID FROM OBJECTS WHERE OBJECTED=ObjectED, where ObjectED is the argument to the AnnotateObject method.
The ACM then builds a final access control query SQL and uses this SQL to check for access control. Because the ACM does not have to return the SQL, it merely invokes it directly in order to check for access control. In addition, because it is a binary check (access or no access), the ACM merely checks whether the denial query returns at least one row. For instance, a final query might look like:
SELECT OBJECTID FROM OBJECTS WHERE OBJECTED = ObjectED AND NOT IN (SELECT OBJECTED FROM OBJECTS WHERE OWNERUSERNAME o 'JOHNDOE') The ACM then runs this query (via the SQL query processor) and asks for the count of the number of rows in the result set. If there is one row, access is granted, else access is denied. Tins model is implemented this way in order to have consistency with the denial query model (the ACM always builds a denial query and uses this as a basis for all access control checks).
L. DEEP INFORMATION SPECIFICATION FOR THE INFORMATION NERVOUS SYSTEM
Deep Information Overview
INTRODUCTION
In the preferred embodiment, the Nervana 'Deep Info' tool is aimed at providing context- sensitive story-like information for a Nervana information object. Deep Info essentially provides Nervana users with information that otherwise would be lost, given a particular context. By way of rough analogy, Deep Info is like the contextual information that gets displayed on music videos on MTV (showing information on the current artist, the current song, and in some case, the current musical instrument in the song).
The 'deep' in 'deep info' refers to the fact that the contextual information will often span multiple "hops" in the semantic network on the agency from whence the object came. 'Deep Info' is comprised of 'deep info nuggets' which can either be plan textual metadata or metadata with semantic query links (via SQML).
In the preferred embodiment, there are at least five kinds of Deep Info nuggets:
1. Basic Semantic Link Nuggets
2. Context Template Nuggets
3. Trivia Nuggets
4. Matchmaker Nuggets
5. Recursive Nuggets a. Basic Semantic Link Nuggets
With basic semantic link truths, deep info nuggets merely convey a semantic link of the current object. These nuggets involve a semantic link distance of 1. hi this case, there is overlap with what will be displayed in the 'Links' context/task pane. Examples are: Patrick Schmitz reports to Nosa Omoigui Patrick Schmitz has 5 Direct Reports Patrick Schmitz annotated 47 objects Patrick Schmitz authored 13 objects Patrick Schmitz was copied on 56 email objects b. Context Template Nuggets
Context template nuggets display contextual information for each relevant context template, based on the information at hand. These nuggets are identical to those that will be displayed in the context bar or context panel for each type of context template. For example:
Patrick Schmitz posted 3 breaking news items Patrick Schmitz posted 14 classics Patrick Schmitz authored 7 headlines Patrick Schmitz is involved in 13 discussions Patrick Schmitz is a newsmaker on 356 objects c. Trivia Nuggets
For all email objects on an agency:
° Steve Judkins appears on the "To" list of all of them
• Steve Judkins replied to 23% of them
• Patrick Schmitz annotated 50%o of them
• Only 3 of these have a thread depth greater than 2
For all people objects on an agency:
Patrick Schmitz has sent email to 47% of them ° 14% of them report to Nosa Omoigui
Sally Smith has had discussions with' 85% of them
12% of them are newsmakers on at least one topic
All of them have been involved in at least one discussion this week
33% of them are experts on at least one topic
8%> of them are experts on more than three topics
For a given distribution list on an agency:
Steven Judkins has posted the most email to this list
Sarah Trent has replied to the most email on this list
Nosa Omoigui has never posted to this list
Patrick Schmitz has posted 87 messages to this list this month
Richard Novotny has posted 345 messages to this list this year For all distribution lists on an agency:
Steven Judkins has posted the most email to all lists
Lisa Heibron has replied to email on only 2% of the lists
Nosa Omoigui has never posted to any list
Patrick Schmitz has posted at least once every week to all the lists
Richard Novotny has posted messages on 3 lists
For all information objects on an agency:
Steven Judkins has been the most prolific publisher (he published 5% of them)
Sally Smith has been the most prolific annotator (she annotated 2% of them)
Nosa Omoigui has been the most active newsmaker
Patrick Schmitz has the most aggregate expertise
Steve Judkins has the most expertise for information published this year
Gavin Schmitz has been involved in the most discussions (12% of them)
Richard Novotny has been involved in the most discussions this month (18% of them) d. Matchmaker Nuggets
Person To Person Semantic Link Based
• Patrick Schmitz has sent mail to 13 people
• 47 people have appeared on same To list as Patrick Schmitz
• 47 people have appeared on same CC list as Patrick Schmitz
89 people in total have been referenced on email sent by Patrick Schmitz 24 people have annotated the same information as Patrick Schmitz ° 3 people are on all the same distribution lists as Patrick Schmitz
• 29 people are on at least one of Patrick Schmitz's distribution lists
Context-Template Based
• 12 people have expertise on the same information categories as Patrick Schmitz
• 14 people and Patrick Schmitz are newsmakers on the same information items
• 27 people are in discussions with Patrick Schmitz
Information To Person
Semantic Link Based
• Patrick Schmitz posted this information item
• Steve Judkins authored this information item
• This information item was copied to 2 people
• 3 people annotated this information item Context Template Based (similar to context template nuggets)
• There are 4 experts on this information item
• There are 27 newsmakers on this information item
Information To Information
Context Template Based (similar to context template nuggets)
There are 578 relevant 'all bets'
• There are 235 relevant 'best bets'
• There are 4 relevant breaking news items
• There are 46 relevant headlines
Semantic Link Based (via people)
• There are 21 information items that have the same experts with this one
• There are 23 information items that have the same newsmakers with this one
• There are 34 information items posted by the same person that posted this one
• There are 34 information items authored by the same person that authored this one
• There are 44 information items annotated by people that annotated this one e. Recursive Nuggets
With recursive nuggets, displaying deep info on the subject of the current information nugget forms a contextual hierarchy. The system then recursively displays the nuggets based on the object type of the subject. With recursive nuggets, the system essentially probes the semantic network starting from the source object and continues to display nuggets along the path of the network. Probing is preferably stopped at a depth that is consistent with resource limitations and based on user feedback.
Another way to think of recursive nuggets is like a contextual version of an business organization chart. However, with Deep Information in the Information Nervous System, users will be able to browse a tree of KNOWLEDGE, as opposed to a tree of INFORMATION. To take an example, if a user selects an object and a tree view will show up like what is displayed below:
Example with document as context: [+]Newsmakers on 'Title of document' [+] Gavin Schmitz
[+] Reports To -> [+] Steve Judkins
[+] Experts Like Steve Judkins ->
[+] Nosa Omoigui
[+] Patrick Schmitz [+] Interest Group Like Steve Judkins ->
[+] Patrick Schmitz
[+] Chuck Johnson
[+]Direct Reports ->
[+]Joe Williams
[+] Direct Reports D
[+] Interest Group Like Joe Williams ->
[+] Richard Novotny
[+] Nosa Omoigui
[+] Interest Group [+] Experts
Example with email as context:
[+] Email is From:
[+] Nosa Omoigui
[+] Experts like Nosa Omoigui
[+] Email is To:
[+] Chuck Johnson
[+] Experts like Chuck Johnson
[+] Email is Copied To:
[+] Richard Novotny
[+] Experts like Richard Novotny
[+] Email Attachments: foo.doc
[+] Experts on foo.doc
[+] Gavin Schmitz
[+] Newsmakers like Gavin Schmitz
[+] Newsmakers on 'Title of Email'
Example with conversation object as context: [+] Conversation Participants [+] Steve Judkins
[+] Interest Group Like Steve Judkins...
[+]Nosa Omoigui
[+] Interest Group Like Nosa Omoigui
[+] Experts on 'Title of Conversation' [+] Richard Novotny
[+] Interest Group Like Richard Novotny
Notice the use of default predicates in the above example - e.g., with People subjects linked to People objects, the LIKE predicate is uses (e.g., Interest Group LIKE Richard
Novotny).
Another example of recursive nuggets is shown below:
[+] Patrick Schmitz authored this email
[+] Patrick Schmitz reports to Nosa Omoigui
[+] Nosa Omoigui has 6 Direct Reports
[+] Steve Judkins ... [+] Steve Judkins posted ...
[+] Steve Judkins is an expert on ... [+] Steve Judkins is a newsmaker on ... [+] Steve Judkins has been involved in 6 discussions [etc.]
[+] Richard Novotny... [+] [The remaining 6 direct reports] [+] Nosa Omoigui annotated 13 objects...
[+] [More context template nuggets on the 13 objects] [+] Nosa Omoigui has authored 278 objects [+] Nosa Omoigui has annotated 23 items [...]
[+] Patrick Schmitz has 5 Direct Reports [+] John Doe ...
[+] More Native Nuggets based on the direct reports [+] Patrick Schmitz annotated 47 objects hi the preferred embodiment, recursive nuggets will most typically be displayed via a drill-down pane beside each result object in the semantic browser. This will allow the user to select a result object and then recursively and semantically "explore" the object (as illustrated above). Also, each header item in the Deep Info drill down tree view will be a link to a request (e.g., Experts Like Steve Judkins), and each result will be a link to an entity. For example, users will be able to "navigate" to the "person" (semantically) Patrick Schmitz from anywhere in the Deep Info tree view. Users will then be able to view a dossier on Patrick Schmitz, copy Patrick Schmitz, and Paste it on, say, Breaking News - in order to open a request called Breaking News by Patrick Schmitz. Again, notice the use of a default predicate based on the Person subject ("BY").
The preferred embodiment Presenter Deep frifo tree view (with support from the semantic runtime API in the semantic browser) will keep track of those links that are requests and those links that are result objects; that way, it will intelligently interpret the user's intent when the user clicks on a link the tree view (it will navigate to a request or navigate to an entity).
M. CREATE REQUEST WIZARD SPECIFICATION FOR THE INFORMATION
NERVOUS SYSTEM
Introducing the Create Request Wizard
OVERVIEW
The preferred embodiment Create Request (or Smart Agent) Wizard allows the user to easily and intuitively create new requests that represent semantic queries to be issued to one or more knowledge sources (running the Knowledge Integration Service).
Wizard Page 1 : Select a Profile and Request Type: This page allows the user to select what profile the request is to be created in. The page also allows the user to select the type of request he/she wants to create. This type could be a Dossier (Guide) which will create a request containing sub-requests for each context template (based on the filters indicated in the request), knowledge types (corresponding to context templates such as Best Bets, Headlines, Experts, Newsmakers, etc.), information types (corresponding to types such as Presentations, General Documents, etc.), and request collections which are Blenders and allow the user to view several requests as a cohesive unit. See Figure 17 A. Wizard Page 2: Select Knowledge Communities (Agencies): This page allows the user to select which knowledge communities (running on Knowledge Integration Servers (KISes) the request should get its knowledge from. The user can indicate that the request should use the same knowledge communities as those configured in the selected profile. The user can alternatively select specific knowledge communities. See Figure 17B.
Wizard Page 3: Select Filters: This page allows the user to select which filters to include in the request. Filters can include one or more of the following: keywords, text, categories, local documents, Web documents, email addresses (for People filters), and Entities. In alternate embodiments, other filter types will be supported. The property page also allows the user to select the predicate with winch to apply a specific filter. Preferably, the most common predicate that will be exposed is "Relevant to." Other predicates can be exposed consistent with the filter type (for instance a filter that refers to a Person via an email address or entity will use the default predicate "BY" if the requested type is not 'People' - e.g. Headlines BY John Smith and will use the default predicate "LIKE" if the request type is 'People' - e.g., Experts LIKE John Smith). The property page also allows the user to select the operation with which to apply the filters. The two most common operators are AND (in which case only results that satisfy all the filters are returned) and OR (in which case results that satisfy any of the filters are returned). See Figure 17C.
Wizard Page 4: Name and describe this request: This page allows the user to enter a name and description for the request. The wizard automatically suggests a name and description for the request based on the semantics of the request. Examples include:
1. Headlines on Security AND on Application Development AND on Web Services.
2. Experts from R&D on Encryption Techniques OR on User Interface Design, etc.
3. Presentations on Artificial Intelligence.
4. Dossier on Data Mining AND on Web Development. See Figure 17D.
The user is allowed to override the suggested name/description. The suggestions are truncated as needed based on a maximum name and description length. The semantic browser also exposes the properties of an existing request via a property sheet. This allows the user to "edit" a request. The property sheet exposes the same user interface as the wizard except that the fields are initialized based on the semantics of the request (by de-serializing the request's SQML representation). See Figure 17E.
N. CREATE PROFILE WIZARD SPECIFICATION FOR THE INFORMATION NERVOUS SYSTEM
Introducing the Create Profile Wizard
OVERVIEW
The Create Profile Wizard allows the user to easily and intuitively create new user profiles.
Wizard Page 1 : Select your areas of interest: This page allows the user to select his/her areas of interest. This allows the semantic browser to get some high-level information about the user's knowledge interests (such as the industry he/she works in). This information is then used to nan-ow category selections in the categories dialog, recommend new knowledge communities (agencies) configured with knowledge domains consistent with the user's area(s) of interests, etc. See Figure 45 A.
Wizard Page 2: Select your knowledge communities: This page allows the user to subscribe to knowledge communities for the profile. This allows the semantic browser to "know" winch knowledge sources to issue requests to, when those requests are created for the profile. The semantic browser also uses the knowledge communities in the profile when it invokes Visualizations, semantic alerts, the smart lens (when the lens is a request/agent for the given profile), the object lens (when the target object is a result from the given profile), when the user drags and drops (or copies and pastes) an object to a request/agent for the given profile, etc. See Figure 45B.
Wizard Page 3: Name and describe this profile: This page allows the user to enter a name and description for the profile. The page also allows the user to indicate whether the profile is preferably made the default profile. The default profile is used when the user does not explicitly indicate a profile in any operation in the semantic browser (for example, dragging and dropping a document from the file system to the icon representing the semantic browser will open a bookmark with that document from the default profile, whereas dragging and dropping a document to an icon representing a specific profile will open a bookmark with that profile). See Figure 45C.
O. CREATE BOOKMARK WIZARD SPECIFICATION FOR THE INFORMATION NERVOUS SYSTEM
1. Introducing the Create Bookmark Wizard
OVERVIEW
The Create Bookmark (or Local/Dumb Request Agent) Wizard allows the user to easily and intuitively create new bookmarks (local/dumb requests) to view local/Web documents, entities, etc. in the semantic browser via which he/she can get access to the toolbox of the system (i.e., drag and drop, smart copy and paste, smart lens, smart alerts, Visualizations, etc.).
Wizard Page 1 : Select a Profile and Request Type: This page allows the user to select what profile the bookmark is to be created in. The page also allows the user to add/remove items to/from the bookmark. See Figure 46A.
Wizard Page 2: Name and describe this bookmark: This page allows the user to enter a name and description for the bookmark. The wizard automatically suggests a name and description for the bookmark based on the items in the bookmark. Examples include:
• Document 1, Document 2, and Document 3
• Documents Matching 'Encryption'
• Documents in the Folder 'My Documents' and Subfolders
• Nervana Presentation (July 2003).ppt AND Documents Matching "Security" in the Folder 'My Documents' and Subfolders
The user is allowed to override the suggested name/description. The suggestions are truncated as needed based on a maximum name and description length. See Figure 46B. 2. Scenarios
Show me all Presentations on Protein Engineering
Using the Create Request Wizard, select the Presentations information-type (in Documents\Presentations), and then select the Protein Engineering category as a filter. Hit Next - the wizard intelligently suggests a name for the request (Presentations on Protein Engineering) based on the semantics of the request. The wizard also selects the right default predicates. Hit Finish. The wizard compiles the query, sends the SQML to the KISes in the selected profile, and then displays the results.
3. Intelligent Publishing-Tool Metadata Suggestion and Maintenance
While the Information Nervous System does not rely or depend on metadata that is stored by Publishing Tools (e.g., the author of a document), having such metadata available and reliable can be advantageous. One problem with prior art is that publishing tools (e.g., Microsoft Word, Adobe Acrobat, etc.) do not intelligently manage the metadata creation and maintenance process. Here are some ways that the preferred embodiment of the present invention can be used to make the metadata creation and maintenance process better: a. When the user creates a new document, add the author's email address (this can be programmatieally retrieved from the user's email client and in the event that the user has several addresses, the publishing tool should prompt the user for which address to use) to the metadata header of the document (rather than merely the author's name). This is because email addresses provide much more uniqueness (for instance, the name 'John Smith' could refer to one of millions of people - as such the existence of such data in the metadata of a document is not that useful). Note that one possible email address to use in the metadata header can be retrieved from, say, the logged on user's single sign-on account (e.g., Microsoft Passport™). b. When the document is edited and if the current user is different from the author of the document (as is indicated in the metadata header), prompt the user if he/she wants to change the metadata header accordingly. This provides some basis form of intelligent metadata maintenance. This model can be applied across different object types and metadata fields in cases where the publishing tool can validate the field (e.g., as in the case of the currently logged on user's name and email address).
P. SEMANTIC THREADS SPECIFICATION FOR THE INFORMATION
NERVOUS SYSTEM™
1. Semantic Threads
OVERVIEW
In the preferred embodiment, semantic threads are objects in the KIS semantic network that represent threads of annotations or conversations. They are different from regular email threads in that they are also semantic - they have object identifiers and type identifiers (the OBJECTTYPEED THREAD identifier) thread-specific semantic links, they convey meaning via one or more ontology-based knowledge domains and they support dynamic linking. Also, because they are first-class objects in the Information Nervous System, they can be queried, copied, pasted, dragged, dropped, and used with the smart and object lenses. Figure 23 illustrates a semantic thread object and its semantic links.
Because a semantic thread object is a first-class member of the semantic network and the entire Information Nervous System, it is subject to manipulation, presentation, and querying like other objects in the system. For example, the semantic browser will allow the user to navigate from a Person object to all threads that that person has participated in (via the "Participant" predicate - with a predicate type id of PREDICATETYPEED_PARTICff ANTOFTHREAD). The user can then navigate from the thread to all the thread's participants (People) and keep dynamically navigating from then on. To take another example, a thread object can also be a Best Bet in a given context (or none, if none is specified).
In the preferred embodiment, the semantic thread object also conveys meaning. This is advantageous because it means that the thread can be returned via a semantic query in the system. For instance, "Find me all threads on Topic A and Topic B." The KIS maintains semantic links for semantic threads just like it does with other objects such as documents. However, because semantic threads can refer to multiple objects, the semantics of the thread evolve with the objects the thread contains. For example, a thread can start with one topic and quickly evolve to include other topics. Email threads can end in a very different "semantic domain" from where they started - participants introduce new perspectives, new information is added to the thread, email attachments might be added to the thread, etc., all on the basis of meaning.
The KIS manages the "semantic evolution" of semantic threads. It does this by adding semantic links to the thread to "track" the contents of the thread. For instance, if a thread starts off with one document and an annotation, the KIS adds a semantic link to the thread for each to which the category the document and annotation belong. In other words, the thread is asserted to have the same semantics as the document and annotation it contains. If another annotation is added to the thread (e.g., if a user annotates the first annotation), the KIS computes a new link strength for the categories of the new annotation that are already linked off the thread. It is preferable if it does this because the new annotation can attenuate or strengthen the semantics of the entire thread from a particular perspective. However, this modification of the strength of the semantic link(s) for the categories that are already present off the thread are preferably done on a per-category basis - as with other objects, the thread can belong to multiple categories with different strengths. The new link strength can be computed in at least two ways: in a simple embodiment, the average of all link strengths for the category being linked to the thread is used. However, this has the disadvantage that too many items in the thread of weak strength can erode the "perceived" (as far as the KIS semantic query processor is concerned) semantics of the entire thread. An alternative embodiment is to use the maximum link strength. However, this also has a disadvantage that the semantics of the thread might remain fixed to a domain/category even though the thread "has moved on" to new domains/categories. From a weighted-average perspective, this would likely return confusing results as the thread grows in size. hi the preferred embodiment, the KIS preferably computes a weighted average of all the link strengths for the categories to be linked to the thread. This new weighted average becomes the link strength. The weighted average is preferably computed using the number of concepts in each object in the thread. This has the benefit of ensuring that "semantically light" objects (such as short postings) do not erode the semantics of the thread relative to "semantically denser" objects in the thread (such as email attachments and long postings). The number of concepts, and not the size, is preferably used in the preferred embodiment because the size of the object is a less reliable indicator of the conceptual weight of the object. For instance, a document could contain images or could include much information that does not map well to key phrases or concepts
Preferably, the computed weight could also include the time when the entry was added (thereby "aging" the semantics of older items relative to newer ones). This weight is then multiplied by the category link strength and the multiples are added and then divided by the number of entries. Other weighting schemes can also be applied.
The following rules are applied when a new item is added to the semantic network and which is to be added to a semantic thread:
1. Categorize the new item to be added to the thread
2. For each category in the returned list of categories which are already on the semantic thread
{
° Compute new weighted-average link strength
• Update category semantic link off the semantic thread object }
3. For each category in the returned list of categories which are not already on the semantic thread
{
• Assign link strength
• Add category semantic link off the semantic thread object }
The weighted-average link strength is computed as follows: New Link Strength = ∑ Ci * Li
N Where Ci is the normalized number of concepts (from 0 to 1) of object i, Li is the link strength of object i, and N is the number of objects in the thread (including the new object). The normalized number of concepts is computed by dividing the number of concepts in each object (extracted by the Knowledge Domain Manager (KDM)) by the number of concepts in the largest object in the thread (including the new object).
If a semantic thread comprises of standard, intrinsic (and unedited) email threads, the KIS modifies the semantic network differently. This is because most email clients include all prior email messages that form the thread in the most recent email message. As such, in this case, the KIS preferably simply uses the most recent email message as being representative of the entire thread. To accomplish tin's, the KIS preferably categorizes the most recent email message, and replace all prior semantic links (relating to categories) from the thread object with new semantic links corresponding with the new categories and link strengths.
For non-email threads (for example, threads that form based on an annotation of an existing object in the semantic network), the model described above should be employed. Alternatively, the KIS can maintain an Aggregate Thread Document (ATD) which is then categorized. This document should contain the text of the objects in the thread - roughly analogous to how an email message contains the text of prior messages in the same thread.
When a new object is added to the thread, the KIS preferably updates the last-modified- time of the thread object in the Semantic Metadata Store (SMS).
2. Semantic Thread Conversations
Semantic thread conversations in the Information Nervous System are a special form of semantic threads. Essentially, a conversation is a semantic thread that has more than one participant. Semantic thread conversations have the object type id,
OBJECTTYPEID THREADCONVERSATION. The KIS creates a thread based on the number of participants in that thread and could immediately create the thread as a thread conversation. Alternatively, the KIS could "upgrade" a thread to a conversation once additional participants are detected.
3. Semantic Thread Management
The pseudo-code below illustrates how the KIS adds preferred threads and conversations to the semantic network:
1. If an individual email message is detected and is a member of an existing thread object
{
Add the new email object to the thread and update the semantic network
If the thread has more than one participant, change the thread's object type identifier to
OBJECTTYPEID THREADCONVERSATION
}
2. If an email thread is detected {
Create a new thread object and update the semantic network If the thread has more than one participant, change the thread's object type identifier to OBJECTTYPEED THREADCONVERSATION
If an email annotation of an existing object is detected
{
• Add the annotation to the semantic network
• If the annotated obj ect is not itself an annotation
{
• Create a new thread object and update the semantic network
} Else
{
• Add the new annotation to the thread containing the annotated object (i.e., the existing annotation) and update the semantic network
• If the updated thread has more than one participant, change the thread's object type identifier to OBJECTTYPEED THREADCONVERSATION } }
Q. SAMPLE SCREEN SHOTS
Figures 24-44B are additional screen shots further illustrating the functions, options and operations discussed above.
R. SPECIFICATION FOR SEMANTIC QUERY DEFINITIONS &
VISUALIZATIONS FOR THE INFORMATION NERVOUS SYSTEM
1. Semantic Images & Motion a. Overview
Semantic images and motion can be an advantageous component of the preferred embodiment in terms of the Nervana semantic user experience. In other words, the user's experience with the system can be enhanced in an embodiment that has semantic image/motion metadata stored on a Nervana agency (information community) and accessed via the Nervana XML Web Service. In that embodiment, via Nervana, end users will have context and time- sensitive semantic access to their images. Imagine, for example only, using a Getty Images (or Corbis) agent as a smart lens over an email message - when invoked, this will open images that are semantically related to the message. Or, imagine dragging and dropping a document from your hard drive to a Getty agent to view semantically related images. This will involve having image metadata (consistent with an image schema). The Nervana toolbox remains the same - we merely add a new information object type for images. Also, there are semantic skins for semantic images - different views, thumbnails, slide shows, filtering, aggregation, etc. For examples of semantic images, visit: http://creative.gettyimages.com/source/search/resultsmain.asp?source=advSearch&hdnSy nc=Medicine%7E0%2C12%2C449%2C3%2C15%2Cl%2C0%2C0%2C0%2C12287%2C0%2C 7%2C14%2C6%2C3%2C3%2C0%2C12%2C449%2Cen%2Dus&UQR=tfxfwz
Very generally, the properties of the semantic visualizations will vary depending upon several different variables. Among these variables will often be the context, including the context of what feature or property of the system is being invoked. In the next several sections some of the contextual variables that influence the semantic determinations will be listed and/or described, i many instances, there will be overlap or commonality of the variables or determinants of the semantic visualizations, but in some cases, the considerations or combination of considerations will be unique to the particular situation. b. Industry-Specific Semantic Images and Motion Industry-specific semantic images/motion are images/motion that can be used (and in the preferred embodiment are used) as part of the presentation atmosphere for semantic results for one or more categories (that map to industries). For instance, visit http://www.corbis.com and http://www.gettyimages.com and enter a search for the keywords listed below (which, in the aggregate, map to target industries, based on industry-standard taxonomies). Such images/motion can also be used as backgrounds, filter effects, transformations, and animations for context and category skins (that map to context templates and categories), hi addition, these images/motion can be used for visuals for motion paths extracted from some of these images for superior screensavers. For example, imagine a skin displaying metadata and visualizations along a motion path extracted from one of these semantic images (e.g., metadata rotating inside a light bulb - for the "electric utilities" industry), along with chrome with other surrounding images and animations, etc. Other industries, with industry specific images and motion might include:
Figure imgf000150_0001
Figure imgf000151_0001
For example, if the user launches a request/agent, Headlines on Bioinformatics or on Protein Engineering, the semantic browser will map the biotechnology-related categories from the SQML to a set of images in the biotechnology industry. It will then display one or more images as part of the skin for the results of the request/agent (thereby proving a pleasant user experience as well as visually conveying the "mood" of the request/agent).
Figure 101 as a sample semantic image for Pharmaceuticals/Biotech industry (artistic DNA helix superimposed over a human face on the left and a organic chemical chart on the right, licensed from the Corbis web site).
The same applies to information types and context templates. Skins will do the smart thing based on the context/information type and the category/ontology and mix and match semantic images/motion across these properties in an intelligent manner. For instance, an agent titled "Headlines on Wireless Technology" can have chrome (and/or a smart hourglass - see below) that shows an image/motion-based animation toggling between a "Headlines" image/motion and a "Wireless" image/motion. A blender titled "Headlines on Wireless and Breaking News on Semiconductors and Email by anyone in my group related to the product specification" can have chrome (and/or a smart hourglass) that "toggles" between images/motion for "Headlines," "News," "Wireless," "Semiconductors," and "Email." The Presenter's query processor can enumerate all context template and information types and all categories (from the agent/blender SQML) and set up the chrome animation accordingly.
For information types, enter searches (e.g., on Corbis and Getty) for:
• Documents • Online Learning
• Email • People
• Books • Users
• Magazines • Customers
• Multimedia
Also, for context templates, enter searches for:
• Headlines • Favorites
• News • Places
• Discovery • Time (for "timeline" and "upcoming events")
• Conversations • Schedule o Experts o Appointment
Also, note that semantic images/motion are preferably not completely random. However, preferably they are not from a bounded set either. Preferably, they are carefully picked and then skins can randomly select from the chosen set. But, preferably they are not random from the entire set on, for example, Corbis or Getty Images. Otherwise there may be silly images, cartoons, and some potentially offensive or inappropriate images. Also, some of these guidelines preferably vary depending on whether the skin theme is in subtle, moderate, exciting, or super- exciting mode. In subtle mode, the skin might decide to choose one image/motion per visualization pivot. In other modes, this would likely lead to a boring user experience.
In low-flashiness mode, the skin can use a semantic image/motion as part of the chrome - not unlike a PowerPoint slide-deck background (e.g., alpha blended). Semantic images/motion can also be used in the smart hourglass (see below) as well as in part of the visualization (on the context bar, panel, or palette). For visualizing context and information types, semantic images/motion are preferably carefully picked to clearly indicate the information type or context, h addition, the selection mode can also be a skin property.
Also, the number of possible semantic images/motion used per skin would likely need to be capped - depending on where the images/motion are being displayed. However, in some scenarios, this might not be necessary. For instance, a blender skin might cycle between chrome backgrounds as the user navigates the blender results (from page to page or agent to agent) - to be consistent with what is currently being displayed from the blender. This can also be a skin property. c. The Client-Side Semantic Image & Motion Cache
The Presenter has a smart expandable client-side cache with semantic images and motions that are downloaded and stored on the client (on installation). Skins can then select from these pre-cached images and motions. The images/motions can be pre-cached based on the user's favorite categories and areas of interest (which he or she selects) - which map to target industries. Skins can then complement the pre-cached semantic images/motions with on-demand image queries to an image server (an XML Web Service that exposes server-side images/motions - hosted by Nervana or a third party like Corbis or Getty Images).
The Presenter will also do the smart thing and have a bias function such that recently downloaded images/motions are selected before older ones (as a tiebreaker). A "usage count" is also cached along with each image/motion - the Presenter uses this count in filtering which images/motions to display and when. Such "load balancing" will yield a fresher and non- repetitive user experience.
The cache is preferably populated on demand (based on the user's semantic queries) - for instance, there is no point in pre-caching pharmaceutical images/motions for a user's machine at Boeing. Preferably, he cache size is also capped and the image cache manager preferably purges "old" and "unused" images using an LRU algorithm or the equivalent. This way, the cache can be in "semantic sync" with the user's agent usage pattern and favorite agent's list. 2. The Smart Hourglass
A majority of the calls that the Nervana Presenter will make to provide the "semantic user experience" probably will be remote calls to the XML Web Service. As such, there will be unpredictable, potentially unbounded delays in the UI. One can expect a fair amount of bandwidth and server horsepower within the enterprise but the Nervana user interface must still "plan" for unknown latency in method invocations.
Operating systems today have this problem with unbounded I/O calls to disk or to the network. Some CPU-bound operations also have substantial delays. In the Windows and Mac UI, the user is made to perceive delay via a "wait" cursor - usually in the shape of an "hourglass."
In the preferred embodiment, the Presenter will have semantic hints (via direct access to the SQML "method call") with which it can display the equivalent of a "smart or semantic hourglass." This could be in the form of an intermediate page that displays "Loading" or some other effect. Additionally, the Presenter can convey the semantics of the query by reading the SQML to get hints on the categories that the query represents and the information type or context template. The Presenter can then use these hints to display semantic images and text consistent with the query, even though it has not received the results. The more hints the query has, the smarter the hourglass can get. The "Loading" page can then convey the atmosphere of "what is to come" - even before the actual results arrive from the Web service and are merged (if necessary) by the Presenter to yield the final results.
This "smart hourglass" can be displayed not just on the main results pane, but perhaps also on smart lens balloon popup windows and inline preview windows (essentially at every call site to the Web service and where there is "focus"). The Presenter can do the smart thing by timing out on the query (perhaps after several hundred milliseconds - the implementation should use usability tests to arrive at a figure for this) before displaying the "hourglass." 3. Visualizations — Context Templates
INTRODUCTION
Context templates are scenario driven information query templates that map to specific semantic models for information access and retrieval. Essentially, context templates can be thought of as personal, digital semantic information retrieval "channels" that deliver information to a user by employing a predefined semantic template. Context templates preferably aggregate information across one or more Agencies.
The context templates described below have been defined. Additional context templates, directed towards the integration and dissemination of varied types of semantic information, are contemplated (examples include context templates related to emotion, e.g., "Angry," "Sad," etc.; context templates for location, mobility, ambient conditions, users tasks, etc.).
BREAKING NEWS
The Breaking News context template can be analogized to a personal, digital version of CNN's "Breaking News" program insert in how it conveys semantic information. The context template allows a user to access information that is extremely time-critical from one or more Agencies, sorted according to the information creation or publishing time and a configurable amount of time that defines information criticality.
Figure 102 is an illustration of a semantically appropriate image visualization for the Breaking News context template.
BREAKING NEWS - SAMPLE OBJECT AND CONTEXT BAR VISUALIZATIONS
Below is a list of sample or representative elements of visualizations appropriate to the Breaking News context. As with all Visualizations (or components thereof) in the preferred embodiment, the "mood" or semantic feeling or connotation will be appropriate to the specific context. By way of very rough analogy, the Visualization will be appropriate to the context within the application in the same way that a "set" must be appropriate to the particular scene in a screenplay for a movie. This will be true not only for this particular Object and Context Bar Visualization, but for all Visualizations in the preferred embodiment. 1. Ticking clock showing publication or scheduled time of most recent or pending breaking news item over a background of the total number of upcoming breaking news items
2. Ticking clock showing publication or scheduled time of most recent or pending breaking news item over semantic image(s)
3. Ticking clock showing publication or scheduled time of most recent or pending breaking news item over semantic image(s) and the total number of breaking news items
4. Ticking clock showing publication or scheduled time of most recent or pending breaking news item over a plain background
5. Non-ticking clocks showing publication or scheduled time of all breaking news items (sequentially) over various backgrounds
6. Calendar view showing publication or scheduled time of most recent or pending breaking news item over various backgrounds
7. Calendar view showing publication or scheduled time of all breaking news items (sequentially) over various backgrounds
8. Scaled font size - depending on the publication or scheduled time of the most recent or pending breaking news item
9. Scaled font size - depending on the number of breaking news items
10. Animated font (e.g., flashing text, rotating text, text on motion path, etc.) with animation rate depending on the publication or scheduled time of the most recent or pending breaking news item
11. Animated font (e.g., flashing text, rotating text, text on motion path, etc.) with animation rate depending on the number of breaking news items
12. Varying font color - depending on the publication or scheduled time of the most recent or pending breaking news item
13. Varying font color - depending on the number of breaking news items
14. Animated graphic of breaking news semantic image(s) or an equivalent
15. Number of breaking news items
16. Titles of breaking news items animated in a sequence (list view)
17. Titles and details of breaking news items animated in a sequence (tiled view)
18. Semantic image/motion moving on an orbital motion path around the object
19. Balloon popup showing number of items on semantic image/motion background
20. Balloon popup showing number of items with plain background but animated with semantic image/motion
HEADLINES
The Headlines context template can be analogized to a personal, digital version of CNN's "Headline News" program in how it conveys semantic information. The context template allows a user to access information headlines from one or more Agencies, sorted according to the information creation or publishing time and a configurable amount of time or number of items that defines information "freshness." For example, CNN's "Headline News" displays headlines every 30 minutes (around the clock). In a preferred embodiment, the Headlines context template will be implemented as a SQL query on the server with the following sub queries chained in sequence: Recommendations Published Today, Favorites Published Today, Best Bets Published Today, Upcoming Events Occurring Today and Tomorrow, Annotated Items Published Today.
Preferably, all sub queries will be sorted by the publishing date/time and then be chained together. Additional filters will be applied to the query based on the predicate list in the SQML. The foregoing principles are illustrated in Figure 103, which is a Headlines Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
CONVERSATIONS CONTEXT TEMPLATE
The Conversations context template can be analogized to a personal, digital version of CNN's "Crossfire" program in how it conveys semantic information. Like "Crossfire," which uses Conversations and debates as the context for information dissemination, in the preferred embodiment, the Conversations context template tracks email postings, annotations, and threads for relevant information.
The Conversations context template comprises the following information object types:
1. Email of a thread depth of at least one (An email reply to an email message)
2. Annotations of a thread depth of at least one (The annotation of an annotation of an object)
3. Internet News Postings (A news posting reply to a news posting)
The query will be sorted by thread depth. Additional filters will be applied to the query based on the predicate list in the SQML. In addition, the context skin should display the information items by thread.
Figure 104 is a Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc. (Two People working at a desk)
CONVERSATIONS CONTEXT - SAMPLE OBJECT AND CONTEXT BAR
VISUALIZATIONS
Below is a list of considerations for, or characteristics of visualization elements semantically appropriate to the corresponding indicated context (in parentheses).
1. Animated graphic of semantic imageΛnotion(s) (icon and context guide view) 2. Maximum thread depth over plain background (icon and context guide view)
3. Maximum thread depth over semantic image/motion (icon and context guide view)
4. Titles of conversations animated in a sequence (list view)
5. Titles and details of conversations animated in a sequence (tiled view)
6. The number of conversations over a plain background (icon and context guide view)
7. The number of conversations over semantic image/motion(s) (icon and context guide view)
Newsmakers Context Template
The Newsmakers context template can be analogized to a personal, digital version of NBC's "Meet the Press" program in how it conveys semantic information. In this case, the emphasis is on "people in the news," as opposed to the news itself or Conversations. Users navigate the network using the returned people as Information Object Pivots. The Newsmakers context template can be thought of as the Headlines context template, preferably with the "People" or "Users" object type filters, and the "authored by," "possibly authored by," "hosted by," "annotated by," "expert on," etc. predicates (predicates that relate people to information). The "relevant to" default predicate preferably is used to cover all the germane specific predicates. The sort order of the relevant information, e.g., the newsmakers, is sorted based on the order of the "news they make," e.g., headlines.
The query will be sorted by number of headlines. Additional filters will be applied to the query based on the predicate list in the SQML.
Figure 105 illustrates a semantic "Newsmaker" Visualization or Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc. (Football Championship) NEWSMAKERS - SAMPLE OBJECT AND CONTEXT BAR VISUALIZATIONS
1. Animated graphic of 2 talking heads in conversation (icon and context guide view)
2. Animated graphic of semantic image/motion(s) (icon and context guide view)
3. Total number of newsmakers (icon and context guide view)
4. Total number of newsmakers over semantic image/motion (icon and context guide view)
5. Names of newsmakers animated in a sequence (list view)
6. Names and details of newsmakers animated in a sequence (tiled view) Upcoming Events Context Template
The Upcoming Events context template (and its resulting Special Agent) can be analogized to a personal digital version of special programs that convey information about upcoming events. Examples include specials for events such as "The World Series," "The NBA Finals," "The Soccer World Cup Finals," etc. The equivalent in a knowledge-worker scenario is a user that wants to monitor all upcoming industry events that relate to one or more categories, documents or other Information Object Pivots. The Upcoming Events context template is preferably identical to the Headlines context template except that only upcoming events are filtered and displayed (preferably using a semantically appropriate "context Skin" that connotes events and time criticality). Returned objects are preferably sorted based on time criticality with the most impending events listed first.
Figure 106 illustrates a semantic "Upcoming Events" Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc. (Appointment Binder).
UPCOMING EVENTS - SAMPLE OBJECT AND CONTEXT BAR VISUALIZATIONS
1. Ticking clock showing time till next event over a background of the total number of upcoming events (icon and context guide view)
2. Ticking clock showing time till next event over semantic image/motion(s) (icon and context guide view)
3. Ticking clock showing time till next event over semantic image/motion(s) and the total number of upcoming events (icon and context guide view)
4. Ticking clock showing time till next event over a plain background (icon and context guide view)
5. Non-ticking clocks showing time till all upcoming events (sequentially) over various backgrounds (icon and context guide view)
6. Calendar view showing scheduled time of next upcoming event over various backgrounds (icon and context guide view)
7. Calendar view showing scheduled time of all upcoming events (sequentially) over various backgrounds (icon and context guide view)
8. Animated graphic showing calendar motion (icon and context guide view)
9. Animated graphic of semantic image/motion(s) (e.g., schedule book) (icon and context guide view)
10. The total number of upcoming events over semantic image/motion(s) (icon and context guide view) 11. The total number of upcoming events over a plain background (icon and context guide view)
12. Titles of upcoming events animated in a sequence (list view)
13. Titles and details of upcoming events animated in a sequence (tiled view)
Discovery
The Discovery context template can be analogized to a personal, digital version of the "Discovery Channel." In this case, the emphasis is on "documentaries" about particular topics. The Discovery context template simulates intelligent aggregation of information by randomly selecting information objects that relate to a given set of categories and which are posted within an optionally predetermined, configurable time period. The semantic weight as opposed to the time is the preferred consideration for determining how the information is to be ordered or presented. The context template can be implemented by filtering all information types by the semantic link strength for the categorization predicate. In this case, the filter should be less selective than the 'Best Bets' filter - the context template lies somewhere between 'Best Bets' and 'All Items' in terms of filtering.
Figure 107 is a "Discovery" Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc. (Petri Dish).
DISCOVERY - SAMPLE OBJECT AND CONTEXT BAR VISUALIZATIONS
1. Animated graphic of semantic image/motion(s) (e.g., a telescope, a voyager spacecraft, an old ship at sea) (icon and context guide view)
2. Titles of the first N information items in a sequential animation (list view)
3. Titles and details of the first N information items in a sequential animation (tiled view)
4. The total number of items over semantic image/motion(s) (icon and context guide view)
5. The total number of items (icon and context guide view)
History
The History context template can be analogized to a personal, digital version of the "History Channel." hi this case, the emphasis is on disseminating information not just about particular topics, but also with a historical context. For this template, the preferred axes are category and time. The History context template is similar to the Discovery context template, further in concert with "a minimum age limit." The parameters are preferably the same as that of the Discovery context template, except that the "maximum age limit" parameter is replaced with a "minimum age limit" parameter (or an optional "history time span" parameter), hi addition, returned objects are preferably sorted in reverse or random order based on their age in the system or their age since creation.
Figure 108 illustrates a semantic "History" Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc. (War Memorial).
HISTORY - SAMPLE OBJECT AND CONTEXT BAR ANIMATIONS
VISUALIZATIONS
1. Animated graphic of semantic image/motion(s) or an equivalent
2. Titles of the oldest (or random) N information items in a sequential animation (list view)
3. Titles and details of the oldest (or random) N information items in a sequential animation (tiled view)
4. Total number of items over semantic image/motion(s) (icon and context guide view)
5. Total number of items over plain background (icon and context guide view)
All Items
The All Items context template represents context that returns any information that is relevant based on either semantics or based on a keyword or text based search. In this case, the emphasis is on disseminating information that may be even remotely relevant to the context. The primary axis for the All Items context template is preferably the mere possibility of relevance. In the preferred embodiment, the All Items context template employs both a semantic and text- based query in order to return the broadest possible set or universe of results that may be relevant.
Figure 109 illustrates a semantic Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc. (Outer Space).
ALL ITEMS - VISUALIZATION & SAMPLE OBJECT AND CONTEXT BAR
ANIMATIONS
1. Animated graphic of semantic image/motion(s) or an equivalent 2. Titles of the most recent J information items in a sequential animation (list view)
3. Titles and details of the most recent N information items in a sequential animation (tiled view)
4. Total number of items over semantic image/motion(s) (icon and context guide view)
5. Total number of items over plain background (icon and context guide view)
Best Bets
The Best Bets context template (and its resulting Special Agent) represents context that returns only highly relevant information. In a preferred embodiment, the emphasis is on disseminating information that is deemed to be highly relevant and semantically significant. For this context template, the primary axis is relevance. In essence, the Best Bets context template employs a semantic query and will not use text based queries since it cannot guarantee the relevance of text-based query results. The Best Bets context template is preferably initialized with a category filter or keywords. If keywords are specified, the server performs categorization dynamically. Results are preferably sorted based on the relevance score, or the strength of the "belongs to category" semantic link from the object to the category filter.
Figure 110 illustrates a "Best Bets" Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc. (Microscope).
BEST BET VISUALIZATION - SAMPLE OBJECT AND CONTEXT BAR
ANIMATIONS
1. Animated graphic of semantic image/motion(s) or an equivalent
2. Titles of the most recent N information items in a sequential animation (list view)
3. Titles and details of the most recent N information items in a sequential animation (tiled view)
4. Total number of items over semantic image/motion(s) (icon and context guide view)
5. Total number of items over plain background (icon and context guide view)
FAVORITES
The Favorites context template (and its resulting Special Agent) represents context that returns "favorite" or "popular" information. In this case, the emphasis is on disseminating information that has been endorsed by others and has been favorably accepted. In the preferred embodiment, the axes for the Favorites context template include the level of readership interest, the "reviews" the object received, and the depth of the annotation thread on the object. In one embodiment, the Favorites context template returns only information that has the "favorites" semantic link, and is sorted by counting the number of "votes" for the object (based on this semantic link).
Figure 111 illustrates a semantic Visualization- Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc. (coffee and pastry).
FAVORITES VISUALIZATION - SAMPLE OBJECT AND CONTEXT BAR
ANIMATIONS
1. Animated graphic of semantic image/motion(s) or an equivalent
2. Titles of the most recent N information items in a sequential animation (list view)
3. Titles and details of the most recent N infoπnation items in a sequential animation (tiled view)
4. Total number of items over semantic image/motion(s) (icon and context guide view)
5. Total number of items over plain background (icon and context guide view)
CLASSICS
The Classics context template (and its resulting Special Agent) represents context that returns "classical" information, or information that is of recognized value. Like the Favorites context template, the emphasis is on disseminating information that has been endorsed by others and has been favorably accepted. For this context template, the preferred axes include a historical context, the level of readership interest, the "reviews" the object received and the depth of the annotation thread on the object. The Classics context template is preferably implemented based on the Favorites context template but with an additional minimum age limit filter and voting score, essentially functioning as an "Old Favorites" context template.
Figure 112 illustrates a semantically appropriate Sample Image for "Classics" for smart hourglass, interstitial page, transition effects, background chrome, etc. (Car)
CLASSICS VISUALIZATIONS - SAMPLE OBJECT AND CONTEXT BAR
ANIMATIONS
1. Animated graphic of semantic image/motion(s) or an equivalent
2. Titles of the most recent N information items in a sequential animation (list view) 3. Titles and details of the most recent N information items in a sequential animation (tiled view)
4. Total number of items over semantic image/motion(s) (icon and context guide view)
5. Total number of items over plain background (icon and context guide view)
RECOMMENDATIONS
The Recommendations context template represents context that returns "recommended" information, or information that the Agencies have inferred would be of interest to a user. Recommendations will be inserted by adding "recommendation" semantic links to the "SemanticLinks" table and by mining the favorite semantic links that users indicate. Recommendations are preferably made using techniques such as machine learning and collaborative filtering. The emphasis of this context template is on disseminating information that would likely be of interest to the user but which the user might not have already seen. For this context template, the primary axes preferably include the likelihood of interest and freshness.
Figure 113 illustrates a semantically appropriate "Recommendation" Visualization - Sample Image for the contextual/application elements of smart hourglass, interstitial page, transition effects, background chrome, etc. (Thumbs up).
RECOMMENDATION VISUALIZATION - SAMPLE OBJECT AND CONTEXT BAR
ANIMATIONS
1. Animated graphic of semantic image/motion(s) or an equivalent
2. Titles of the most recent N information items in a sequential animation (list view)
3. Titles and details of the most recent N information items in a sequential animation (tiled view)
4. Total number of items over semantic image/motion(s) (icon and context guide view)
5. Total number of items over plain background (icon and context guide view)
TODAY
The Today context template represents context that returns information posted or holding (in the case of events) "today." The emphasis with this context template is preferably on disseminating information that is deemed to be current based on "today" being the filter to determine freshness.
Figure 114 illustrates a semantic "Today" Visualization - Sample Image for the elements smart hourglass, interstitial page, transition effects, background chrome, etc.
"TODAY VISUALIZATION" - SAMPLE OBJECT AND CONTEXT BAR
ANIMATIONS
1. Animated graphic of semantic image/motion(s) or an equivalent
2. Titles of the most recent N information items in a sequential animation (list view)
3. Titles and details of the most recent N information items in a sequential animation (tiled view)
4. Total number of items over semantic image/motion(s) (icon and context guide view)
5. Total number of items over plain background (icon and context guide view) ANNOTATED ITEMS
The Annotated Items context template represents context that returns annotated information. The emphasis with this context template is on disseminating information that is likely to be important based on the fact that one or more users have annotated the items.
Figure 115 illustrates a semantic "Annotated Items" Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc..
"ANNOTATED ITEMS55 VISUALIZATION - SAMPLE OBJECT AND CONTEXT BAR
ANIMATIONS
1. Animated graphic of semantic image/motion(s) or an equivalent
2. Titles of the most recent N information items in a sequential animation (list view)
3. Titles and details of the most recent N information items in a sequential animation (tiled view)
4. Total number of items over semantic image/motion(s) (icon and context guide view)
5. Total number of items over plain background (icon and context guide view) ANNOTATIONS
The Annotations context template represents context that returns annotated information. The emphasis with this context template is on disseminating information that are annotations.
Figure 116 illustrates a semantic Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc. (Note pinned to Bulletin Board)
"ANNOTATIONS" VISUALIZATION - SAMPLE OBJECT AND CONTEXT BAR
ANIMATIONS
1. Animated graphic of semantic image/motion(s) or an equivalent
2. Titles of the most recent N information items in a sequential animation (list view)
3. Titles and details of the most recent N information items in a sequential animation (tiled view)
4. Total number of items over semantic image/motion(s) (icon and context guide view)
5. Total number of items over plain background (icon and context guide view)
EXPERTS
Figure 117 illustrates a semantic "Experts" Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc. (Professor)
"EXPERTS" VISUALIZATION - SAMPLE OBJECT AND CONTEXT BAR
ANIMATIONS
1. Animated graphic of semantic image/rnotion(s) or an equivalent
2. Names of the most recent N experts in a sequential animation (list view)
3. Names and details of the most recent N experts in a sequential animation (tiled view)
4. Total number of experts over semantic image/motion(s) (icon and context guide view)
5. Total number of experts over plain background (icon and context guide view)
PLACES
Figure 118 illustrates a semantic "Places" Visualization - Sample hnage for smart hourglass, interstitial page, transition effects, background chrome, etc. (Paris)
"PLACES" VISUALIZATION - SAMPLE OBJECT AND CONTEXT BAR
ANIMATIONS
1. Animated graphic of semantic image/motion(s) or an equivalent
2. Names of the most recent N places in a sequential animation (list view) 3. Names and details of the most recent N places in a sequential animation (tiled view)
4. Total number of places over semantic image/motion(s) (icon and context guide view)
6. Total number of places over plain background (icon and context guide view)
BLENDERS
Figure 119 illustrates a semantic "Blenders" Visualization — Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc. (Blenders)
"BLENDERS" VISUALIZATION - SAMPLE ICONIC ANIMATIONS
1. Animated graphic of semantic image/motion(s) or an equivalent
2. Animated graphic of blender or mixer in action
3. Titles of the blender items in a sequential animation (list view)
4. Titles and details of the blender items in a sequential animation (tiled view)
5. Total number of items over semantic image/motion(s) (icon and context guide view)
6. Total number of items over plain background (icon and context guide view)
INFORMATION OBJECT TYPES
Figures 120 through 138 illustrate semantic Visualizations for the following Information
Object Types, respectively: Documents, Books, Magazines, Presentations, Resumes,
Spreadsheets, Text, Web pages, White Papers, Email, Email Annotations, Email Distribution
Lists, Events, Meetings, Multimedia, Online Courses, People, Customers, and Users.
PRESENTATION SKIN TYPES
TIMELINE
Figure 139 illustrates a semantic "Timeline" Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc..
"TIMELINE" VISUALIZATION - SAMPLE OBJECT AND CONTEXT BAR
ANIMATIONS
1. Calendar view showing effective time (publication time, scheduled time, etc.) of information item over various backgrounds (icon and context guide view)
2. Calendar view showing effective time of all information items (sequentially) over various backgrounds (icon and context guide view)
3. Animated graphic showing calendar motion (icon and context guide view) 4. Animated graphic of semantic ιmage/motion(s) (e.g., time warp image/motion) (icon and context guide view)
5. The total number of information items over semantic image/motion(s) (icon and context guide view)
6. The total number of information items over a plain background (icon and context guide view)
7. Titles of information items animated in a sequence (list view)
8. Titles and details of information items animated in a sequence (tiled view)
9. Scrolling, linear timeline control with items populated based on effective date/time
10. Animated timeline ticker control sorted by effective date/time
The Power of Semantic Visualizations.
One final note concerning Visualizations. The preferred embodiment not only searches for information semantically, and not only organizes and stores it semantically, it also presents it semantically. And, the presentation is not semantic only in the sequence, organization and relationships of the information, but also visually, as the foregoing Visualizations are, in part, intended to convey. As a result, the user is aided in understanding the information being presented by the system in roughly in the same way that a viewer of a movie is aided in understanding the meaning of dialogue by the surrounding context of the lighting, costume, music and entire set or scene. Put differently, the Visualizations, as with everything else presented or managed by, or located with, the preferred embodiment system, serve the purpose of conveying meaningful information; or, just as aptly, to convey information meaningfully. Meaning is a unifying theme of the preferred embodiment; it permeates the design and operation of the system, and each constituent component part of which the system is comprised.
While the preferred and some alternate embodiments of the invention have been illustrated and described, as noted above, many changes can be made without departing from the spirit and scope of the invention. Accordingly, the scope of the invention is not limited by the disclosure of the preferred embodiment. Instead, the invention should be determined entirely by reference to the claims that follow. Appendix A
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL, MANAGEMENT, DELIVERY AND PRESENTATION INVENTOR
Nosa Omoigui
PRIORITY CLAIM
This application claims priority from earlier filed U.S. Provisional Patent Application Serial No. 60/300,385 filed June 22, 2001 and U.S. Provisional Patent Application Serial No. 60/360,610 filed February 28, 2002.
COPYRIGHT NOTICE
This disclosure is protected under United States and International Copyright Laws. © 2002 Nosa Omoigui. All Rights Reserved. A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
FIELD OF THE INVENTION This invention relates generally to information management systems and, more specifically, to an integrated and seamless implementation framework and resulting medium for knowledge retrieval, management, delivery and presentation. BACKGROUND OF THE INVENTION
Knowledge is now widely recognized as a core asset for organizations around the world, and as a tool for competitive advantage. In today's connected, information-based world, knowledge-workers must have access to the knowledge and the tools they need to make better, faster, and more-informed decisions to improve their productivity, enhance customer relationships, and to make their businesses more competitive. In addition, industry observers have touted "agility" and the "real-time enterprise" as important business goals to have in the information economy.
Many organizations have begun to realize the value of disseminating knowledge within their organizations in order to improve products and customer service, and the value of having a well-trained workforce. The investments businesses are making in e-Learning and corporate training provides some evidence of this. Companies have also invested in tools for content management, search, collaboration, and business intelligence. Companies are also spending significant resources on digitizing their business processes, particularly with respect to acquiring and retaining customers.
However, many knowledge/learning and customer-relationship assets are still stored in a diverse set of repositories that do not understand each other's language, and as a result are managed and interacted with as independent islands of infoπnation. As such, what many organizations call "knowledge" is merely data and infonnation. The information economy in large part is a struggle to find a way to provide context, meaning and efficient access to this ever increasing body of data and infoπnation. Or, stated differently, to turn the mass of available data and infonnation into usable knowledge.
Information has been long accessible in a variety of foπns, such as in newspapers, books, radio and television media, and in electronic form, with varying degrees of proliferation. Information management and access changed dramatically with the use of computers and computer networks. Networked computer systems provide access throughout the system to information maintained at any point along the system. Users need only establish the requisite connection to the network, provide proper authorization and identify . the desired infonnation to obtain access.
Infoπnation access further improved with the advent of the Internet, which connects a large number of computers across diverse geography to provide access to a vast body of infoπnation. The most wide spread method of providing infoπnation over the Internet is via the World Wide Web. The Web consists of a subset of the computers or Web servers connected to the Internet that typically run Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), GOPHER or other servers. Web servers host Web pages at Web sites. Web pages are encoded using one or more languages, such as the original Hypertext Markup Language (HTML) or the more current extensible Markup Language (XML) or the Standard Generic Markup Language (SGML). The published specifications for these languages are incorporated by reference herein. Web pages in these formatting languages may be accessed by Internet users via web browsing software such as Microsoft's Internet Explorer or Netscape's Navigator.
The Web has largely been organized based on syntax and structure, rather than context and semantics. As a result, infoπnation is typically accessed via search engines and Web directories. Current search engines use keyword and corresponding search techniques that rely on textual or basic subject matter infoπnation and indices without associated context and semantic information. Unfortunately, such searching methods produce thousands of largely unresponsive results; documents as opposed to actionable knowledge. Advanced searching techniques have been developed to focus queries and improve the relevance of search results. Many such techniques rely on historical user search trends to make basic assumptions as to desired information. Alternatively, other search techniques rely on categorization of Web sites to further focus the search results to areas anticipated to be most relevant. Regardless of the search technique, the underlying organization of searchable infoπnation is index-driven rather than context-driven. The frequency or type of textual infonnation associated the document determines the search results, as opposed to the attributes of the subject matter of the document and how those attributes relate to the user's context. The result is continued ambiguity and inefficiency surrounding the use of the Web as a tool for acquiring actionable knowledge.
In enterprises around the world today, the Web is the information platfoπn for knowledge-workers. And there lies the problem. The Web as we know it is a platfoπn for data and infoπnation while its users operate at the level of "knowledge." This disconnect is a very fundamental one and cannot be understated. The Web, in large measure, has fulfilled the dream of "infoπnation at your fingertips." However, knowledge-workers demand "knowledge at your fingertips" as opposed to mere "infoπnation at your fingertips." Unfortunately, today's knowledge-workers use the Web to browse and search for documents — compilations of data and infoπnation — rather than actual knowledge relevant to their inquiry. To achieve improved knowledge requires providing proper context, meaning and efficient access to data and infoπnation, all of which are missing with the traditional Web.
Efforts have been made to achieve the goal of "knowledge at your fingertips." One example is a new concept for information organization and distribution referred to as the Semantic Web. The Semantic Web is an extension of the current Web in which infoπnation is given well-defined meaning, better enabling computers and people to work in cooperation. While conceptually a significant step forward in supporting improved context, meaning and access of information on the Internet, the Semantic Web has yet to find successful implementation that lives up to its stated potential.
Both the current Web and the Semantic Web fail to provide proper context, meaning and efficient access to data and infoπnation to allow users to acquire actionable knowledge. This is partially a problem related to the ways in which Today's Web and the contemplated Semantic Web are structured or, in other words, related to their technology layers. As shown in FIGURE 1, Today's Web, for example, which is a hypertext medium, provides the three technology layers, which include "dumb" links, or links having no context-sensitivity, time-sensitivity, etc. Present conceptualizations of the Semantic Web, also referred to as a "semantic hypeπnedia," provide for five technology layers, as shown in FIGURE 2. As explained in greater detail below, there are serious limitations associated with each of the technology layer structures.
In addition, various properties must be present in a comprehensive information management system to provide an integrated and seamless implementation framework and resulting medium for knowledge retrieval, management and delivery. A non-exhaustive list of these properties include: Semantics/Meaning; Context-Sensitivity; Time-Sensitivity; Automatic and intelligent Discoverability; Dynamic Linking; User-Controlled Navigation and Browsing; Non-HTML and Local Document Participation in the Network; Flexible Presentation that Smartly Conveys the Semantics of the Information being Displayed; Logic, Inference, and Reasoning; Flexible User-Driven Information Analysis; Flexible Semantic Queries; Read/Write Support; Annotations; "Web of Trust"; Information Packages ("Blenders"); Context Templates, and User-Oriented Information Aggregation. Each of these, properties will be discussed below in the context of their application to both Today's Web and the Semantic Web. SEMANTICS/MEANING
Today's Web lacks semantics as an intrinsic part of the platform and user experience. Web pages convey only textual and graphical data rather than the semantics of the data they contain. As a result, users cannot issue semantic queries such as those that one might expect with natural language — for example, "find me all books less than hundred pages long, about Latin Jazz, and published in the last five years." To be able to process such a query, a Web site or search engine must "know" it contains books and must be able to intelligently filter its contents based on the semantics of the query request. Such a query is not possible on the
Web today. Instead, users are forced to rely on text-based searches. These searches usually result in information overload or information loss because the user is forced to pick search terms that might not match the text in the information base. In the aforementioned example, a user might pick the search teπn "Books Latin Jazz" and hope that the search engine can make the connection. The user is usually then left to independently filter the search results. This sort of text-based search also implies that teπns that might convey the same meaning. In the above example, results from search teπns such as "Books on South or Central American Jazz" or "Publications on Jazz from Latino Lands" might be ignored during the processing of the search query.
The lack of semantics also implies that Today's Web does not allow users to navigate based on they way humans think. For example, one might want to navigate a corporate intranet using the organizational structure. For example, from people to the documents they create to the experts on that documents to the direct reports of those experts to the distribution lists the direct reports are members of to the members of the distribution lists to the documents those members created, etc. This "web" is semantic and is based on actual information classification ("things") and not just "pages" as Today's Web is.
The lack of semantics also has other implications. First, it means that the Web is not programmable. With semantics, the Web can be consumed by Smart Agents that can make sense of the pages and the links and then make inferences, recommendations, etc. With Today's Web, the only "Agent" that can make inferences is the human brain. As such, the Web does not employ the enormous processing power that computers are capable of — because it is not represented in a way that computers can understand.
The lack of semantics also implies that information is not actionable. A search engine does not "understand" the results it spits out. As such, once a user receives search results, he or she is "on his or her own." Also, a web browser does not "understand" the information it is displaying and as such cannot do smart things with the information. With semantics in place, a smart display, for example, will "know" that an event is an event and might do interesting things like check if the event is already in the user's calendar, display free/busy infoπnation, or allow the user to automatically insert the event into his/her calendar thereby making the infoπnation actionable. Infoπnation presented without semantics is not actionable or might require that the semantics be inferred, which might result in an unpleasant user experience.
The Semantic Web seeks to address semantics/meaning limitations with Today's Web by encoding information with well-defined semantics. Web pages on the Semantic Web include metadata and semantic links to other metadata, thereby allowing search engines to perfoπn more intelligent and accurate searches. In addition, the Semantic Web includes ontologies that will be employed for knowledge representation, thereby allowing a semantic search engine to interpret terms based on meaning and not merely on text. For example, in the previous example, Latin Jazz ontology might be employed on a Semantic Web site and would allow a search engine on the site to "know" that the terms "Books on South or Central American Jazz" or "Publications on Jazz from Latino Lands" have the same meaning as the term "Books on Latin Jazz." While conceptually overcoming many of the deficiencies with Today's Web, there has not to date been a successful implementation of a well-defined data model providing context and meaning, including in particular the necessary semantic links, ontologies, etc. to provide for additional characteristics such as context-sensitivity and time-sensitivity.
CONTEXT-SENSITIVITY
Today's Web lacks context-sensitivity. The implication of a lack of context is that
Today's Web is not personal. For example, documents in accessible storage are independently static and therefore stupid. Information relevant to the subject matter of the document has already been published, is being newly published, or will soon be published.
Because the document in storage is static, however, there is no way to dynamically associate its subject matter with this relevant infonnation in real-time. Stated differently, users have no way to dynamically connect their private context with external infonnation in real-time. Infoπnation sources (such as the document) that foπn context sit in their own islands, totally isolated from other relevant infonnation sources. This results in information and productivity losses.
The primary reason for this is that Today's Web is a presentation-oriented medium designed to present views of information to a dumb client (e.g., remote computer). The client has virtually no role to play in the user experience, aside from merely displaying what the server tells it to display. Even in cases where there is client-side code (like Java applets and ActiveX controls), the controls usually do one specific thing and do not have coordinated action with the remote server such that code on the client is being orchestrated with code on the server.
From a productivity standpoint, the implication of this is that knowledge-workers and information consumers are totally at the mercy of information authors. Today, knowledge-workers have portals that are maintained and updated to provide custom views of corporate infoπnation, external data, etc. However, this is still very limiting because knowledge-workers are completely helpless if nothing dynamically and intelligently connects relevant information in the context of their task with information that users have access to.
If a knowledge-worker does not see a link to a relevant piece of information on his of her portal, of if a friend or colleague does not email him or her the link, the information gets dropped; information does not connect with or adapt to the user context or the context in which it is displayed. Likewise, it is not enough to just notify a user that new data for an entire portal is available and shove it down to their local hard drive. It lacks a customizable presentation with context sensitive alert notifications. '' The Semantic Web suffers from the same limitations as Today's Web when it comes to context-sensitivity. On the Semantic Web, users are likewise at the mercy of infoπnation authors. The Semantic Web itself will be authored, but the authoring will include semantics. As a result, users are still largely on their own to locate and evaluate the relevance of available infoπnation. The Semantic Web, as a standalone entity, will not be able to make these dynamic connections with other infoπnation sources. TIME-SENSITIVITY
Today's Web lacks time-sensitivity. The Web platform (e.g., browser) is a dumb piece of software that merely presents infoπnation, without any regard to the time-sensitivity of the information. The user is left to infer time sensitivity or do without it. This results in a huge loss in productivity because the Web platform cannot make time-sensitive connections in real-time. While some Web sites focus on presenting time-sensitive information, for example, by indexing information past a predeteπnined date, the Web browser itself has no notion of time-sensitivity. Instead, it is left to individual Web sites to include time-sensitivity in the information they display in their own island. In other words, there is no axis of time on a Web link. The Semantic Web, like Today's Web, also does not address time-sensitivity.
A Semantic Web can have semantic links that do not internalize time. This is largely because the Semantic Web implicitly has no notion of software Web services that address context and time-sensitivity. AUTOMATIC AND INTELLIGENT DISCOVERABILITY Today's Web lacks automatic and intelligent discoverability of newly created infoπnation. There is currently no way to know what Web sites started anew today or yesterday. Unless the user is notified or the user serendipitously discovers a new site when he or she does a search, he or she might not have any clue as to whether there are any new Web sites or pages. The same problem exists in enterprises. On Intranets, knowledge-workers have no way of knowing when new Web sites come up unless informed via some external means. The Web platfoπn itself has no notion of announcements or discovery. In addition, there is no context-sensitive discovery to determine new sites or pages within the context of the user's task or current information space.
The Semantic Web, like Today's Web, does not address the lack of automatic discoverability. Semantic Web sites suffer from the same problem — users either will have to find out about the existence of new infoπnation sources from external sources or ''through personal discovery when they perfoπn a search. DYNAMIC LINKING
Today's Web employs a pure network or graph "data structure" for its information model. Each Web page represents a node in the network and each page can contain links to other nodes in the network. Each link is manually authored into each page. This has several problems. First, it means that the network needs to be maintained for it to have continuous value. If Web pages are not updated or if Web page or site authors do not have the discipline to add links to their pages based on relevance, the network loses value. Today's Web is essentially prone to having dead links, old links, etc. Another problem with a pure network or graph infoπnation model is that the infoπnation consumer is at the mercy of — rather than in control of — the presentation of the Web page or site. In other words, if a Web page or site does not contain any links, the user has no recourse to find relevant information. Search engines are of little help because they merely return pages or nodes into the network. The network itself does not have any independent or dynamic linking ability. Thus, a search engine can easily return links to Web pages that themselves have no links or dead, stale or irrelevant links. Once users obtain search results, they are on their own and are completely at the mercy of whether the author of the returned pages inserted relevant, time-sensitive links into the page.
The Semantic Web suffers from the same problem as Today's Web because the Semantic Web is merely Today's Web plus semantics. Even though users will be able to navigate the network semantically (which they cannot currently do with the Web), they will still be at the mercy of how the information has been authored. In other words, the Semantic Web is also dependent on the discipline of the authors and hence suffers from the same aforementioned problems of Today's Web. If the Semantic Web includes pages with ontologies and metadata, but those pages are not well maintained or do not include links to other relevant sources, the user will still be unable to obtain current links and other infoπnation. The Semantic Web, as currently contemplated, will not be a smart, dynamic, self-authoring, self-healing network.
USER-CONTROLLED NAVIGATION AND BROWSING
With Today's Web, the user has no control over the navigation and browsing experience, but rather is completely at the mercy of a Web page and how it is authored with links (if any). As shown with reference to prior art FIGURE 3, Today's Web consists of "dumb links," or statically authored generic links that are wholly dependent on continuous maintenance to be navigable.
The Semantic Web suffers from a similar problem as Today's Web in that there is no user-controlled browsing. Instead, as shown with reference to prior art FIGURE 4, the Semantic Web consists of "dumb links," further including semantic information and metadata. However, the Semantic Web links remain equally dependent on continuous maintenance to be navigable. NON-HTML AND LOCAL DOCUMENT PARTICIPATION IN THE NETWORK Another problem with Today's Web is the requirement that only documents that are authored as HTML can participate in the Web, in addition to the fact that those documents have to contain links. The implication is that other infoπnation objects like non-HTML documents (e.g., PDF, Microsoft Word, PowerPoint, and Excel documents, etc.) — especially those on users" hard drives — are excluded from the benefits of linking to other objects in the network. This is very limiting, especially since there might be semantic relevance between information objects that are not HTML and which do not contain links. Furtheπnore, search engines do not return results for the entire universe of information since vast amount of content available on the web is inaccessible to standard web crawlers. This includes, for example, content stored in databases, unindexed file repositories, subscription sites, local machines and devices, proprietary file foπnats (such as Microsoft Office documents and email), and non-text multimedia files. These form a vast constellation of inaccessible matter on the Internet, referred to as "the invisible Intranet" inside corporations. Today's Web servers do not provide web crawler tools that address this problem.
The Semantic Web also suffers from this limitation. It does not address the millions of non-HTML documents that are already out there, especially those on users" hard drives. The implication is that documents that do not have RDF metadata equivalents or proxies cannot be dynamically linked to the network.
FLEXIBLE PRESENTATION THAT SMARTLY CONVEYS THE SEMANTICS OF THE INFORMATION BEING DISPLAYED Today's Web does not allow users to customize or "skin" a Web site or page. This is because Today's Web servers return information that is already formatted for presentation by the browser. The end user has no flexibility in choosing the best means of displaying the information — based on different criteria (e.g., the type of information, the available amount of real estate, etc.) The Semantic Web does not address the issue of flexible presentation. While a semantic Web site conceptually employs RDF and ontologies, it still sends HTML to the browser. Essentially, the Semantic Web does not provide for specific user empowennent for presentation. As such, a Semantic Web site, viewed by Today's Web platform, will still not empower the user with flexible presentation. Moreover, despite industry movement towards XML,, only a new platform can dictate that data will be separated from presentation and define guidelines for making the data programmable. Authors building content for the Semantic Web either return XML and avoid issues with presentation entirely, or focus their efforts on a single presentation style (vertical industry scenario) for rendering. Neither approach allows the Semantic Web to achieve an optimum degree of knowledge distribution. LOGIC, INFERENCE AND REASONING Because Today's Web does not have any semantics, metadata, or knowledge representation, computers cannot process Web pages using logic and inference to infer new links, issue notifications, etc. Today's Web was designed and built for human consumption, not for computer consumption. As such, Today's Web cannot operate on the infoπnation fabric without resorting to brittle, unreliable techniques such as screen scraping to try to extract metadata and apply logic and inference.
While the Semantic Web conceptually uses metadata and meaning to provide Web pages and sites with encoded information that can be processed by computers, there is no current implementation that is able to successfully achieve this computer processing and which illustrates new or improved scenarios that benefit the infoπnation consumer or producer.
FLEXIBLE USER-DRIVEN INFORMATION ANALYSIS
Today's Web lacks user-driven information analysis. Today's Web does not allow users to display different "views" of the links, using different filters and conditions. For example, Web search engines do not allow users to test the results of searches under different scenarios. Users cannot view results using different pivots such as information type (e.g., documents, email, etc.), context (e.g., "Headlines," "Best Bets," etc.), category (e.g., "wireless," "technology," etc.) etc.
While providing a greater degree of flexible information analysis, the Semantic Web does not describe how the presentation layer can interact with the Web itself in an interactive fashion to provide flexible analysis. FLEXIBLE SEMANTIC QUERIES
Today's Web only allows text-based queries or queries that are tied to the schema of a particular Web site. These queries lack flexibility. Today's Web does not allow a user to issue queries that approximate natural language or incorporate semantics and local context. For example, a query such as "Find me all email messages written by my boss or anyone in research and which relate to this specification on my hard disk" is not possible with Today's Web. . . . - .
By employing metadata and ontologies, the conceptual Semantic Web allows a user to issue more flexible queries than Today's Web. For example, users will be able to issue a query such as "Find me all email messages written by my boss or anyone in research." However, users will not be able to incorporate local context. In addition, the Semantic Web does not define an easy manner with which users will query the Web without using natural language. Natural language technology is an option but is far from being a reliable technology. As such, a query user interface that approximates natural language yet does not rely on natural language is required. The Semantic Web does not address this.
READ/WRITE SUPPORT
Today's Web is a read-only Web. For example, if users encounter a dead link
(e.g., via the "404" error), they cannot "fix" the link by pointing it to an updated target that might be known to the user. This can be limiting, especially in cases where users might have important knowledge to be shared with others and where users might want to have input as to how the network should be represented and evolve.
While the Semantic Web conceptually allows for read/write scenarios as provided by independent participating applications, there is no current implementation that provides this ability. ANNOTATIONS
Today's Web has no implicit support for annotations. And while some specific Web sites support annotations, they do so in a very restricted and self-contained way. Today's
Web medium itself does not address annotations. In other words, it is not possible for users to annotate any link with their comments or additional infoπnation that they have access to.
This results in potential infonnation loss.
While the Semantic Web conceptually allows for annotations to be built into the system subject to security constraints, there is no current implementation that provides this ability. "WEB OF TRUST"
Today's Web lacks seamless integration of authentication, access control, and authorization into the Web, or what has been referred to as a "Web of Trust." With a Web of
Trust, for example, users are able to make assertions, fix and update links to the Web and have access control restrictions built in for such operations. On Today's Web, this lack of trust also means that Web services remain independent islands that must implement a proprietary user subscription authorization, access control or payment system. Grand schemes for centralizing this information on 3rd party servers meet with consumer and vendor distrust because of privacy concerns. To gain access to rich content, asset users must log in individually and provide identity information at each site. While the Semantic Web conceptually allows for a Web of Trast, there is no current implementation that provides for this ability.
INFORMATION PACKAGES (BLENDERS)
Neither Today's Web nor the Semantic Web allows users to deal with related semantic information as a whole unit by combining characteristics of potentially divergent semantic information to produce overlapping results (for example, like creating a custom, personal newspaper or TV channel). CONTEXT TEMPLATES
Neither Today's Web nor the Semantic Web allows users to independently create and niap to specific and familiar semantic models for infoπnation access and retrieval.
USER-ORIENTED INFORMATION AGGREGATION Today's Web lacks support for user-oriented infoπnation aggregation. The user can only access one Web site or one search engine at a time, within the context of one browsing session. As such, even if there is context or time-sensitive infoπnation on other information sources that relate to the information that the user is currently viewing, those sources cannot be presented in a holistic fashion in the current context of the user's task. The Semantic Web also suffers from a lack of user-oriented information aggregation.
The medium itself is an extension of Today's Web. As such, users will still access one site or one search engine at a time and will not be able to aggregate information across information repositories in a context or time-sensitive manner.
Given the growing demand for "knowledge at your fingertips" as well as the deficiencies in Today's Web and the conceptual Semantic Web, many of which are noted above, there is a need for a new and comprehensive system and method of knowledge retrieval, management and delivery.
SUMMARY OF THE INVENTION
The present invention is directed in part to an integrated and seamless implementation framework and resulting medium for knowledge retrieval, management, delivery and presentation. The system includes a server comprised of several components that work together to provide context and time-sensitive semantic information retrieval services to clients-operating a presentation platform via a communication medium. The server includes a
;.ι first server component that is responsible for adding and maintaining domain-specific semantic information • or intelligence. The first server component preferably includes structure or methodology directed to providing the following: a Semantic Network, a Semantic Data Gatherer, a Semantic Network Consistency Checker, an Inference Engine, a Semantic Query Processor, a Natural Language Parser, an Email Knowledge Agent and a Knowledge Domain Manager. The server includes a second server component that hosts domain-specific infoπnation that is used to classify and categorize semantic infoπnation. The first and second server components work together and may be physically integrated or separate.
Within the system, all objects or events in a given hierarchy are active Agents semantically related to each other and representing queries (comprised of underlying action code) that return data objects for presentation to the client according to a predeteπnined and customizable theme or "Skin." This system provides various means for the client to customize and "blend" Agents and the underlying related queries to optimize the presentation of the resulting infoπnation.
The end-to-end system architecture of the present invention provides multiple client access means of communication between diverse knowledge information sources via an independent Semantic Web platform or via a traditional Web , portal (e.g., Today's Web access browser) as modified by the present invention providing additional SDK layers that enable programmatic integration with a custom client.
The methodology of the present invention is directed in part to the operational aspects of the entire system, including the retrieval, management, delivery and presentation of knowledge. This preferably includes securing infoπnation from infoπnation sources, semantically linking the information from the information sources, maintaining the semantic attributes of the body of semantically linked information, delivering requested semantic infoπnation based upon user queries and presenting semantic information according to customizable user preferences. Alternative embodiments of the methodology of the present invention are directed to the operation of Agents representing queries that are used with server-side and client-side applications to enable efficient, inferential-based queries producing semantically relevant infoπnation.
BRIEF DESCRIPTION OF THE DRAWINGS
The preferred and alternative embodiments of the present invention are described in detail below with reference to the following drawings.
FIGURE 1 is a table showing the technology layers of Today's Web. FIGURE 2 is a table showing the technology layers of the conceptual Semantic Web. FIGURE 3 is a diagram showing user navigation to links in Today's Web. FIGURE 4 is a diagram showing user navigation to links in the conceptual Semantic Web.
FIGURE 5 is a screenshot showing a sample Infoπnation Agent Results Pane in accordance with the present invention.
FIGURE 6 shows the technology platform stacks of Today's Web and the Information Nervous System of the present invention. FIGURE 7 is a diagram showing an overview of the system of the present invention.
FIGURE 8 is a diagram showing the end-to-end system architecture for the Infonnation Nervous System of the present invention.
FIGURE 9 is a diagram showing the system architecture for the Knowledge Integration Server (KIS) of the Information Nervous System of the present invention. FIGURE 10 is a comparison between the high-level descriptive platfqrm layers of
Today's Web and the equivalents (where applicable) in the Infomiation Nervous System of the present invention.
FIGURE 11 illustrates the preferred embodiment of the Infonnation Nervous System and illustrates the heterogeneous, cross-platform context for the present invention. FIGURES 12-14 show exemplar screenshots of aspects of the Blender Wizard user interface according to a preferred embodiment of the present invention. FIGURE 15 is an exemplar pane of a Breaking News Agent user interface.
FIGURE 16 illustrates a preferred embodiment showing the Open Agent dialog of the present invention.
FIGURES 17-19 illustrate the Tree View of a sample Semantic Environment involving the Open Agent dialog.
FIGURE 20 shows the Agent schema of the prefened embodiment of the present invention.
FIGURE 21 shows the AgentTypelDs of the preferred embodiment of the present invention. FIGURE 22 shows the AgentQueryTypelDs of the preferred embodiment of the present invention.
FIGURE 23 illustrates sample semantic queries that conespond to Agent names showing how server-side Agents are preferably configured on the KIS of the present invention. FIGURE 24 is a diagram showing an overview of the KIS of the present invention.
FIGURE 25 is a diagram showing a sample Semantic Network directed towards an enterprise situation in accordance with the present invention.
FIGURE 26 is a table showing the preferred schema of the Object type in accordance with the present invention. FIGURE 27 shows the SemanticLinks table of the present invention.
FIGURE 28 is a table showing predicate type IDs of the preferred embodiment of the present invention.
FIGURE 29 is a table showing the preferred user object schema made in accordance with the present invention. FIGURE 30 is a table showing MailingAddressTypelDs preferably associated with the User (person) object schema. FIGURE 31 is a table of the preferred category object schema made in accordance with the present invention.
FIGURE 32 is a table of the preferred document object schema made in accordance with the present invention. FIGURE 33 shows the Print Media Type IDs of the preferred embodiment.
FIGURE 34 shows the prefened FORMATTYPEID.
FIGURE 35 shows the preferred email message list object schema made in accordance with the present invention.
FIGURES 36 and 37' are exemplar tables showing the email distribution list and email public folder object schemas, respectively, of a prefened embodiment of the present invention.
FIGURE 38 shows the prefened PublicFolderTypelD of the present invention.
FIGURE 39 shows the prefened event object schema message list object schema made in accordance with the present invention. FIGURE 40 shows the events types of a prefened embodiment of the present invention.
FIGURE 41 shows the prefened media object schema message list object schema made in accordance with the present invention.
FIGURE 42 shows the media types of a prefened embodiment of the present invention.
FIGURES 43-45 illustrate additional samples showing how objects are categorized and utilized in the prefened embodiment of the present invention.
FIGURE 46 is an object graph showing mapping of raw email XML metadata to the Semantic Network according to the present invention; FIGURES 47-53 are exemplar screenshots showing aspects of Agent management by the KIS. FIGURE 54 shows a sample user interface illustrating an information object displayed in the Information Agent Results Pane.
FIGURE 55 shows an example of a balloon popup associated with an hitrinsic Semantic Link showing an email sample according to the present invention. FIGURE 56 shows an example of a balloon popup associated with a Verb user interface according to the present invention.
FIGURE 57 shows an example of a balloon popup associated with a Deep Infoπnation Mode user interface according to the present invention.
FIGURES 58 and 59 are illustrations showing an exemplar Semantic Environment according to the present invention.
FIGURES 60-68 provide exemplar screenshots of an Information Agent according to a prefened embodiment of the present invention.
FIGURES 69-71 provide exemplar balloon popup menus associated with the Smart Lens feature of an Information Agent according to the present invention. FIGURE 72 shows a sample of a variant of the balloon popup menu of FIGURE 71 showing the relatedness measure of the two objects.
FIGURES 73-75 show sample tables illustrating the behaviors and relational contains objects types predicates when using Smart Lenses.
FIGURE 76 is a user interface sample illustrating semantic results Player/Preview Control according to the present invention.
FIGURE 77 is a user interface sample showing the semantic results of a Blender. FIGURES 78 and 79 illustrate exemplar functionality mappings of the present invention.
FIGURE 80 illustrates a user interface showing Agent results and conesponding Context Palettes according to the present invention. FIGURE 81 shows a sample Smart Recommendations popup context Results Pane according to the present invention.
FIGURE 82 is a table showing the technology layers of the Infoπnation Nervous System of the present invention.
FIGURE 83 illustrates dynamic linking and user-controlled navigation and browsing according to a preferred embodiment of the present invention.
DOCUMENTS INCORPORATED BY REFERENCE
The Appendix attached hereto and referenced herein is incorporated by reference. This Appendix includes exemplar code illustrating a prefened embodiment of the present invention.
CONTENTS OF DETAILED DESCRIPTION OF THE INVENTION
A. DEFINITIONS B. OVERVIEW
1. INVENTION CONTEXT
2. VAL UE PROPOSITIONS
3. TODA Y'S- "INFORMATION" WEB VS. THE INFORMATION NERVOUS SYSTEM OF THE PRESENT INVENTION
SYSTEM ARCHITECTURE AND TECHNOLOGY CONSIDERATIONS
1. SYSTEM OVERVIEW
2. SYSTEM ARCHITECTURE
3. TECHNOLOGY STACKS
4. SYSTEM HETEROGENEITY
5. SECURITY
6. EFFICIENCY CONSIDERA TIONS D. SYSTEM COMPONENTS AND OPERATION
1. AGENCIES AND AGENTS a. Agencies b. Agents
2. KNOWLEDGE INTEGRATION SERVER a. Semantic Network b. Semantic Data Gatherer c. Semantic Network Consistency Checker d. Inference Engine e. Semantic Query Processor f Natural Language Parser g. Email Knowledge Agent h. Knowledge Domain Manager i. Other Components
3. KNO WLEDGE BASE SER VER
4. INFORMATION AGENT (SEMANTIC BROWSER PLATFORM) a. Over-view b. Client Configuration c. Client Framework Specification d. Client Framework e. Semantic Query Document Semantic Environment g. Semantic Environment Manager h. Environment Browser (Semantic Browser or Information Agent™) i. Additional Application Features
5. PROVIDING CONTEXT IN THE PRESENT INVENTION a. Context Templates b. Context Skins c. Skin Templates d. Default Predicates e. Context Predicates f Context Attributes g. Context Palettes h. Intrinsic Alerts i. Smart Recommendations
6. PROPERTY BENEFITS OF THE PRESENT INVENTION E. SCENARIOS
1. EXAMPLES OF SEMANTIC QUERIES UTILIZING THE PRESENT INVENTION
2. BUSINESS PROBLEMS
3. SITUA TIONS
DETAILED DESCRIPTION OF THE INVENTION A. DEFINITIONS
ActϊonScript. Scripting language of Macromedia Flash. This two-way communication assists users in creating interactive movies. See http://www.macromedia.com/support/flash/action_scripts/actionscript_tiitorial/. ' . Agency. A named instance of a Knowledge Integration Server (KIS) that is the semantic equivalent of a website.
Agency Directory. A directory that stores metadata infonnation for Agencies and allows clients to add, remove, search, and browse Agencies stored within. Agencies can be published on directories like LDAP or the Microsoft Active Directory. Agencies can also be published on a proprietary directory built specifically for Agencies.
Agent. A semantic filter query that returns XML information for a particular semantic object type (e.g., documents, email, people, etc.), context (e.g., Headlines, Conversations, etc.) or Blender.
• Blender™ or Compound Agent™. Trademarked name for an Agent that contains other Agents and allows the user (in the case of client-side blenders) or the Agency administrator (in the case of server-side blenders) to create queries that generate results that are the union or intersection of the results of their contained Agents. In the case of client-side blenders, the results can be generated using different views (showing each Agent in the blender in a different frame, showing all the objects of a particular object type across the contained Agents, etc.)
• Breaking News Agent™. Trademarked name for a Smart Agent that users specially tag as being indicative of time-criticality. Users can tag any Smart Agent as a Breaking News Agent. This attribute is then stored in users' Semantic Environment. A Breaking News Agent preferably shows an alert if there is breaking news related to any information being displayed.
• Default Agent™. Trademarked name for standardized, non-user modifiable Agents presented to the user.
• Domain Agent™. Trademarked name for an Agent that belongs to a semantic domain. It is initialized with an Agent query that includes reference to the
"categories" table. • Dumb Agent™. Trademarked name for an Agent that does not have an Agency and which refers to local infoπnation (on a local hard drive), on a network share or on a Web link or URL. Dumb Agents are used to essentially load infomiation items (e.g., documents) from a non-smart sandbox (e.g., the file-system or the Internet) to a smart sandbox (the Information Nervous System via the Infomiation
Agent (semantic browser)).
• Email Agent™ (or Email Knowledge Agent™). Trademarked names for a Public Agent used to publish or annotate infoπnation and share knowledge on an Agency. • Favorite Agent™. Trademarked name for Agents that users indicate they like and access often.
• Public Agent™. Trademarked name for Agents that are created and managed by the system administrator.
• Private or Local Agents™. Trademarked names for Agents that are created and managed by users.
• Search Agent™. Trademarked name for a Smart Agent that is created by searching the semantic environment with keywords or by searching an existing Smart Agent, in order to invoke an additional, text-based query filter on the Smart
' Agent. o Simple or Standard Agent™. Trademarked names for Standalone Agents that encapsulate structured, non-semantic queries (e.g., from the local file system or data source).
• Smart Agent™. Trademarked name for a standalone Agent that encapsulates structured, semantic queries that refers to an Agency via its XML Web Service. • Special Agent™. Trademarked name for a Smart Agent that is created based on a
Context Template.
Agent Discovery. The property of the information medium of the present invention that allows users to easily, and automatically discover new server-side Agents or client-side
Agents created by others (friends or colleagues). Also see "Discoverability." Annotations. Notes, comments, or explanations that are used to add personal context to an information object. In the prefened embodiment, annotations are email messages that are linked to the object they qualify, and which can have attachments (just like regular email messages). In addition, annotations are first class information objects in the system and as such can be annotated themselves, thereby resulting in threaded annotations or a tree of annotations with the initial object as the root.
Application Programming Interface (API). Defines how software programmers utilize a particular computer feature. APIs exist for windowing systems, file -systems, database systems, networking systems, and other systems.
Calendar Access Protocol (CAP).- Internet protocol that permits users to digitally access a calendar store based on the iCalendar standard.
Compound Agent Manager™. Trademarked name for an Agency component that programmatically allows the user to create and delete Compound Agents and to manage them by adding and deleting Agents.
Context. Information surrounding a particular item that provides meaning and otherwise assists the infoπnation consumer in interpreting the item as well as finding other relevant infoπnation related to the item. Context Results Pane. A Results Pane that displays results for context-based queries.
These include results for Context Palettes, Smart Lenses, Deep Information, etc. See "Results Pane."
Context-Sensitivity. The property of an information medium that enables it to intelligently and dynamically perceive the context of all the information it presents and to present additional, relevant information given that context. A context-sensitive system or medium understands the semantics of the information it presents and provide appropriate behaviors (proactive and reactive based on the user's actions) in order to present infoπnation in its proper context (both intrinsically and relationally).
Context Template™. Trademarked name, for scenario-driven infomiation query templates that map to specific and familiar semantic models for information access and retrieval. For example, a "Headlines" template in the preferred embodiment; has parameters that are consistent with the delivery of "Headlines" (where freshness and the likelihood of a high interest level are the primary axes for retrieval). An "Upcoming Events" template has parameters that are consistent with the delivery of "Upcoming Events." And so on. Essentially, Context Templates can be analogized to personal, digital semantic infomiation retrieval "chaimels" that deliver infomiation to the user by employing a well-known semantic template.
Deep Information™. Trademarked name for a feature of the present invention that enables the Information Agent to display intrinsic, contextual infomiation relating to an information object. The contextual infomiation that includes information that is mined from the Semantic Network of the Agency from whence the object came.
Discoverability. The ability of the infomiation medium of the present invention to intelligently and proactively make information known or visible to the user without the user having to explicitly look for the infomiation.
Domain Agent Wizard™. Trademarked name for a system component and its user interface for allowing the Agency administrator to create and manage Domain Agents.
DOTNET (.NET). Microsoft® .NET is a set of Microsoft software technologies for connecting infonnation, people, systems, and devices. It enables software integration through the use of XML Web Services: small, discrete, building-block applications that connect to each other, as well as to other, larger applications, via the Internet. .NET-connected software facilitates the creation and integration of XML Web Services. See http://www.microsoft.com/net/defined/default.asp).
Dynamic Linking™. Trademarked name for the ability of the Infomiation Nervous System of the present invention to allow users to link information dynamically, semantically, and at the speed of thought, even if those infoπnation items do not contain links themselves. By virtue of employing smart objects that have intrinsic behavior and using recursive intelligence embedded in the Information Agency's XML Web Service, each node in the Semantic Network is much smarter than regular link or node on Today's Web or the conceptual Semantic Web. In other words, each node in the Smart Virtual Network or Web of the present invention can link to other nodes, independent of authoring. Each node has behavior that can dynamically link to Agencies and Smart Agents via drag and drop and smart copy and paste, create links to Agencies in the Semantic Environment, respond to lens requests from Smart Agents to create new links, include intrinsic alerts that ill dynamically create links to context and time-sensitive infonnation on its Agency, include presentation hints for breaking news (wherein the node can automatically link to breaking news Agents in the namespace), form the basis for deep info that can allow the user to find new links, etc. A user of the present invention is therefore not at the mercy of the author of the metadata. Once the user reaches a node in the network, the user has many semantic means of navigating dynamically and automatically — using context, time, relatedness to Smart Agencies and
Agents, etc.
Email XML Object. An infoπnation object with the "Email" information object type. The XML object has the "Email" SRML schema (which uses XML). Environment Browser. See Information Agent.
Favorite Agents Manager™. Trademarked name for a system component and user interface element that allows the Agency administrator to manage server-side Favorite Agents. Flash. Macromedia Flash user interface platform that enables developers and content authors to embed sophisticated graphics and animations in their content. See http://www.macromedia.com/flash.
Flash MX. Macromedia Flash MX is a text, graphics, and animation design and development environment for creating a broad range of high-impact content and rich applications for, the Internet. See http://www.macromedia.com/software/flash/productinfo/product_overview/. Global Agency Directory™. Trademarked name for an instance of an Agency Directory that runs on the Internet (or other global network). The Global Agency Directory allows users to find, search, and browse Internet-based Agencies using their Infomiation Agent (directly in their semantic environment). Also, see "Agency Directory." HTTP. Hypertext Transfer Protocol (HTTP) is an application-level protocol for distributed, collaborative, hypermedia infonnation systems. It is a generic, stateless, protocol that can be used for many tasks beyond its use for, hypertext, such as name servers and distributed object management systems, through extension of its request methods, error codes and headers. A feature of HTTP is the typing and negotiation of data representation, allowing systems to be built independently of the data being transfened. See http://www.w3.Org/Protocols/ and http://www.w3.org/Protocols/Specs.html.
Inference Engine™. Trademarked name for the methodology of the present invention that observes patterns and data to anive at relevant and logically sound conclusions by reasoning. Preferably utilizes Inference Rules (a predetermined set of heuristics) to add semantic links to the Semantic Network of the present invention.
Information. A quantitative or qualitative measure of the relevance and intelligence of content or data and which conveys knowledge.
Information Agent™. Trademarked name for the semantic client or browser of the present invention that provides context and time-sensitive delivery and presentment of actionable information (or knowledge) from multiple sources, infoπnation types, and templates, and which allows dynamic linking of information across various repositories.
Information Nervous System™., Trademarked name for the dynamic, self-authoring, context and time-sensitive infoπnation system of the present invention that enables users to intelligently and dynamically link information at the speed of thought, and with context and time-sensitivity, in order to maximize the acquisition and use of knowledge for the task at hand. Information Object™ (or Item or Packet). Trademarked name for a unit of infonnation of a particular type and which conveys knowledge in a given context.
Information Object Pivot™. Trademarked name for an infoπnation object that users employ as a navigational pivot to find other relevant infomiation in the same context. Information Object Type. See Object Type.
;j
Intelligent Agent. Software Agents that act on behalf of the user to find and filter infoπnation, negotiate for services, easily automate complex tasks, or collaborate with other software Agents to solve complex problems. By definition, Intelligent Agents must be autonomous or, in other words, freely able to execute without user intervention. Additionally, Intelligent Agents must be able to communicate with other software or human Agents and must have the ability to perceive and monitor the environment in which they reside. See http://www.findarticles.com/cf_dls/m0FWE/7_4/64694222/pl/article.jhtml).
Internet Calendaring and Scheduling (iCalendar). Protocol that enables the deployment of interoperable calendaring and scheduling services for the Internet. The protocol provides the definition of a common foπnat for openly exchanging calendaring and scheduling infoπnation across the Internet.
Internet Message Access Protocol (IMAP). Communications mechanism for mail clients to interact with mail servers, and manipulate mailboxes thereon. Perhaps the most popular mail access protocol cunently is the Post Office Protocol (POP), which also addresses remote mail access needs. IMAP offers a superset of POP features, which allow much more complex interactions and provides for much more efficient access than the POP model. See http://www-smi.stanford.edu/projects/imap/ml/imap.html.
Intrinsic Semantic Link™. Trademarked name for semantic links that are intrinsic to the schema of a particular information object. For instance, an email information object has intrinsic links like "from," "to," "cc," "bcc," and "attachments" that are native to the object itself and are defined in the schema for the email infoπnation object type. Island. An infoπnation repository that is isolated from other repositories which may contain relevant, semantically related, context and time-sensitive infoπnation but which are disconnected from other contexts in which such information might be relevant.
J2EE. The JavaTM 2 Platfoπn, Enterprise Edition (J2EE) used for developing multi-tier enterprise applications. J2EE bases enterprise applications on standardized, modular components by providing a set of services to those components and by handling many details of application behavior automatically. See http://java.sun.com/j2ee/overview.html.
Knowledge. Infonnation presented in a context and time-sensitive manner that enables the information consumer to leam from the information and apply the infomiation in order to make smarter and more timely decisions for relevant tasks. Knowledge Agent™. See Information Agent. ■ Knowledge Base Server™ (KBS). Trademarked name for a server that hosts knowledge for the Knowledge Integration Server (KIS). Knowledge Domain Manager™ (KDM). Trademarked name for a component of the Knowledge Integration Server that is responsible for adding and maintaining domain-specific intelligence on the Semantic Network.
Knowledge Integration Server™ (KIS). Trademarked name for a server that semantically integrates data from multiple diverse sources into a Semantic Network, which can also host server-side Agents that provide access to the network and which hosts XML Web Services that provide context and time-sensitive access to knowledge on the server. Knowledge Web™. See Infoπnation Nervous System.
Liberty Alliance. The vision of the Liberty Alliance is to enable a networked world in which individuals and businesses can more easily conduct transactions while protecting the privacy and security of vital identity information. To accomplish its vision, the Liberty Alliance seeks to establish an open standard for federated network identity tlirough open technical specifications. See http://www.projectliberty.org/index.html.
Lightweight Directory Access Protocol (LDAP). Technology for accessing common directory infoπnation. LDAP has been embraced and implemented in most network-oriented middleware. As an open, vendor-neutral standard, LDAP provides an extendable architecture for centralized storage and management of information that needs to be available for today's distributed systems and services. LDAP is cunently supported in most network operating systems, groupware and even shrink-wrapped network applications. See http://publib-b.boulder.ibm.com/Redbooks.nsf/RedbookAbstracts/sg244986.html7Open. Link Template™. See Context Template.
Local Context. Local Context refers to client-side information objects and Agents accessible to the users. This includes Agents in the Semantic Enviromnent, local files, folders, email items in users' email inboxes, users' favorite and recent Web pages, the cunent Web page(s), cunently opened documents, and other information objects that represent users' cunent task, location, time, or condition.
Meaning. The attributes of behavior of information that allows the consumer of the information to locate and navigate to it based on its relevant information content (as opposed to its text or data) and to act on it in a context and time-sensitive manner, in order to maximize the utility of the information. Metadata. "Data about data." It includes those data fields, links, and attributes that fully describe an infonnation object.
Natural Language Parser. Parsing and interpreting software component that understands natural language queries and can translate them to structured semantic infoπnation queries. Nervana™. Trademarked name for a proprietary, end-to-end implementation of the Infonnation Nervous System information medium/platfonn. The name also defines a proprietary namespace for resource type and predicate name qualifiers.
.NET Passport. Microsoft .NET Passport is a suite of Web-based services directed towards the Internet and online purchasing. .NET Passport provides users with single sign-in (SSI) and fast purchasing capability at a growing number of participating sites, reducing the amount of infomiation users must remember or retype. .NET Passport provide a high-quality online experience for a large user base and uses powerful encryption technologies — such as Secure Sockets Layer (SSL) and the Triple Data Encryption Standard (3DES) algorith — for data protection. Privacy is a key priority as well, and all participating sites sign a contract in which they agree to post and follow a privacy policy that adheres to industry-accepted guidelines.
Network Effects. This exists when the number of other users affects the value of a product or service to a particular user. Telephone service provides a clear example. The value of telephone service to users is a function of the number of other subscribers. Few would be interested in telephones that were not connected to anyone, and most would assess higher value to a phone service linked to a national network rather than just a local network. Similarly, many computer users prize a computer system that allows them to exchange information readily with other users. Network Effects are thus demand-side externalities that generate a positive feedback effect in which successful products become more successful. In this way, Network Effects are analogous to supply-side economies of scale and scope. As a firm increases output, economies of scale lead to lower average costs, permitting the fiπn to lower prices and gain additional business from rivals. Continued expansion results in even lower average costs, justifying even lower prices. Similarly, the positive feedback from Network Effects builds upon previous successes. In the computer industry, for example, users pay more for a more popular computer system, all else equal, or opt for a system with a larger installed base if the prices and other features of two competing systems are equivalent. See •http://www.ei.com/publications/1996/falll .htm.
Network News Transfer Protocol (NNTP). Protocol for the distribution,.: inquiry, retrieval, and posting of news articles using a reliable stream-based transmission of news among the ARPA-Internet community. NNTP is designed so that news articles are stored in a central database allowing subscribers to select only those items they wish to read. Indexing, cross-referencing, and expiration of aged messages are also provided.
Notifications. Notifications are alerts that are sent by the Information Agent or an Agency to indicate to a user that there is new information on an Agent (either a client-side
Agent or a server-side Agent). Users can request notifications from Agents in their Semantic
Environment. Users can indicate that they have received the notification. The notification source (the client or server) stores infomiation for the user and the Agent indicating the last time the user acknowledged a notification for the Agent. The notification source polls the Agent to check if there is new infomiation since the last acknowledge time. If there is, the notification source alerts the user. Alerts can be sent via email, pager, voice, or a custom alert mechanism such as Microsoft's .NET Alerts service. Users have the option of indicating their prefened notification mechanism for the entire notification source (client or server) — which applies to all Agents on the notification source — on a per-Agent basis (which overrides the indicated preference on the notification source.
Object. See Information Object.
Object Type. Identification data associated with infomiation that allows the consumer to understand the nature of the infoπnation, to interpret its contents, to predict how the infomiation can be acted upon, and to link it to other relevant information items based on how the object types typically relate in the real world. Examples include documents, events, email messages, people, etc. Ontology. Hierarchical structuring of knowledge according to essential qualities. Ontology is an explicit specification of a conceptualization. The tenn is bonowed from philosophy, where "Ontology" is a systematic account of Existence. For artificial intelligence systems, what "exists" is that which can be represented. When the knowledge of a domain is represented in a declarative formalism, the set of objects that can be represented is called the universe of discourse. This set of objects, and the describable relationships among them, are reflected in the representational vocabulary with which a knowledge-based program represents knowledge. Thus, in the conlext of artificial intelligence, the ontology of a program is described by defining a set of representational terms. In such ontology, definitions associate the names of entities in the universe of discourse (e.g., classes, relations, functions, or other objects) with human-readable text describing what the names mean, and formal axioms that constrain the interpretation and well-formed use of these terms. Formally, ontology is the statement of a logical theory.
The subject of ontology is the study of the categories of things that exist or may exist in some domain. The product of such a study, called ontology, is a catalog of the types of things that are assumed to exist in a domain of interest D from the perspective of a person who uses a language L for the purpose of talking about D. The types in the ontology represent the predicates, word senses, or concept and relation types of the language L when used to discuss topics in the domain D. See, generally, http://www-ksl.stanford.edu/kst/what- is-an-ontology.html and http://users.bestweb.net/~sowa/ontology/).
Predicates. A Predicate is an attribute or link whose result represents the truth or falsehood of some condition. For example, the predicate "authored by" links a person with an infoπnation object and indicates whether a person authored the object.
Presenter™. System component in the Information Agent (semantic browser) of the present invention that handles the aggregation and presentation of results from the semantic query processor (that preferably interprets SQML). The Presenter handles layout management, aggregation, navigation, Skin management, the presentation of Context Palettes, interactivity, animations, etc.
RDF. Resource Description Framework (RDF) is a foundation for processing metadata; it provides interoperability between applications that exchange machine-understandable infoπnation on the Web. RDF emphasizes facilities to enable automated processing of Web resources. RDF defines a simple model for describing relationships among resources in terms of named properties and values. RDF properties may be thought of as attributes of resources and in this sense conespond to traditional attribute- value pairs. RDF properties also represent relationships between resources. As such, the RDF data model can therefore resemble an entity-relationship diagram.
RDF can be used in a variety of application areas including, for example: in resource discovery to provide better search engine capabilities, in cataloging for describing the content and content relationships available at a particular Web site, page, or digital library, by intelligent software Agents to facilitate knowledge sharing and exchange, in content rating, in describing collections of pages that represent a single logical "document", for describing intellectual property rights of Web pages, and for expressing the privacy preferences of a user as well as the privacy policies of a Web site. RDF with digital signatures is preferably a component of building the "Web of Trust" for electronic commerce, collaboration, and other applications. See, generally, http://www.w3.org/TR/PR-rdf-syntax/ and http://www.w3.org/TR/rdf-schema/.
RDFS. Acronym for RDF Schema. Resource description communities require the ability to say certain things about certain kinds of resources. For describing bibliographic resources, for example, descriptive attributes including "author", "title", and "subject" are common. For digital certification, attributes such as "checksum" and "authorization" are often required. The declaration of these properties (attributes) arid their conesponding semantics are defined in the context of RDF as an RDF schema. A schema defines not only the properties of the resource (e.g., title, author, subject, size, color, etc.) but may also define the kinds of resources being described (books, Web pages, people, companies, etc.). See http://www.w3.org/TR/rdf-schema/).
Results Pane™. Trademarked name for the graphical display area within the Infonnation Agent (semantic browser) that displays results of an SQML query. See FIGURE 5, showing a sample Infomiation Agent screenshot illustrating server-side Agents, an optional player control/navigation/filter toolbar, a "Server-Side Agents Dialog" (which allows users to browse and open server-side Agents), and sample results (with the "Documents" infoπnation object type) from a server-side Agent.
Semantics. Connotative meaning.
Semantic Environment™. This refers to all the data stored on users' local machines, in addition to user-specific data on an Agency server (e.g., subscribed server-side Agencies, server-side Favorite Agents, etc.). Client-side state includes favorite and recent Agents and authentication and authorization infomiation (e.g., user names and passwords for various Agencies), in addition to the SQML files and buffers for each client-side (user-created) Agent. The Infomiation Agent is preferably configured to store Agents for a set amount of time before automatically deleting them, except those that have been added to the "favorites" list. For example, users may configure the Information Agent to store Agents for two weeks. In this case, Agents older than two weeks are automatically purged from the system and the Semantic Environment is adjusted accordingly. The Semantic Environment is employed for Context Palettes (Context Palettes use the Agencies in the "recent" and "favorites" list in order to predict what default Agencies users want to view context from).
Semantic Environment Manager™. Trademarked name for a software component that manages all the local state for the Semantic Environment (in the Information Agent). This includes storing and managing the metadata for all the client-side Agents (and the history and favorites Agent sub-lists), per-Agent state (e.g., Agent Skins, Agent preferences, etc.), notification management, Agency browsing (on Agency directories), listening for Agencies via multicast and peer-to-peer announcement protocols, services to allow users to browse the Semantic Environment via the semantic browser (via the Tree View, the "Open " Agent" dialog, and the Results Pane), etc. Semantic Data Gatherer™ (SDG). Trademarked name for XML Web Service used by the Knowledge Integration Server (KIS) and which is responsible for' adding, removing and updating entries in the Semantic Network via the Semantic Metadata Store (SMS).
Semantic Metadata Store™ (SMS). Trademarked name for a software component on the KIS that employs a database (e.g., SQL Server, Oracle, DB2) having tables for each primary object type to store all the metadata on the KIS.
Semantic Network. System and method of linking objects associated with schemas together in a semantic way via the database tables on the Semantic Metadata Store.
Semantic Network Consistency Checker™. Trademarked name for a software component that runs on an Agency of the present invention that is tasked with maintaining the integrity and consistency of the Semantic Network. The checker runs periodically and ensures that entries in the "SemanticLinks" table exist in the native object tables, that entries in the "objects" table exist in the native object tables and that all entries in the Semantic Metadata Store still exist at the repositories from where they were gathered.
Semantic Queries. Queries that incorporate meaning, context, time-sensitivity, context-templates, and richness that approach natural language. Much more powerful than simple, keyword-based queries in that they are context and time-sensitive and incorporate meaning or semantics.
Semantic Query Markup Language (SQML). A proprietary XML-based query language used by this invention to define, store, interpret and execute client-side semantic queries. SQML includes tags to define a query that gets its data from diverse resources (that represent data sources) such as files, folders, application repositories, and references to Agency XML Web Services (via resource identifiers and URLs). In addition, SQML includes tags that enable semantic filtering (via custom links and predicates) which indicate how data is to be queried and filtered from the resources, and arguments that indicate how the resources are to be queried and how the results are to be filtered. In particular, the arguments can include references to local or remote context. The context arguments are then resolved by the client-side SQP at run-time to XML metadata. The XML metadata is then passed to the appropriate resource (e.g., an Agency's XML Web Service) as a method call along with the reference to the resource and the semantic links and predicates that indicate how the query is to be resolved by the resource (e.g., the Agency's XML Web Service). SQML is to the Infomiation Nervous System as HTML is to Today's Web. The main difference is that SQML defines the rules for semantic querying while HTML defines the rules for Hypertext presentation. However, SQML is superior in that it enables the client to recursively create new semantic queries from existing ones (by creating new SQML with new links derived ■from an existing SQML query), e.g., via drag and drop and smart copy and paste, the Smart Lens, Context Templates and Palettes, etc. In addition, because SQML does not define the rules for presentation, the results of the semantic query can be presented in multiple ways, using a "skin" that takes the results (in SRML) to generate presentation based on the user's preferences, interests, condition, or context. Furthermore, SQML can contain abstract links and predicates such as those that refer to or employ Context Templates. The resource (e.g., the Agency's XML Web Service) then resolves the SQML to an appropriate query foπnat (e.g., SQL or the equivalent in the case of an Agency's XML Web Service) and then invokes the "actual" query in order to generate the results (which will then account for the user's context or Context Template). Also, an SQML buffer or file can refer to multiple resources (and Agencies), thereby empowering the client to view results in an aggregated fashion (e.g., based on context or time-sensitivity), rather than based on the source of the data - this is a powerful feature of the invention that enables user-controlled browsing and information aggregation (see the sections on both below). Lastly, every client-side Agent has an SQML definition and file, just as every Web page has an HTML file.
Semantic Query Processor™ (SQP). Trademarked name for the server-side semantic query processor (XML Web Service in the prefened embodiment) that takes SQML and converts it to SQL (in the prefened embodiment) and then returns the results as XML. On the Knowledge Integration Server (KIS), the SQP is the main entry point to the Semantic Network of the present invention responsible for responding to semantic queries from clients of the KIS. On the server, this is the software component that processes semantic queries represented as SQML from the client. On the client, the client-side SQP takes aggregate SQML and compiles or maps it to individual SQML queries that can be sent to a server (or Agency) XML Web Service.
Semantic Results Markup Language (SRML). A proprietary XML-based data schema and format used by this invention to define, store, interpret and present semantic results. On the client, SRML is returned from the SQP via semantic resource handlers that interpret, format, and issue query requests to semantic data sources. Semantic data sources will include an Agency's XML Web Sendee, local files, local folders, custom data sources from local or remote applications (e.g., a Microsoft Outlook email application inbox), etc. The XML Web Service will return SRML to a client, in response to the client's semantic query. This way, the XML Web Service will not "care" how the results are being presented at the client. This is in contrast with Today's Web and the Semantic Web where servers return ahead y- formatted HTML for a client to present and where clients merely present presentation data (as opposed to semantic data) and camiot customize the presentation of the data. In this invention, two clients can render the same SRML in completely different ways, based on the cunent "skin" that has been selected or applied by the user of either client. The "skin" then' converts " the SRML to a presentation-ready format such' as XHTML, DHTML+TIME, SVG, Flash MX, etc. SRML is a meta-schema, meaning that it is a container foπnat that can include data for different infonnation object types (e.g., documents, email, people, events, etc.). An SRML file or buffer can contain intertwined results for each of these object types. Well-fonned SRML will contain well-foπned XML document sections that are consistent with the schema of the infoπnation object types that are contained in the semantic result the SRML represents. See Sample A of the Appendix hereto.
Semantic Web. Extension of Today's Web in which infoπnation is given well-defined meaning, better enabling computers and people to work in cooperation. See Tim Berners-Lee, James Hendler, Ora Lassila, The Semantic Web, Scientific American, May 2000.
Facilities to put machine-understandable data on Today's Web are becoming a high priority for many communities. The Web can reach its full potential only if it becomes a place where data can be shared and processed by automated tools as well as by people. For the Web to scale, tomonow's programs must be able to share and process data even when these programs have been designed totally independently. The Semantic Web is a conceptual vision: the idea of having data on the Web defined and linked in a way that it can be used by machines not just for display purposes, but for automation, integration and reuse of data across various applications. See also http://www.w3.org/2001/sw/.
Session Announcement Protocol (SAP). In order to assist the advertisement of multicast multimedia conferences and other multicast sessions, and to communicate the relevant session setup information to prospective participants, a distributed session directory may be used. An instance of such a session directory periodically multicasts packets containing a description of the session, and these advertisements are received by other session directories such that potential remote participants can use the session description to start the tools required to participate in the session. In its simplest form, this involved periodically multicasting a session announcement packet describing a particular session. To receive SAP, a receiver simply listens on a well-known multicast address and port. Sessions are described using the Session Description Protocol (ftp://ftp.isi.edu/in-notes/rfc2327.txt). If a receiver receives a session announcement packet it simply decodes the SDP message, and then can display the session infonnation for the user. The interval between repeats of the same session description message depends on the number of sessions being announced (each sender at a particular scope can hear the other senders in the same scope) such that the bandwidth being used for session announcements of a particular scope is kept approximately constant. If a receiver has been listening for a set time, and fails to hear a session announcement, then the receiver can conclude that the session has been deleted and no longer exists. The set period is based on the receivers' estimate of how often the sender should be sending.
See, generally, http://www.faqs.org/rfcs/rfc2974.html, http://www.video.ja.net/mice/archive/sdr_docs/nodel.html, ftp://ftp.isi.edu in- notes/rfc2327.txt.
Simple Mail Transfer Protocol (SMTP). Protocol designed to transfer mail reliably and efficiently. SMTP is independent of the particular transmission subsystem and requires only a reliable ordered data stream channel. An important feature of SMTP is its capability to relay mail across transport environments. See http://www.ietf.org/rfc/rfc0821.txt. Skins. Presentation templates that are used to customize the user experience on a per- Agent basis or which customizes the presentation of the entire layout (independent of the Agent), or object (based on the infonnation object type), context (based on the Context Template), Blender (for Agents that are Blenders), for the semantic domain name/path or ontology, and other considerations. Each Agent will include a Skin which in turn will have an XML metadata representation of parameters to customize the layout of the XML 'results that represent information objects (the layout Skin), for example, whether or not those results are animated, the manner in which each result is displayed, including a representation of the object type (the object Skin), styles, colors, graphics, filters, transforms, effects, animations (and so on) that indicate the ontology of the cunent results (the ontology Skin), styles that indicate the Context Template of the cunent results (the context Skin) and styles that indicate how to view and navigate results from Blenders (i.e., the Blender Skin).
Smart Lens™. Trademarked name for a proprietary feature of this invention that allows users to select a Smart Agent or an object as a context with which to view another object or Agent. The lens then displays metadata, links, and result previews that give users an indication of what they should expect if the context is invoked. Essentially, the Smart Lens displays the results of a "potential query." The Smart Lens allows users to quickly preview context results without actually invoking queries (thereby increasing their productivity). In addition, the Smart Lens can display views that are consistent with the context, using pivots, templates and preview windows, thereby allowing -users to analyze the context in different ways before invoking a query. Smart Virtual Web™. Trademarked name for the property of the present invention to integrate semantics, context-sensitivity, time-sensitivity, and dynamism in order to empower users to browse a dynamic, virtual, "on-the-fly," user-controlled "Web" that they control and can customize. This is in contrast with Today's Web and the conceptual Semantic Web, both of which employ a manually authored network wherein users are at the mercy of the authors of the information on the network.
Structured Query Language (SQL). Pronounced "ess-que-el." SQL is used to communicate with a database. According to ANSI (American National Standards Institute), it is the standard language for relational database management systems. SQL statements are used to perform tasks such as update data on a database, or retrieve data from a database. Some common relational database management systems that use SQL are: Oracle, Sybase, Microsoft SQL Server, Access, Ingres, etc. Although most database systems use SQL, most of them also have their own additional proprietary extensions that are usually only used on their system. However, the standard SQL commands such as "Select", "Insert", "Update", "Delete", "Create", and "Drop" can be used to accomplish almost everything that one needs to dd with a database. SQL works with relational databases. A relational database stores data in tables
(relations).. A' database is a collection of tables. A table consists of a list '. of records, each record in a table preferably includes the same structure, and each has a fixed number of "fields" of a given type.
See, generally, http://www.sqlcourse.com/intro.html and http://www.dcs.napier.ac.Uk/~andrew/sql/0/w.htm.
Scalable Vector Graphics (SVG). Language for describing two-dimensional graphics in XML- SVG allows for three types of graphic objects: vector graphic shapes (e.g., paths consisting of straight lines and curves), images and text. Graphical objects can be grouped, styled, transformed and composited into previously rendered objects. Text can be in any XML namespace suitable to the application, which enhances searchability and accessibility of the SVG graphics. The feature set includes nested transformations, clipping paths, alpha masks, filter effects, template objects and extensibility. SVG drawings can be dynamic and interactive. The Document Object Model (DOM) for SVG, which includes the full XML DOM, allows for straightforward and efficient vector graphics animation via scripting. A rich set of event handlers such as onmouseover and onclick can be assigned to any SVG graphical object. Because of its compatibility and leveraging of other Web standards, features like scripting can be done on SVG elements and other XML elements from different namespaces simultaneously within the same Web page. See http://www.w3.org/Graphics/SVG/Overview.htm8. Taxonomy. An organizational stracture wherein divisions are ordered into groups or categories. Time-Sensitivity. Property of an infomiation medium to deliver and present infonnation based on when the infonnation would be most relevant in time. For instance, freshness is an attribute that denotes time-sensitivity. In addition, the delivery and presentation of upcoming events (which, by definition, are time-sensitive) and the manner in which the time-criticality of the events are displayed are properties of a time-sensitive medium.
Today's Web. This refers to the World Wide Web as we know it today. Today's Web is a universe of hypertext servers (HTTP servers), which are the servers that allow text, graphics, sound files, etc. to be linked together. Hypertext is simply a non-linear way of presenting infoπnation. Rather than reading or learning about things in the order that an author, or editor, or publisher sets out for us, readers of hypertext may follow their own path, create their own order or meaning out the material. This is accomplished by creating "links" between infoπnation. These links are provided so that user may "jump" to further information about a specific topic being discussed (which may have more links, leading each reader off into a different direction). The Hypertext medium can incorporate pictures., sound, and video present a multimedia approach to presenting infoπnation, also refened to as hypeπnedia. See, generally, ' http://www.w3.org/History.html and http://www.umassd.edu/Public/People/KAmaral/Thesis/hypertext.html.
Multicast Time to Live (TTL). Multicast routing protocol uses the field of datagrams to decide how "far" from a sending host a given multicast packet should be forwarded. The default TTL for multicast datagrams is 1, which will result in multicast packets going only to other hosts on the local network. A setsockopt(2) call may be used to change the TTL. As the value for TTL increases, routers will expand the number of hops they will forward a multicast packet. To provide meaningful scope control, multicast routers typically enforce the following "thresholds" on forwarding based on the TTL field: • • 0 restricted to the same host • 1 restricted to the same subnet
• 32 restricted to the same site
• 64 restricted to the same region
• • 128 restricted to the same continent • 255 unrestricted
See http://www.isl.org/projects/eies/mbone/mbone27.htm.
User State. This refers to all state that is either created by a user or which is needed to cache a user's preferences, favorites, or other personal infomiation on a client or server.
Client-side User State includes authentication credential infoπnation, users' Agent lists (and all the metadata including the SQML queries for the Agents), home Agent, configuration options, preferences such as Skins, etc. Essentially, client-side User State is a persisted form of users' Semantic Environment. Server-side User State includes infoπnation such as users' Favorite Agents, subscribed Agents, Default Agent, semantic links to information objects on the server (e.g., "favorites" links) etc. Server-side User State is optional for servers but support for it is preferred. Servers preferably support user logon and a "people" object type (even without server-side Agents) because these are needed for features such as favorites, recommendations, and for Context Templates such as "Newsmakers," "Experts," "Recommendations," "Favorites," and "Classics."
Virtual Information Object Type™. Trademarked name for object types that do not map to distinct object types, yet are semantically of interest to users.
Virtual Parameter™. Trademarked name for variables, parameters, arguments, or names that are dynamically interpreted at runtime by the semantic query processor. This allows the Agency administrator to store Agents that refer to virtual names and then have those names be converted to actual relevant terms when the query is invoked. Web of Trust. Term coined by members of the Semantic Web research community that refers to a chain of authorization that users of the Semantic Web can. use to validate assertions and statements. Based on work in mathematics and cryptography, digital signatures provide proof that a certain person wrote (or agrees with) a document or statement. Users can preferably digitally sign all of their RDF statements. That way, users can be sure that they wrote them (or at least vouch for their authenticity). Users simply tell the program whose signatures to trust. Each can set their own levels of trust (or paranoia), and the computer can decide how much of what it reads to believe.
By way of example, with a Web of Trust, a user can tell a computer that he or she trusts his or her best friend, Robert. Robert happens to be a rather popular guy on the Net, and trusts quite a number of people. All the people he trusts in turn trast another set of people. Each of these measures of trust is to a certain degree (Robert can trust Wendy a whole lot, but Sally only a little). In addition to trust, levels of distrust can be factored in. If a user's computer discovers a document which no one explicitly trusts, but no one has said it has totally false either, it will probably trust that information a little more than one which many people have said is false. The computer takes all these factors into account when deciding the trustworthy of a piece of infomiation. Preferabty, the computer combines all this infoπnation into a simple display (thumbs-up / thumbs-down) or a more complex explanation (a description of all the various trust factors involved). See http://blogspace.com/rdf/SwartzHendler.
Web Services-Interoperability (WS-I). An open industry organization chartered to promote Web services interoperability across platfoπns, operating systems, and programming languages. The organization works across the industry and standards organizations to respond to user needs by providing guidance, best practices, and resources for developing Web services solutions. See http://www.ws-i.org.
Web Services Security (WS-Security). Enhancements to SOAP messaging providing quality of protection through message integrity, message confidentiality, and single message authentication. These mechanisms can be used to accommodate a wide variety of security models and encryption technologies. WS-Security also provides a general-purpose mechanism for associating security tokens with messages. No specific type of security token is required by WS-Security. It is designed to be extensible (e.g. support multiple security token foπnats). For example, a client might provide proof of identity and proof that they have a particular business certification. Additionally, WS-Security describes how to encode binary security tokens. Specifically, the specification describes how to encode X.509 certificates and Kerberos tickets as well as how to include opaque encrypted keys. It also includes extensibility mechanisms that can be used to further describe the characteristics of the credentials that are included with a message. See http://msdn.microsoft.com/library/default.asp?url:=/library/en-us/dnglobspec tml/ws- security.asp.
Extensible Markup Language (XML). Universal format for structured documents and data on the Web. Structured data includes things like spreadsheets, address books, configuration parameters, financial transactions, and technical drawings. XML is a set of rules (you may also think of them as guidelines or conventions) for designing text formats that let you structure your data. XML is not a programming language, and one does not have to be a programmer to use it or leam it. XML makes it easy for a computer to generate data, read data, and ensure that the data structure is unambiguous. XML avoids common pitfalls in language design: it is extensible, platform-independent, and it supports internationalization and localization. XML is fully Unicode-compliant. See http://www.w3.org/XML/1999/XML-in-10-points.
XML Web Service (also known as "Web Service"). Service providing a standard means of communication among different software applications involved in presenting dynamic context-driven infoπnation to the user. More specific definitions include:
•A. A software application identified by a URI whose interfaces and binding are capable of being defined, described and discovered by XML artifacts. Supports direct interactions with other software applications using XML based messages via Internet-based protocols. 2. An application delivered as a service that can be integrated with other Web Services using Internet standards. It is an URL-addressable resource that programmatically returns information to clients that want to use it. The major communication protocol used is the Simple Object Access Protocol (SOAP), which in most cases is XML over HTTP.
3. Programmable application logic accessible using standard Internet protocols. Web Services combine aspects of component-based development and the Web. Like components, Web Services represent black-box functionality that can be reused without wonying about how the service is implemented. Unlike cunent component technologies, Web Services are not accessed via object-model-specific protocols, such as DCOM, RMI, or HOP. Instead, Web Services are accessed via ubiquitous Web protocols (ex: HTTP) and data formats (ex: XML).
See http://www.xmlwebservices.cc/, http://www.perfectxml.com/WebSvcl.asp and http://www.w3.Org/2002/ws/arch/2/06/wd-wsa-reqs-20020605.html. XQuery. Query language that uses the structure of XML to intelligently express queries across all these kinds of data, whether physically stored in XML or viewed , s XML via middleware. See http://www.w3.org/TR/xquery/ and http://www-
106.ibm.com developerworks/xml/library/x-xquery.html.
XPath. The result of an effort to provide a common syntax and semantics for functionality shared between XSL Transfomiations (http://www.w3.org/TR/XSLT) and XPointer (http://www.w3.Org/TR/xpath#XPTR). The primary purpose of XPath is to address parts of an XML [XML] document. In support of this primary purpose, it also provides basic facilities for manipulation of strings, numbers and Booleans. XPath uses a cpmpact, non-" XML syntax to facilitate use of XPath within URIs and XML attribute values. XPath operates on the abstract, logical structure of an XML document, rather than its surface syntax. XPath gets its name from its use of a path notation as in URLs for navigating through the hierarchical structure of an XML document.
In addition to its use for addressing, XPath is also designed so that it has a natural subset that can be used for matching (testing whether or not a node matches a pattern); this use of XPath is described in XSLT. XPath models an XML document as a tree of nodes. There are different types of nodes, including element nodes, attribute nodes and text nodes. XPath defines a way to compute a string-value for each type of node. Some types of nodes also .. have names. ' XPath fully supports XML Namespaces (http://www.w3.0rg/TR/xpath#XMLNAMES). Thus, the name of a node is modeled'as a pair consisting of a local part and a possibly null namespace URI; this is called an
(http://www.w3.Org/TR/xpath#dt-expanded-name). See http://www.w3.Org/TR/xpath#XPTR.
XSL. A style sheet language for XML that includes an XML vocabulary for specifying fonnatting. See http://www.w3.org/TR/xsltl l/. XSLT. Used by XSL to describe how a document is transformed into another XML document that uses the fonnatting vocabulary. See http://www.w3.org/TR/xsItl 1/. B. OVERVIEW
1. INVENTION CONTEXT
There is a misconception that the Holy Grail for infonnation access is the provision of natural language searching capability. Prior teclmologies for infomiation access have focused principally on improving the interface for searching for or accessing information to optimize infoπnation retrieval. The presumption has largely been that providing a natural language interface to infonnation will perfectly solve users' infomiation access problems and end the frustration users have with finding information. In truth, however, many axes of analysis are involved in how people acquire knowledge in the real world. One example is context. There are many things people know only because of where they were at a certain place and time. If they were not at that place at that time, they would not know what is in fact known or, indeed, might not care to know. Having the ability to search for what is presently known with natural language does not assist in uncovering the knowledge related to that particular time and place. There are simply no natural parameters that 'form the conect query to retrieve the desired information. The conundrum is that a person cannot ask for what he or she might not even know would have value until after the fact. Stated differently, one cannot query for what they do not know they do not know, or for what they do not know that they might want to know. Context-sensitivity, time-sensitivity, discovery, dynamic linking, user-controlled browsing, users' "Semantic Environment," flexible presentation, Context Skins, context attributes, Context Palettes (which bring up relevant, context and time-sensitive infoπnation based on Context Templates) and other aspects of this invention recognize and correct this fundamental deficiency with existing infoπnation systems.
For example, people may have many CDs in their library (thereby adding to the "knowledge" of music) because they attended certain parties and spoke with certain people. Those people at those parties mentioned the CDs to the person, thereby increasing the person's knowledge of music. As another example, a person may purchase a book (if read, increasing the person's knowledge on the particular topic of the book), based on a recommendation from a hitherto unknown stranger the person happened to sit beside on an aiφlane flight. In the real world, people acquire knowledge based not just on what they read and search for, but also based on the friends they keep, the people with whom they interact and the people whose judgment they trust. The "knowledge environment" is arguably as critical if not more critical for knowledge dissemination and acquisition as the model for retrieval (whether digital or analog). The present invention minors virtually every real-world knowledge-acquisition scenario in the digital world. The resulting Information Nervous System™ is the medium doing most of the work but the scenarios map very cleanly to the analog (real) world. The • inability of efforts such as natural-language search techniques of Today's Web as well as the Semantic Web to recognize the many ways in which knowledge is disseminated and acquired render them ultimately ineffective. The present invention accounts for the variety of ways in which humans have always acquired knowledge — independent of the actual technology used for infonnation delivery.
By way of example, there has always been context and there has always been time. Likewise there has always been the notion of discovery and the need to link infonnation dynamically and with user control. There have always been certain Context Templates, albeit in different mediums that presented herein, including "classics," "history," "timelines," "upcoming events," "headlines." These templates existed before the creation of the Internet, Today's Web, Email, e-Learning, etc. Nevertheless, prior to the present invention, there was no ability in the electronic medium to focus on the mode, protocol and presentation of knowledge delivery which maps to real-world scenarios (for example, via Context Templates, context-sensitivity, time-sensitivity, dynamic linking, flexible presentation, Context Skins, context attributes, etc.) as opposed to actual information types, semantic links, metadata, etc. There will always be new infomiation types. But the dissemination and acquisition axes of knowledge (e.g., Context Templates) have always and will always remain the same. The present invention captures this reality.
In addition, the present invention provides the ability to disseminate knowledge via serendipity. Serendipity plays a large part in knowledge acquisition in the real world and it is a first-class mode of knowledge delivery. The present invention enables a user to acquire infoπnation serendipitously (albeit intelligently) by its support for context, time, Context Templates, etc.
Infoπnation models or mediums that employ a strict, static structure like a "Web" break down because they assume the presence of an authored "network" or "Web" and fail to account, for the various axes of knowledge formation. Such infoπnation models are not user-focused, do not incoφorate context, time, dynamism and templates, and do not map to real-world knowledge , acquisition and dissemination scenarios. The present invention minimizes information loss and maximizes infomiation retained, even without the presence of a "Web" per se, and even if no natural language is employed to find infomiation. This is possible because, unlike existing mediums for infonnation access, a prefened embodiment of the present invention focuses on the knowledge dissemination models that incoφorate context, time, dynamism, and templates (for the benefit of both the end-user and the content producer) and not on the specifics of the access interface, or the linking (semantic or non- semantic) of infomiation resources based on static data models or human-based authoring. In many scenarios, a "Web" (semantic or non-semantic) is necessary as a means of navigation, but is far from being sufficient as a means of knowledge dissemination and acquisition. The Infoπnation Nervous System of the present invention incoφorates "knowledge axes" described in the invention (including but not limited to link-based navigation) and intelligently and seamlessly integrates them to facilitate the dissemination and acquisition of knowledge and to benefit all parties involved in the transfer of knowledge. 2. VALUE PROPOSITIONS Today, knowledge must be "manually hard-coded" into the digital fabric of an information structure, whether it be for an enteφrise, a consumer or the general inquiring population. If it is not authored and distributed properly, no one knows "of its existence, knows how it relates to other sources of intelligence, or knows how to act on it in real-time and in the proper fashion. This is largely because Today's Web was not designed to be a platform for knowledge. It was designed to be a platform for presentation and is intentionally dumb, static, and reactive. Today, knowledge-workers — those who seek to use infoπnation by adding context and meaning — are at the mercy of knowledge- authors.
A significant aspect of knowledge interaction is to have knowledge-workers be able to navigate their way through a knowledge space in a very intuitive manner, and at the speed at which they wish to make decisions and act on the knowledge. In other words, knowledge-workers do not have to "think" about an e-Learning island as being separate from documents in their organizations, e-mail that contains customer feedback, media files, upcoming video-conferences, a meeting they had recently, infomiation stored in newsgroups, or related books. The preferred situation is to relegate the infonnation "type" and "source" and to create a "seamless knowledge experience" that cuts across all those islands in a semantic way. In creating a knowledge experience, it is also prefened to be able to integrate knowledge assets across content-provider, partner, supplier, customer and people boundaries. In the enteφrise scenario, for example, no single organization has all the knowledge it needs to remain competitive. Knowledge is stored in industry reports, research documents from consulting fiπns and investment banks, media companies like Reuters™ and Bloomberg™, etc. All this constitutes "knowledge." It is not enough to deploy an e-Leaming repository to train users on a one-time or periodic basis. Users should have always-on access to knowledge from a variety of sources, in-place, and in an intelligent context that is relevant to their cunent task.
All this requires a layer of intelligence and pro-activity that is not available today. Today, for example, enteφrises use information portals, such as intranets and the Internet, as a way of disseminating infomiation to their employees. However, this is far from being enough, as it provides only presentation-level integration. This is akin to subscribing to newsletters to keep updated with information, as opposed to having an Agent that manages your information for you, helps you discover new information on-the-fly, helps you capture and share information with colleagues, etc.
To accomplish the desired level of knowledge interaction requires Agents working in the background, reasoning, learning, infening, matching users together based on their profiles, capturing new knowledge and automatically deducing new knowledge, and federating knowledge from external sources so that they become a seamless part of the knowledge experience. This in turn requires the semantic integration of knowledge assets so that they all make sense in a holistic fashion, rather than merely providing the basis for presentation-level integration and document searching. The implementation framework and i resulting medium must provide real-time, agile discovery and recommendation services so that context and time-sensitive infomiation is "honored" and such that knowledge-workers can be more productive and get more done faster and with less. And lastly, the system must work with existing infoπnation sources in a plug-n-play manner, must seamlessly and automatically classify and integrate known knowledge assets, and must embed the knowledge tools in the knowledge themselves, thereby adding another "dimension" into knowledge assets.
The present invention is designed to be an intelligent, proactive, real-time knowledge platform that co-exists with Today's Web (or any other layer of presentation). Incoφoration and use of the present invention will allow knowledge-workers to be in control of their knowledge experiences because authoring (via "connections") will be done intelligently, dynamically, automatically, and at the speed, of thought.
3. TODA Y'S "INFORMATION" WEB VS. THE INFORMATION NER VOUS SYSTEM OF THE PRESENT INVENTION
With Today's Web environment, the semantics of infonnation presented are lost upon conversation of the structured data to HTML at the server, meaning that the "knowledge" is stripped from the objects before the user has an opportunity to interact with them. In addition, Today's Web is authored and "hard-coded" on the server based on how the author "believes" the infomiation will be navigated and consumed. Users consume only infoπnation as it is presented to them.
The present invention adds a layer of intelligence and layers of customization that Today's HTML-based Web environment cannot support. The present invention provides an XML-based dynamic Web of smart knowledge objects rather than dumb Web pages wherein the semantics of the objects are preserved between the server and the client, thereby giving users much more power and control over their knowledge experience. In addition, with the Web of the present invention, knowledge-workers are able to consume and act on infonnation on their own tenns because they will interactively author their own knowledge experiences via "dynamic linking" and "user-controlled browsing."
The Infoπnation Agent (semantic browser) of the present invention is designed to co-exist with Today's Web and to integrate with and augment all facets of private and public intranets as well as the Internet. The technology platform stacks of Today's Web and the Infonnation Nervous System of the present invention are summarized in FIGURE 6. With reference to FIGURE 6, the stack for the Today's Web has at the bottommost layers Structured Infoπnation Sources, including such infonnation as the data stored in databases, and Unstructured Infoπnation Sources, including such information as documents, email messages, etc. Infoπnation in both of these layers is handled distinctly. No semantics are used at the Information Indexing Layer; rather, search engines based on keywords are used. The Logic Layer consists primarily of a database that allows programmability for searching, rules, view, triggers, etc. The Application Layer consists of server-side scripts that drive e- Business applications based on user input. At the topmost or Presentation Layer, Today's Web has presentation information (in the foπn of Web pages) that is exposed via portals with a Web platform (e.g., browser).
Apart from overlapping layers of processing, the present invention uniquely handles information from the bottommost level of operation in a manner that preserves the semantics of the underlying infomiation sources. At both the Structured and Unstructured Information Sources Layers, the system 10 handles infomiation uniformly, taking into account metadata and semantics associated with the infoπnation. At the Infoπnation Indexing Layer, information metadata and semantics are extracted from unstructured. The system 10 adds three additional platfoπn layers not present in Today's Web: Knowledge Indexing and Classification Layer, wherein information from both structured and unstructured sources are semantically. encoded; -Knowledge Representation Layer, wherein associations are created that allows maintenance of a self-conecting or healing Semantic Network of knowledge objects; and Knowledge Ontology and Inference Layer, wherein new connections and properties are infened in the Semantic Network. At the Logic Layer a knowledge-base is created that allows for programmability at a semantic level. At the Application Layer, seiver-side scripts are used in association with the knowledge-base. These scripts dynamically generate knowledge objects based on user input, and may include semantic commands for retrieval, notifications and logic. This Layer may also include Smart Agents to optimize the handling of semantic user input. The Presentation Layer of the system 10 preserves the semantics that are tracked from the bottommost layers. Presentation at this Layer is dynamically generated on the client computer system and completely customizable.
By the maintenance, integration and use of semantics in all technology layers, the present invention creates a virtual Web of actionable "objects" that directly conespond to
"things" that humans interact with physically or virtually or, in other words, as familiar
"Context Templates." As opposed to Today's Web, which is a dumb Web of documents, the present invention provides for a smart virtual Web of actionable objects that have properties and relationships, and in which events can dynamically cause changes in other parts' of the virtual Web.
The present invention provides a programmable Web. Unlike Today's Web which is a dumb Web of documents, the Web of the present invention is programmable akin to a database — it is able to process logic and rales, and will be able to initiate events.
While Today's Web is encoded for human, and thus is focused primarily on presentation of static information, the virtual Web of the present invention is encoded primarily for machines, albeit ultimately presented to humans as the end of the knowledge delivery chain. The present invention provides an intelligent, learning Web. This means that the virtual Web of the present invention will be able to leam new connections and become smarter over time. The Web is dynamic, virtual and self-authoring, thereby providing much more power to knowledge-workers by intelligently and proactively making semantic connections that Today's Web is unable to provide, thereby leading to a reduction in and ■ eventual elimination of infoπnation loss.
The Web of the present invention is a self-healing Web. Unlike Today's web which has to be manually maintained by document authors, the present invention provides a Web that is self-maintained by machines. This feature rectifies broken links because the Web will fix disconnections in the network automatically.
Finally, as will be set forth in greater detail below, the various embodiments of the present invention incoφorate some or all of the axes of knowledge acquisition described above to provide substantial advantages over existing systems directed to Today's Web or the conceptual Semantic Web.
C. SYSTEM ARCHITECTURE AND TECHNOLOGY CONSIDERATIONS i. SYSTEM OVERVIEW
The present invention is directed to a system and method for knowledge retrieval, management and delivery. This system and method is referred to herein by the trademarked - term Information Nervous System™. With reference to FIGURE 7, at its highest level the system 10 includes a server 20 comprised of several components that work together to provide context and time-sensitive semantic information retrieval services to clients 30 operating a presentation platform (e.g., a browser) via a communication medium 40, such as the Internet or an intranet. The server components preferably include a Knowledge
Integration Server (KIS) 50 and a Knowledge Base Server (KBS) 80, which may be physically integrated or separate. Within the system, all objects or events in a given hierarchy are active Agents 90 semantically related to each other and representing queries (comprised of underlying action code) that return data objects for presentation to the client according to a predeteπnined and customizable theme or "Skin." This system contemplates, wide variety of applications, as well as various means for the client to customize and "blend" Agents and the underlying related queries to optimize the presentation of the resulting infomiation. Each of the preferred components of the system 10 of the present invention, as well as the interaction among the components, is described in greater detail below.
2. SYSTEM ARCHITECTURE The end-to-end system architecture for the Infoπnation Nervous System of the present invention is shown with reference to FIGURE 8. FIGURE 8 illustrates how the present invention provides multiple client access means of communication between the Infomiation Nervous System XML Web Service (KIS) and Smart Agents. In the prefened embodiment, this occurs via the Information Agent. In an alternative embodiment, the communication may occur programmatically via an Enteφrise Knowledge Portal (e.g., Today's Web access browser) or via an SDK layer that enables programmatic integration with a custom client.
The system architecture for the KIS of the Infonnation Nervous System, including components thereof, are shown with reference to FIGURE 9. These components are described in greater detail below.
3. TECHNOLOGY STACKS
The significant differences between Today's Web and the conceptual Semantic Web are further highlighted by reference to the technology stacks of each as shown with reference to FIGURE 10. FIGURE 10 is a side-by-side comparison of the high-level descriptive platform layers of Today's Web and the equivalents (where applicable) in the Information Nervous System of the present invention. FIGURE 10 illustrates how scenarios in Today's Web map to scenarios in the Information Nervous System in certain instances, thus providing users with a logical migration path, but also highlights aspects of the Infomiation Nervous System that do not exist in Today's Web. 4. SYSTEM HETEROGENEITY
Heterogeneity is an advantage of the present invention. In the prefened embodiment, the KIS Agency XML Web Service is portable. This means that it supports open standards such as XML, XML Web Services that are interoperable (e.g., that employ the WS-I standard for interoperability), standards for data storage and access (e.g., SQL and ODBC/JDBC) and standard protocols for the infoπnation repositories from which the DSAs gather data (e.g., LDAP, SMTP, HTTP, etc.), etc.
For. example, in a prefened embodiment, a KIS (on which an Agency is running) is able to: • Gather its "people" metadata from an LDAP store (using an LDAP DSA). This allows it to support Microsoft's Windows 2000 Active Directory, Sun's Directory Server, and other Directory products that support LDAP. This is preferable to having a platform-specific Active Directory DSA that uses platform-specific APIs to gather "people" metadata. • Gather its email metadata from an SMTP store (for email from any source or for the system inbox). This allows it to support Microsoft Exchange, Lotus Notes, and other email servers (which support SMTP). This is preferable to having a platform-specific Microsoft Exchange Email DSA or a Lotus Notes Email DSA.
• Gather its "event" metadata from a calendar store supporting an open standard like iCalendar and use a protocol such as Calendar Access Protocol (CAP). This allows it to support any event repository that supports the iCalendar or CAL protocol standard. This is preferable to having a platform-specific Microsoft Exchange Calendar (or Event) DSA, a Lotus Notes Calendar DSA, etc.
In an alternative embodiment, the KIS Agency may be configured to extract metadata stored in a proprietary repository (via an appropriate DSA).
To achieve heterogeneity, in the prefened embodiment, for client-server communications, the system 10 uses XML Web Service standards that work in an interoperable manner (across platforms). These include appropriate open and interoperable standards for SOAP, XML, Web Services Security (WS-Security), Web Services Caching (WS-Caching), etc. In the preferred embodiment of the present invention, the semantic browser (also referred to by the trademarked teπn Information Agent™) is able to operate cross-platform and in different environments, such as Windows, .NET, J2EE, Unix, etc. This ability is consistent with the notion of a semantic user experience in that users do not and should not care about what "platfomi" the browser is running on or what platform the Agency (server) is running on. The semantic browser of the present invention provides users with a consistent experience regardless whether they are "talking" to a Windows (or .NET) server or a J2EE server. Users are not required to take any extra steps while installing or using the client based on the platfoπn on which any of the Agencies they are interacting with is running. The Information Agent preferably uses open standards for its Skins and other presentation effects. These include standards such as XSLT, SVG, and proprietary presentation formats that work across platforms (e.g., appropriate versions of Flash MX/ActionScript).
A sample, heterogeneous, end-to-end implementation of a preferred embodiment of the Infomiation Nervous System of the present invention is shown with reference to FIGURE 11. FIGURE" 11 illustrates the prefened embodiment of the Information Nervous System and illustrates the heterogeneous, cross-platfoπn context for the present invention. The components shown in FIGURE 11 are described in greater detail below. 5. SECURITY The prefened embodiment of the Information Nervous System provides support for all aspects of security: authentication, authorization, auditing, data privacy, data integrity, availability, and non-repudiation. This is accomplished by employing standards such as WS-Security, which provides a platform, for security with XML Web Service applications. Security is preferably handled at the protocol layer via security standards in the XML Web Service protocol stack. This includes encrypting method calls from clients (semantic browsers) to servers (Agencies), support for digital signatures, authenticating the calling user before granting access to an Agency's Semantic Network and XML Web Service methods, etc.
The prefened embodiment that the present invention supports local (client-side) credential management. This is preferably implemented by requiring users to enter a list of their usemames and passwords that they use on multiple Agencies (within an Intranet) or over the Internet. The semantic browser aggregates infomiation from multiple Agencies that may have different authentication credentials for the user. Supported authentication credentials optionally include common schemes such as basic authentication using a usemame and password, basic authentication over SSL, Microsoft's .NET Passport authentication service, the new Liberty Alliance authentication service, client certificates over SSL, digest authentication, and integrated Windows authentication (for use in Windows environments).
In the prefened embodiment, with the users' credentials cached at the client, the semantic browser uses the appropriate credentials for a given Agency by checking the supported authentication level and scheme for the Agency (which is part of the Agency's schema). For example, if an Agency supports integrated Windows authentication, the semantic browser invokes the XML Web Service method with the logon handle or other identifier for the current user. If the Agency supports only basic authentication over SSL, the semantic browser passes either the usemame and password or a cached copy of the logon handle (if the client was previously logged on and the logon handle has not expired) in order to logon. The prefened embodiment employs techniques such as logon handle caching, aging and expiration on the KIS in order to speed up the authentication process (and logon handle lookups) and in order to provide more security by guarding against hijacked logon handles.
The Agency XML Web Service preferably supports different authentication schemes either implicitly (if the feature is natively supported by the server operating system or application, server) or at the application-level by the XML Web Service implementation itself. Alternative embodiments of the KIS Agency's XML Web Service preferably employ a variety of authentication schemes such as basic authentication, basic over SSL, digest, integrated Windows authentication, and client certificates over SSL, and integrated .NET passport authentication. 6. EFFICIENCY CONSIDERATIONS
Client-Side and Server-Side Query and Object Caches. The present invention provides for query caches, which are responsible for caching queries for quick access. On the client, the client-side query cache caches the results of SQML queries with specified arguments. The cache is preferably configured to purge its contents after a predetermined amount of time (e.g., a few minutes). The amount of time is preferably set by modeling system usage and aniving at an optimal value for the cache time limit. Other parameters may also be considered, such as the data anival rate on the Agency (in the case of per-Agency caches, which is another implementation option), the usage model (e.g., navigation rate) of the user, etc. Caching improves performance because the client does not have to needlessly access recently used servers as the user navigates the semantic environment. In the prefened embodiment, the client employs standard XML Web Service Caching technologies (e.g., WS-Caching). In- addition, on the client, there is preferably an object cache. This cache caches the results of each SQML resource and is tagged with the resource reference (e.g., the file path, the URL, etc.). This optimizes SQML processing because the client can get the XML metadata for an SQML resource directly from the object cache, without having to access the resource itself. The resource may be the local file system, a local application (e.g., Microsoft Outlook), or an Agency's XML Web Service. Like the query cache, the . object cache may be configured to purge its contents after a set amount of time (e.g., a few minutes). In an alternative embodiment, on the server, the -server-side query cache caches the category results for XML arguments. This speeds up the query response time because the server does not have to ask the KDM to categorize XML arguments (via the one or more instances of the KBS that the KIS is configured to get its domain knowledge from) on each query request. In addition, the server can cache the SQL equivalents of the SQML arguments it receives from clients. This speeds up the query response time because the server would not have to convert SQML arguments to SQL each time it receives a request from a client. In the prefened embodiment, aggressive client-side caching is, employed and server-side caching is avoided unless it clearly improves performance. This is because client-side caching scales better than server-side caching since the client caches requests based on its local context.
Virtual, Distributed Queries. The present invention employs virtual, distributed queries. This is consistent with its "dynamic linking" and "user-controlled browsing" functionality. The system does not require static networks that link — or massive individual databases that house — all the metadata for the system. This precludes the need for manual authoring and maintenance on a local or global scope. In addition, this precludes the need for integrated (or universal) storage, wherein all the metadata is required to be stored on a'single metadata store and accessible through one database query interface (e.g., SQL). Rather, the present invention employs the principle of "Dynamic Access" via its use of XML Web Services to dynamically distribute queries across various Agencies (in a context and time- sensitive manner), and to aggregate the results of those queries in a consistent and user- friendly manner on the client. D. SYSTEM COMPONENTS AND OPERATION 1. AGENCIES AND AGENTS The present invention introduces a unique approach to using Agencies and Agents to retrieve, manage and deliver knowledge. a. Agencies
In a prefened embodiment of the present invention, the Agency is an instance of the
Knowledge Integration Server (KIS) 50 and is the invention's equivalent of a Web site. An
Agency is preferably installed as a Web application (on a Web server) so as to expose XML Web Services. An Agency will preferably include an Agency administrator. In a prefened embodiment of the present invention, an Agency has the following primary components:
• A flag indicating whether the Agency supports or requires authentication (or both). If the Agency requires authentication, the Agency will require basic user infomiation and a password and will store infoπnation on the type of authentication it supports. For Agencies that store user infoπnation, the Agency will also require user subscription infomiation (for subscription to Agents on a specific Agency). o Structured stores of semantic objects (documents, email messages, etc.) —
Conesponding to schemas for the respective classes. • Runtime components that respond to semantic queries — Components return XML to the calling application and provide system services for all the information retrieval features of the semantic browser.
Server-Side User State. In the prefened embodiment of the present invention, Agencies support server-side User State, which associates related concepts including "people" metadata and user authentication. Server-side User State facilitates many of the implementation details of the present invention, including the storage of user favorites (by semantic links between people objects and infoπnation objects), the inference of favorites in order to generate new links (e.g., recommendations), Annotations (that map users' comments to information objects), and the inference of "experts" based on semantic links that map users to information (e.g., posted emails, annotations, etc.). Server-side User State is preferably used with some Context Templates like "Experts," "Favorites," Recommendations," and "Newsmakers."
Client-Side User State. The Information Agent (semantic browser) preferably supports roaming of local client-side User State. This includes users' Semantic Environment and users' credentials (securely transfened). In the prefened embodiment, users are able to easily export their client-side User State to another machine in order to replicate their Semantic Environment onto another machine. This is preferably achieved by transfemng users' Agent list (recent and favorites), the metadata for the Agents (including the SQML buffers), users' local security credentials, etc. to an XML foπnat that serializes all this state and enables the state to be easily transfened. Alternatively, an XML schema may be developed for all the local client-side User State. Caching the User State on a server and synchronizing the User State using common synchronization techniques can also facilitate roaming. The semantic browser preferably downloads and uploads all client-side User State onto the server, rather than storing the state locally (in an XML file or a proprietary store like the Windows registry). b. Agents
An Agent is the main entry point into the Semantic Network of the present invention. An Agent preferably consists of a semantic filter query that returns XML infomiation for a particular semantic object type (e.g., documents, email, people, etc.). In other words, an Agent is preferably configured with a speci ic object type (described below). Agents can also be configured with a Context Template (described below). In this case, the query will return an object type, but it will incoφorate the semantics of the Context Template. For example, Agents configured with a "Headlines" Context Template will be sorted by time and relevance, etc. Agents are also used to filter notifications, alerts and announcements. Agents can be given any name. However, in the prefened embodiment of the present invention, the naming foπnat for most Agents is:
<Agentobjecttype>.<semanticqualifier>.<semanticqualifier>
Agents can be named arbitrarily. However, examples of Agent names include: A11.A11
Email.All Documents. Technology. Wireless.8021 IB. All Events.Upcoming.NextThirtyDays.All
There will also be Domain Agents (see below) that may follow a different naming convention (see below). At the semantic browser of the present invention, a fully qualified Domain Agent name will have the foπnat:
<Agentobjecttype>.<semanticdomainname>.<categoryname> [Agency=<Agency url>, kb=<kb url>]
For example, the Email Domain Agent on the Agency http://research.Agency.asp configured with the category wireless. all from the knowledge-base ABC.com/kb.asp with the semantic domain name industries. informationtechnology will be fully named as:
Email.Industries.InfoπnationTechnology.Wireless.All [Agency=http://research/Agency.asp, kb="http://abccoφ.com/kb.asp"
The semantic browser of the present invention is preferably configurable to use only the Agent name or to include the "Agency" and "kb" qualifiers. Agent Types. There are three primary types of Agents created on server 20: Standard
Agents, Compound Agents, and Domain Agents. A Standard Agent is a standalone Agent that encapsulates structured, non-semantic queries, i.e., without domain knowledge (or an ontology/taxonomy mapping). For example., on the server, the Agent All.PostedToday.All is a simple Agent that is resolved by filtering all objects based on the CreationTime property. Standard Agents can also be more complex. For example, the Agent All.PostedByAnyMemberOfMyTeam.All may resolve into a complicated query that involves joins and sub-queries from the Objects table and the Users table (see below).
A Compound Agent contains other Agents and allows the Agency administrator to create queries that generate results that are the UNION or the INTERSECTION of the results of their contained Agents (depending on the configuration). Compound Agents can also contain other Compound Agents. In the presently prefened embodiment, Compound Agents contain Agents from the same Agency. Flowever, the present invention anticipates the integration of Agents from different Agencies. By way of example, a Compound Agent All.Technology.Wireless.All might be created by compounding the following Agents:
• Documents.Technology.Wireless.All
• Email.Technology.Wireless.All • People.Experts.Technology.Wireless.All
'■ . As described above, a Domain Agent is an Agent that belongs to a semantic domain.
A Domain Agent is initialized with an Agent query, just like any other Agent. However, this query includes the CATEGORIES table, which is populated by the Knowledge Domain
Manager (see below). While the prefened embodiment of the present invention utilizes a KBS 80 having proprietary ontologies conesponding to a private Semantic Environment, the present invention contemplates integrated support of ontology interchange standards that will enable an Agency to connect to one or more custom private KBS, for example within an organization where the Agency was previously initialized with a proprietary ontology for that organization. An example of a Domain Agent is Email.Technology.Wireless.All. This Agent is preferably created with a knowledge source URL such as: category://technology. wireless. all@ABC.com/marketingknowledge.asp
This knowledge source URL conesponds to the Technology. Wireless.All category for the default domain on the knowledge base installed on the ABC.com marketingknowledge.asp Web service. This is resolved to the following HTTP
URL: http://ABC.com/marketingknowledge. asp?category="technology.wireless. all." In this example, a fully qualified version of the category URL may be: category://technology.wireless.all@abccoφ.com/marketingknowledge.asp?se manticdomainname="ιnformationTechnology" In this case, the category URL is qualified with the domain names.
Domain Agents are preferably created via a Domain Agent Wizard, and the Agency administrator is able to add Domain Agents from the KBS 80 to the Semantic Network of the present invention. The Domain Agent Wizard allows users to create Domain Agents for specific categories (using a category URL) or for an entire semantic domain name. In the latter case, the Agency is preferably configured to automatically create Domain Agents as new categories are added to the semantic domain on the KBS. This feature allows domains and categories to remain dynamic and therefore easily adaptable to the user's needs over time. When Domain Agents are managed in this fashion, the Agency is configurable so as to remove Agents that are no longer in the semantic domain. Essentially, in this mode, the
Domain Agents are synchronized with the CATEGORIES table (which in turn is synchronized with the CATEGORIES list at the relevant KBS by the Knowledge Domain Manager, described below).
A Domain Agent is initialized with a structured query that filters the data the Agent manages based on a category name or URL. In this situation, the structured query is identical to the queries for Standard Agents. An example of a resultant query for a category Agent is:
SELECT OBJECT FROM OBJECTS WHERE OBJECTID IN (SELECT OBJECTID FROM SEMANTICLINKS WHERE PREDICATETYPEID=50
AND SUBJECTID=1000 AND OBJECTID IN (SELECT OBJECTID FROM CATEGORIES WHERE URL LIKE category://technology. wireless. all@ABC.com/kb.asρ?domain- 'marketing")) "
In this example, the "belongs to the category" predicate type ID is assumed to have the value 50, and the category objectid is assumed to have the value 1000. This query can be translated to English as follows:
Select all the objects in the Agency that belong to the category whose object has an objectid value of 1000 and whose URL is category://teclmology.wireless.all@abccoφ.com/kb.asp?domain="marketing" This in turn translates to:
Select all the objects in the Agency of the category category://technology.wireless.all@abccoφ.com/kb.asρ?domain- 'marketing" The Domain Agent Wizard asks the user whether he or she wants to name the Agent based on the short category name or a friendly version of the fully qualified category name. An example of the latter is: Marketing.Technology.Wireless.All [@ABC]. The fully qualified Domain Agent naming convention is: <objecttypename>.<semanticdomainname>.<categoryname>.all [@KB Name].
In this example, the Domain Agent name is:
Email.Marketing.Technology.Wirel.ess.All [@ABC].
Blenders. Blenders are users' personal super-Agents. Users are able to create a Blender and add and remove Agents (across Agencies) to and from the Blender. This is analogous to users having their own "Personal Agency". Blenders are preferably invoked only on the system client since they include Agents from multiple Agencies. The client of the present invention aggregates all objects from a Blender's Agents and presents them appropriately. Blenders preferably include all manipulation characteristics of other types of Agents, e.g., drag and drop, Smart Lens (see below). A Blender can contain any type of Agent (e.g., Standard Agents, Search Agents, Special Agents, as well as other Blenders).
The present invention provides for a Blender Wizard, which is a user interface designed to facilitate users in creating Blenders. FIGURES 12-14 show exemplar screenshots of aspects of the Blender Wizard user interface according to a prefened embodiment of the present invention. FIGURE 12 is a sample Information Agent screenshot showing a Tree View of a sample Semantic Environment and a sample of the "Add Blender" wizard that allows users to create and manage a new Blender. FIGURE 13 shows the second page of the Add Blender wizard where users enter the name and description of the Blender and optionally select information object type filters. FIGURE 14 shows the third page of the sample Add Blender wizard in accordance with a prefened embodiment of the present invention. In this example, users add and remove Agents from the Semantic Environment to or from the Blender. When the "Add Agents" option is selected, the "Open Agent" dialog is displayed from which users can add a new Agent, Blender or Agency to the new Blender.
Breaking News Agents. A Breaking News Agent is a specially tagged Smart Agent. In addition to the option of having time-criticality being defined by the Agency administrator, the user has the option of indicating which Agents refer to infonnation that he or she wants to be alerted about. Any information being displayed will show alerts if there is breaking news that relates to it on a Breaking News Agent. For example, a user will be able to create an Agent as: "All Documents Posted on Reuters today" or "All Events relating to computer technology and holding in Seattle in the next 24 hours" as Breaking News Agents. This feature functions in an individual way because each Breaking News Agent is personal ("breaking" is subjective and depends on the user). For example, a user in Seattle perhaps would want to be notified on events in Seattle in the next 24 hours, events on the West Coast in the next week (during which time he or she can find a cheap flight), events in the United States in the next 14 days (the advance notice for most U.S. air caniers to get a modestly priced cross-continental flight), events in Europe in the next month (likely because he or she needs that amount of time to get a hotel reservation), and events anywhere in the world in the next six months.
In a prefened embodiment, the present invention automatically checks the Semantic Environment for breaking news by querying each Breaking News Agent or by querying the "Breaking News" Context Template. It will do this for all objects displayed in the semantic browser window. If a Breaking News Agent indicates that there is breaking news, the Infonnation Agent object Skin so indicates by flashing the window or by showing a user interface that clearly indicates that there is an alert that relates to the object. When the user clicks on the breaking news icon, a breaking news pane or a Context Palette for the "Breaking News" Context Template is displayed allowing the user to see the breaking news, select the Breaking News Agent (if there are multiple with breaking news), select predicates, and select other options. An exemplar pane of a Breaking News Agent user interface is shown in FIGURE 15. This sample user interface illustrates the popup menu in the context Results Pane. The sample shows a similar context pane as a Smart Lens (Agent- Object) popup context Results Pane (discussed below) except that the Agent is a Breaking News Agent.
Default Agents, hi an alternative embodiment, each Agency exposes a list of default Agents. Default Agents are similar to the default page on a Web site; authors of the Agency i determine which Agents they want users to always sees. Alternatively, on the client, Default Agents may be invoked when users click on the root of the Infonnation Agent's Environment (which preferably conesponds to a "Home Agent," for example, the equivalent of the "Home Page" on Today's Web browser). Combined Default Agents may also be configured by users.
Default Special (or Context) Agents. In the prefened embodiment, the client or the Agency support a Default Special or Context Agent that maps to each Context Template (discussed below). These Agents preferably use the appropriate Context Template without any filter. For example, a Default Special Agent called "Today" returns all items on all Agencies in the "recent" and "favorites" lists (or on a configured list of Agencies) that were posted today. In yet another example, the Default Special Agent called "Variety" shows random sets of results for every Agency in the Semantic Environment conesponding to the "variety" Context Template.
Default Special Agents preferably function as a starting point for most users to familiarize themselves with the Infomiation Nervous System of the present invention. In addition, Default Special Agents retain the same functionality as Smart Agents, such as use of drag and drop, copy and past, Smart Lens, Deep Information, etc. Horizontal Decision Agents, hi the prefened embodiment, Agents utilized by the client to assist with user interaction, including: • Schedule Agent: The Schedule Agent intelligently ranks events based on the probability that particular users would want to attend the event.
• Meeting Follow-up Agent: The Meeting Follow-up Agent intelligently notifies users when the time has come to have a follow-up meeting to one that occuned in the past. The Inference Engine (see below) monitors relevant semantic activity to deteπnine whether enough change has occuned to wanant a follow-up meeting. Users preferably use the previous meeting object as an Information Object Pivot to find the relevant knowledge changes (such as new documents, new people that might want to attend, etc.) • Task Follow-up Agent. The Task Follow-up Agent sends recommendations to users in response to tasks users perfoπn (such as reading a document, adding an event to their calendar, etc.). The Agent ensures that users have constant follow- up. The recommendations are based on users' profile, and the Agent preferably uses collaborative filtering to determine recommendations. o Customer Follow-up Agent. The Customer Follow-up Agent sends notifications to users based on customer activity. The Agent intelligently detennines when the user needs attention (based on email received from the user, new documents that might aid user service, etc.)
Public versus Local Agents. Agents that are created by the Agency administrator are "Public Agents." Agents created and managed by users are "Local Agents." Local1 Agents can refer to remote Agencies via SQML that includes references to Agency XML Web
Service URLs, or can refer to local Agencies that ran a local instance of the KIS with a local metadata store.
Saved Agents — Users' My Agents List. In the prefened embodiment, users are able to save a copy of an inyoked Agent or a query result as a local Agent. For example, users may drag and drop a document on their hard drive to an Agent folder to generate a semantic relational query. Users could save that result as an Agent named
"Documents.Technology.Wireless.RelatedToMyDocument." This will then allow the user to navigate to that Agent to see a personalized semantic query. Users would then be able to use that Agent to create new personal Agents, and so on. Personal Agents can also be "published" to the Agency. Other users are preferably able to discover the Agent and to subscribe to it.
In the prefened embodiment, a local Agent is created by a "Save as Agent" button that appears on the client anytime a semantic relational query result is displayed. This is analogous to users saving a new document. Once the Agent is saved, it is added to the users' My Agents list. An Agent responds to a semantic query based on the semantic domain of the Agency on which it is hosted. Essentially, a semantic query to an Agent is analogous to asking whether the Agent "understands the query." The Agent responds to a query to the best of its "understanding." As a further illustration, an Agent that manages "People" responds to a semantic query asking for experts for a document based on its own internal mapping of people in its semantic domain to the categories in that domain.
Alternatively, the system client may be configured ,to use non-semantic queries. In this case, the Agency will use extracted keywords for the query. All Agents support non-semantic queries. Preferably only Agents on Agencies that belong to a semantic domain will support semantic queries. In other words, semantic searches degrade to searches.
Each Agent has an attribute that indicates whether it is "smart" or not. A Smart Agent is preferably created on an Agency if that Agency belongs to a semantic domain. In addition, a Smart Agent only returns objects it fully "understands." In the prefened embodiment, when an Agency is installed, there are several default Smart Agents that the Agency administrator may optionally choose to install, including:
• All.Understood.All
• Documents.Understood.AU
• Email.Understood.All
For example, Email.Understood.All only returns email objects that the Agency can semantically understand based on its semantic domain (or ontology). 1 The present invention preferably includes the capability for users to display all objects and only those the Agency understands
Search Agents. A Search Agent is an Agent that is initialized with a search string. In the prefened embodiment, on invocation, the client issues the search request. A Search Agent is configurable so as to search any part of the Semantic Environment, including:
Frequently Used Agents Recently Used Agents Recently Created Agents Favorite • All [Saved] Agents
Deleted Agents
* Agents on the local area network o Agents on the Global Agency Directory
• Agents on any user-customized Agency directories • All Agents in the entire Semantic Environment
The client issues the search request based on the scope of the Search Agent. If users indicate that they want the search to cover the entire Semantic Environment, the client issues the request to all Agents in the Semantic Environment Manager (see below) and all Agents on the local area network, the Global Agency Directory and user-customized Agency Directories.
Server-Side Favorite Agents. In yet an alternative embodiment, the Agency supports User States support Favorite Agents. In the analogous context of Today's Web, a Web site allows users to customize their favorite links, stocks, etc. When initially queried, an Agency displays both its Default Agents and the Favorite Agents of the calling user (if there is a User State).
Smart Agents. A Smart Agent is a standalone Agent that encapsulates structured, semantic queries that refer to an Agency via its XML Web Service. In the prefened embodiment, user on the client are able to create and edit Smart Agents via a "Create Smart Agent" wizard that allows them to browse the Semantic Environment via the Open Agent dialog, and add links from specified Agencies. Essentially, this corresponds to users creating the SQML query from the user interface. In the prefened embodiment, the user interface only allows users to add links from the same Agency resource. However, users can create Agents of tlie same categories across Agencies, in. addition to Special Agents and Blenders (which are also preferably cross-Agency). The user interface allows the user to add links using existing Smart Agents as Infomiation Object Pivots provided that the Smart Agent refers to the same Agency for the cunent query. FIGURE 16 illustrates a prefened embodiment showing the Open Agent dialog with the user interface controls for selecting link (predicate) templates, the links themselves, and the objects. FIGURES 17-19 illustrate the Tree View of a sample Semantic Environment involving the Open Agent dialog. FIGURE 17 shows the Open Agent dialog allowing users to browse the Semantic Environment and open an Agent., FIGURE 18 illustrates a way of navigating Agencies in the Semantic Environment and the "Open Agent" dialog with the "Small Preview" view. FIGURE 19 illustrates an "Open" tool on the toolbar showing new options to open Agents form the Semantic Environment or to import regular infoπnation (e.g., from the file system) to the Semantic Environment by creating Dumb Agents.
The link templates essentially allow the user to navigate predicate for the cunent object type using predefined filters, thus allowing the user to avoid going through all the predicates for the object type. Examples of link templates include:
• All
• Breaking News (links that refer to time-sensitivity, e.g., "posted in the last")
• Categorization
• Definite (non-probabilistic links) • Probable (probabilistic links)
• Annotations In the prefened embodiment, the Open Agent dialog allows the user to select the object to "link to" and, depending on the type of the object, allows the user to browse the object (e.g., from a calendar control if it is a date/time, from a text box if it is text, from the file system if it is a file or folder path, etc.) The wizard user interface also allows the user to preview the results of the query. A temporary SQML entry is created with the cunent predicate list and that is loaded in a mini-browser window within the wizard dialog box. The user is able to add and remove predicates, and will also have the option of indicating whether he or she wants a union (an "OR") or an intersection (an "AND") of the predicates. The user interface will also check for duplicate predicates. Once the user finishes the wizard to create the Smart Agent, the Smart Agent is added to the Semantic Environment and the SQML is also saved with the associated object entry. In the prefened embodiment, the user can later browse the Smart Agent using the Agent property inspector property sheet. This allows the user to view the simple Semantic Environment properties (e.g., name, description, creation time, etc.) and also to view the resource URL (the WSDL URL to the XML Web Service of the Agency being queried) and the predicate list. The user can edit the list from the property sheet.
Default Smart Agent. A Default Smart Agent is similar to a Default Special Agent except that it is based on infoπnation object types and not on Context Templates. By way of example, "Documents" would return all documents on all Agencies in the users' Semantic Environment; "Email" would return all email messages in user's Semantic Environment, etc.
Special Agent. A Special Agent is a Smart Agent created by users based on a Context
Template (see below). A Special Agent is preferably initialized with an Agent name, albeit without a specific Agent reference. For example, a Special Agent
"Email.Technology.Wireless.All" may be created even if there are no Agents of that name in the Semantic Enviromnent. Like a Search Agent, a Special Agent is scoped to search for any Agent with its name on any part of the Semantic Environment. In the prefened embodiment, when a Special Agent is invoked by users, the client searches for any Agents that bear its name. If or when it finds any Agents with the name, the client invoke the Agent.
In the prefened embodiment, users enter parameters consistent with a Context Template, indicating the category fillers (if required) and what Agency(ies) to query. These can be manually entered using the Open Agent dialog, or users can indicate that they want to query the "recent" Agencies, "favorite" Agencies, or both. In an alternative embodiment, users have the choice of selecting categories (if required) that are in the union or intersection of the selected Agencies, or all categories known to the Global Agency Directory. In yet an alternative embodiment, users are able to select the information type (as opposed to a Context Template) and keywords to search (as opposed to predicates or categories).
Default Special Agents. In the prefened embodiment, the system client installs Default Special Agents that map to all supported Context Templates. By way of example, in the prefened embodiment, Default Special Agents including the following:
Headlines Breaking News
Conversations
Newsmakers
Upcoming Events
Discovery History
All Bets
Best Bets
Experts
Favorites Classics
Recommendati ons
Today
Variety
Timeline Upcoming Events Guide
Custom Special Agents. In contrast to user-created Special Agents, Custom Special
Agents are Special Agents specially developed and signed in order to guarantee that the
Special Agents are safe, secure, and of high-perfoπnance. The present invention provides for a plug-in layer to allow organizations and developers to create their own custom blenders. An example of a custom blender is "All.CriticalPriority.All that relates to my most recent documents or email." This Custom Blender may be implemented by an SQML file with a resource entry as follows:
<resource type= "nervana:url" agent://all.criticalpriority.all@localhost>
<link predicate= "nervana:relevantto" type= "nervana:localsemanticref ' recentdocuments > </link> <link operator= "or" type= "nervana:localsemanticref ' recentemail> </link> </resource> In the prefened embodiment, the Presenter (see below) resolves the "link" entry locally and initiates XML Web Service requests to the target resource with XML arguments conesponding to the newest documents or email messages. This allows the target Agent to focus on responding to semantic queries purely with XML filters without knowing the semantics related to filter origination. In an alternative embodiment, a Custom Blender such as the above example is a Default Agent.
Vertical Decision Agents. Vertical Decision Agents are Agents that provide decision- support for vertical industry scenarios.
Agent Schema. Agents operate within specified parameters and exhibit predetermined characteristics that comprise the Agent schema. Agent schemas may vary widely with being equally applicable within the technology of the present invention. By way of example only, the Agent schema of the prefened embodiment of the present invention is shown in FIGURE 20. The present invention specifically contemplates the addition of further fields. For example, fields for category URL (or path) and Context Template name can be added to the Agent schema to provide the client and server quick access to the category and Context Template the Agent represents (if applicable). This is helpful for the Semantic Environment Manager to provide different views of Agents (by category, by context, etc.). This complements the existence of these fields in the SQML for the Agent (expressed via attributes and/or predicates). The AgentTypelDs included in the prefened embodiment are shown in FIGURE 21. The AgentQueryTypelDs included in the prefened embodiment are shown in FIGURE 22.
In the prefened embodiment, SQL query formats are used. However, multiple query formats, for example XQL, XQuery, etc., are contemplated within the scope of the present invention. The KIS 50 preferably hosts an Agents table (for server-side Agents) in its data store conesponding to this schema. FIGURE 23 illustrates sample semantic queries that conespond to Agent names showing how server-side Agents are preferably configured on the KIS of the present invention.
As explained in greater detail below, Agents may optionally include their own Skins. An Agent Skin is represented as an URL to an XSLT file or equivalent Flash MX or ActionScript. If the Agent's Skin URL is not specified, a default Skin for the Agent's object type is presumed.
Agent Query Rules. Each server-side Agent query must be specified to return the OBJECTID column. Each table has this column for it is what links the Objects table with the tables for the derived object type. Objects and other tables are described in greater detail below. Because each Agent query can form the basis of a sub-query, cascaded query or a join, it is preferable that each query follow this foπnat. By way of example, the query for
News.All will be may appear as "SELECT OBJECTID FROM NEWS" (where "NEWS" is the name of the table hosting metadata for news articles, with the "news" schema). As a result, the server 10 can then use this query as part of a complex query. For example, if the user drags and drops a document onto the Agent, the server might execute this query as:
SELECT OBJECTID FROM NEWS WHERE OBJECTID IN (SELECT OBJECTID FROM SEMANTICLINKS WHERE SUBJECTID IN (50, 67, 89) AND LINKSCORE > 90) This example assumes that the document is classified to belong to categories in the
CATEGORIES table with object identifiers 50, 67, and 89 and that a link probability of 0.9 is the threshold to establish that a document belongs to a category. In this example, the document is used as a filter for the News.All query and the query text is used as part of the complex query. Having a consistent standard for queries allows the semantic query processor to merge queries until they finally have to be presented. For example, each call to the semantic query processor must indicate what object type in which to return the results. The query processor then returns XML infoπnation consistent with the schema for the requested object type. In other words, the query processor preferably returns schema-specific results for presentation. Each query is stored at the semantic layer (to retum an OBJECTID). To use the last example, when the user invokes the News.All Agent, the browser calls the query processor on the Agency XML Web Service. The query processor will then invoke the query and filter it with the 'News Article' object type, as such:
SELECT * FROM NEWS WHERE OBJECTID IN (SELECT OBJECTID FROM NEWS) This returns all the fields for the News schema. The browser (via the Presenter) displays the infonnation using the XSLT (or a presentation tool such as Flash MX or ActionScript) for either the Agent Skin or for a user-specified. Skin (which will ovenide the Agent Skin).
Query Virtual Parameters. Agent queries preferably contain special Virtual Parameter. A typical example may include: '%USERNAME%. hi this example, the Semantic
Query Processor (SQP) resolves the Virtual Parameter to a real argument before invoking the query., An Agent People.MyTeam.All is configured with the SQL query:
SELECT * FROM USERS WHERE Division IN (SELECT Division FROM USERS WHERE Name LIKE %USERNAME%) In this example, the Agent name includes "MyTeam" even though the Agent can apply to any user. The %USERNAME% variable is resolved to the actual calling user's name by the SQP. The SQL call is resolved to as follows:
SELECT * FROM USERS WHERE Division IN (SELECT Division FROM USERS WHERE Name LIKE JohnDoe) In this example, JohnDoe is assumed to be the user name of the caller.
Simple Agent Search. Each Agent will support simple search functionality. In the prefened embodiment, a user is able to right-click on a Smart Agent in the Information
Agent and hit "Search." This will bring up a dialog box where the user enters search text.
This creates the appropriate SQML with the associated predicate, e.g., "nervana ontains". The present invention provides a simple, fast way for users to search Agents (and create
Smart Agents from there) without going tlirough the "Create Smart Agent" wizard and selecting the "contains text" predicate (which alternatively achieves the same result).
Agency Agent Views. An alternative embodiment of the present invention includes
Agency Agent Views. An Agency Agent View is a query that filters Agents based on predefined criteria. For example, the Agent view "Documents" returns only Agents that manage, objects of the document semantic class. The Agent view "Reuters News" returns a list of Agents that manage news objects with "Reuters" as the publisher. Agency Agent Views are important in order to give users an easy way to navigate through Agents. The Agency administrator is able to create and delete Agent views.
Agent Publishing and Sharing. The prefened embodiment makes it easy for Agents to be published and shared. This is preferably implemented by serializing the Semantic Environment into an XML document containing the recent and Favorite Agents, their schema, their SQML buffers, etc. and publishing the document to a publishing point. This XML document may also be emailed to colleagues, friends, etc. in order to facilitate the propagation and sharing of local (user-created) Agents. This is analogous to how Web pages are published today and how web URLs and links are shared by sending links and attachments via email.
2. KNOWLEDGE INTEGRATION SERVER
The Knowledge Integration Server (KIS) 50 is the heart of the server-side of the system 10. The KIS semantically integrates data from multiple diverse sources into a Semantic Network and hosts Agents that provide access to the network. The KIS also hosts semantic XML Web Services to provide clients with access to the Semantic Network via Agents. To users, a KIS installation may be viewed as an Agency. The KIS is preferably initialized with the following properties:
• Agency Name. Name of the Agency (e.g., "ABC") • Agency Friendly Name. Full name of the Agency (e.g., "ABC Coφoration")
• Agency Description. Description of the Agency
• Agency System User Name. User name of the Agency. Each Agency is represented by a user on the directory of the enteφrise (or Web site) on which it is installed. The system user name is used to host the system inbox (tlirough which users will publish documents, email and annotations to the Agency). For authentication, the Agency must be installed on a server that has access to the system user account. • Agency Authentication Support Level. Indicates whether the Agency supports or requires user authentication. An Agency can be configured to not support authentication (in which case it is open to all users and does not have any User State), to support but not require authentication, and to require authentication, in which case it preferably indicates the authentication encryption type.
• Agency User Directory Type. This indicates the type of user directory the Agency authenticates users against and where the Agency gets its user infoπnation from. For example, this could be an LDAP directory, a Microsoft Exchange 2000 User Directory, or a Lotus Notes User Directory on the Windows 2000 Active Directory, etc.
• Agency User Directory Name. This indicates the server name of the Agency user directory (e.g., a Microsoft Exchange 2000 server name).
• Agency User Domain Name. This indicates the name of the user domain for authentication puφoses. This field is optional and included only if the Agency supports authentication.
• Agency User Group Name. This indicates the name of the user group for authentication puφoses. For example, an Agency might be initialized with the domain name "US Employees" and the group name "Marketing." In such a case, the Agency will first check the user name to ensure that the user is a member of the user group, and then forward authentication requests to the user directory authenticator indicated by the user directory type. If the calling user is not a member of the user group, the authentication request is denied. This field is only valid if the Agency supports authentication.
• Data Store Connection Name. This indicates the name of the connection to a database store. This could be represented as, say, an ODBC connection name on
Windows (or a JDBC name, etc.). The KIS will use the database refened to by the connection name to store, update, and maintain its tables (see below).
• Dynamic Properties Evaluation. The Agency XML Web Service preferably exposes methods to return dynamic properties such as the list of semantic domain paths the server cunently supports or "understands." This allows users to browse
Agencies on the client using their supported semantic domain paths or ontologies/taxonomies.
As illustrated with reference to FIGURE 24, the KIS 50 preferably includes the following main components: a Semantic Network 52, a Semantic Data Gatherer 54, a Semantic Network Consistency Checker 56, an Inference Engine 58, a Semantic Query Processor 60, a Natural Language Parser 62, an Email Knowledge Agent 64 and a Knowledge Domain Manager 66. a. Semantic Network The Semantic Network is the core data component of the KIS. The Semantic Network links objects of the defined schemas of the present invention together in a semantic way via database tables. The Semantic Network consists of schemas and the Semantic Metadata Store (SMS). The Semantic Network is preferably comprised of two data schemas: Objects and SemanticLinks. Additional data schemas may be included based on system requirements and enteφrise needs. The SMS is preferably a standard database (SQL Server, Oracle, DB2, etc.) where all semantic data is stored and updated via database tables. The SMS preferably includes tables for each primary object type (described below).
By way of example, a sample Semantic Network directed towards an enteφrise situation is shown with reference to FIGURE 25, which illustrates the relationship between business users of the present invention and the various sources of and results of knowledge retrieval, management, delivery and presentation.
Objects. The Objects table contains every object in the Semantic Network. The "Object" can be thought of as the "base class" from which every semantic object type will be derived. The prefened schema of the Object type is shown with reference to FIGURE 26. The ObjectlD is a unique identifier that tags the object in the Semantic Network. Every object in the system will have a schema that is an extension of the Object schema. Alternatively, semantic object types (e.g., document, email, event, etc.) will have only the ObjectlD field. When a query is invoked, the query processor can then aggregate infonnation from the Object table and the specific semantic table to form the final results. The former approach (having each schema be an extension of the Object schema) results in better runtime performance since joins are avoided. However, the latter approach, while computationally more expensive, results in less wasted storage. The ObjectTypelD is preferably a number that resolves to a string that describes the hierarchy of the object type, e.g., "documents\documents"; "documents\analyst briefs"; and "events\meetings."
The SourcelD refers to the identifier for the Semantic Data Adapter (SDA) from which the object was gathered. The Semantic Data Gatherer (SDG) uses this infoπnation to periodically check whether the object still exists by requesting status infomiation from the SDA from which the object was retrieved.
SemanticLinks. The SMS preferably includes a SemanticLinks schema (and corresponding database table) that will store semantic links. These links will annotate the objects in the other data tables of the SMS and will preferably constitute the data model for the Semantic Network. Each semantic link will have a semantic link ID. The SemanticLinks table preferably includes the field names and types as shown with reference to FIGURE 27. The SubjectID and SubjectTypelD are the object ID and object type ID of the object being linked from. The ObjectlD and ObjectTypelD are the object ID and object type ID of the object being linked to. The LinkScore preferably ranges from 0 to 100, and represents the semantic strength of the link as a probability. These fields are exemplary only; more predicates are contemplated based on the particular object type as well as the user's desire to semantic links. The prefened embodiment of the present invention provides the predicate type IDs shown in FIGURE 28. The present invention contemplates the addition of further predicate type IDs.
By way of example, the semantic link "Steve reports to Patrick" will be represented in the table with a subject ID conesponding to Steve's ID in the Users table, a predicate type of PREDlCATETYPEID_REPORTSTO (see table below), Patrick's object ID in the Users table, a link score of 100 (indicating that it is a "truth" and that the link is not probabilistic) and a Reference Date that qualifies the link. The KIS creates, updates, and maintains database tables for each object type (via the SMS). The following illustrates preferred but nonexclusive list of primary and derived object types:
• Person , • User
• Customer
• Category
• Document
• Analyst Brief • Analyst Report
• Case Study
• White Paper
• Company Profile o E-Book • E-Magazine
• Email Message
• Email Annotation o Email News Posting o Email Distribution List ® Email Public Folder
• Email Public Folder Newsgroup ,
• News Article
• Event Meeting • Coφorate Event
Industry Event TV Event Radio Event Print Media Event • Online Meeting
Arts and Entertainment Event Online Course Media
• Book • Magazine • Multimedia
• Online Broadcast
• Online Conference Object types are preferably expresses as hierarchical paths. The path can be extended, e.g., "events\meetings" can be extended with "qualified Meetings," e.g., "events\meetings\company meetings." This schema model is indefinitely extensible and configurable.
Virtual Information Object Types. Virtual Information Object Types are object types that do not map to distinct object types, yet are semantically of interest to users. An example is the "Customer Email" object type, which derives from the "Email" object type. This object type is "virtual" in that it does not have a distinct schema and, as a consequence, does not have a distinct table in the SMS on the KIS. Rather, it uses the "Email" table on the SMS, since it derives from the "Email" object type. Even though it is not a distinct object type, users will be interested in browsing and searching for "Customer Email" as though it were indeed distinct.
In the prefened embodiment, Virtual Object Types are implemented by storing the metadata in the appropriate table on the SMS (in this case, the "Email" table, since the object type derives from "Email"). However, the resolution of queries for the object type is accomplished differently from regular queries for distinct object types. When the server SQP receives a semantic query request (via the. XML Web Service) for a virtual information object type (such as "Customer Email"), it resolves the request by joining the tables that together form the object type. For instance, in the prefened embodiment, in the case of
"Customer Email," the server will resolve in query with the SQL sub-query: SELECT OBJECTID FROM EMAIL WHERE OBJECTID IN (SELECT
OBJECTID FROM CUSTOMERS WHERE EMAILADDRESS IN (SELECT EMAILADDRESS FROM EMAIL) This query conesponds to "Select all objects from the Email table that have an email address value that is also in the Customers table." This assumes that "Customer Email" refers to email that is sent by or to a customer. Other definitions of the virtual object type are also possible and the query resolution is preferably consistent with the definition. The SQP preferably applies this sub-query to all queries for "Customer Email." This sub-query essentially filters the Email table for those email messages that are from customers. This returns the desired result to the user with the illusion that there is a "Customer Email" table when there really is not.
The present invention contemplates a variety of schemas associated with each object type. Other schemas are in development that will have comparable applicability to the present invention. The "Document" schema, for example, may be extended with fields from the Dublin Core schema (http://www.cis.ohio-state.edu/cgi-bin/rfc/rfc2413.html) and other industry standard schemas. hi yet another example, "News Article" schema may be an extension of the NewsML schema (http://www.newsml.org). By way of example only, prefened user object schema made in accordance with the present invention are shown with reference to FIGURE 29. All schemas preferably have as an identical subset the fields of the Object schema. MailingAddressTypelDs preferably associated with the User (person) object schema includes those shown with reference to FIGURE 30.
By way of example only, the preferred category object schema made in accordance with the present invention is shown with reference to FIGURE 31.-
By way of example only, the prefened document object schema made in accordance with the present invention is shown with reference to FIGURE 32. The "DocumentCategory" field refers to a proprietary category that is tagged with the document (by the document data source) and not to a semantic category managed by the KIS itself. The "DocumentFonnatTypelD" field refers to the type of document. The Print Media Type IDs of the prefened embodiment are shown in FIGURE 33, and the prefened FORMATTYPEID are shown in FIGURE 34.
By way of example only, the prefened email message list object schema made in accordance with the present invention is shown with reference to FIGURE 35. Email Priorities are preferably 0, 1, or 2, conesponding to low, medium, and high priority. The E ailTypelD preferably includes EMAILTYPEID_EMAIL,
EMAILTYPEID_NEWSPOSTLNG and EMAILTYPEID_EMAILANNOTATION
(values 1, 2 and 3). Exemplar tables showing the email distribution list and email public folder object schemas of a prefened embodiment of the present invention are shown in FIGURES 36 and 37, respectively. In the prefened embodiment, the PublicFolderTypelD includes those shown in FIGURE 38.
By way of example only, the prefened event object schema message list object schema made in accordance with the present invention is shown with reference to FIGURE 39. FIGURE 40 shows the events types of a prefened embodiment of the present invention.
By way of example only, the prefened media object schema message list object schema made in accordance with the present invention is shown with reference to FIGURE 41. FIGURE 42 shows the media types of a prefened embodiment of the present invention. By way of example, FIGURES 43-45 illustrate additional samples showing how objects are categorized and utilized in the prefened embodiment of the present invention. FIGURE 43 illustrates root object container types. FIGURE 44 illustrates a hierarchical schema for qualified object types. FIGURE 45 illustrates samples of native container object type predicates. All types except the Person and Customer types preferably inherit all predicates from the root type "All Information." The present invention provides for native container object type predicate templates, for example including for: All; Breaking News; Categorization; Author; Annotations; Definite Links; Probabilistic Links; and Popular. b. Semantic Data Gatherer
In the prefened embodiment, the Semantic Data Gatherer (SDG) is responsible for adding, removing, and updating entries in the Semantic Network via the SMS. The SDG consists of a list of XML Web Service references. These form an Infomiation Source
Abstraction Layer (ISAL). Each of these references is initialized to gather data from via a
Data Source Adapter (DSA). A data source adapter is an XML Web Service that gathers infomiation from a local or remote semantic data source for a give object type. It then returns the XML conesponding to object entries at the data source. All DSAs preferably support the same interface via which the SDG will gather XML data. This interface includes methods to:
• Retrieve the XML metadata for objects for a given start and end index (e.g., objects 0 tlirough 49).
• Check whether there any objects have been added or deleted since a particular date/time (on the DSA's time clock). o Fetch the XML metadata for objects added or deleted since a particular date/time (on the DSA's time clock)
• Check whether an object still exists in the semantic data source - by examining the XML metadata for the object (passed as an argument) If each call to the DSA XML Web Service will be stateless, the API should' include infoπnation, preferably via a string with command parameters, which qualifies the request. For example, a DSA for an email inbox includes parameters such as the name of the user whose inbox is to be gathered. A DSA for a Web site or document store will have to include information on the URL or directory path to be crawled. Each DSA is required to retrieve infomiation in the schema for its object type.
Because a DSA must be implemented for a particular object type, the SDG will expect XML for the schema for that object type when it invokes a gather call to the DSA. The SDG is responsible for maintaining the integrity and consistency of all the database tables in the SMS (the Semantic Network). In this embodiment, the SDG is also refened to as a Semantic Network Manager (SNM). The database tables preferably do not contain redundant or stale entries. Because the SDG retrieves objects with well-known schemas the semantics of each of the object types is understood, and the SDG maintains the consistency of the tables accordingly. For example, the SDG preferably does not add redundant Document XML metadata to the DOCUMENTS table. The SDG uses the semantics of documents to check for redundancy. In the prefened embodiment this is accomplished by comparing the author name, creation date/time, file path, etc. The SDG also perfomis this check for other tables (e.g., EVENTS, CUSTOMERS, NEWS, etc.). For example, the SDG will perfoπn redundancy checking for events by examining the title, the location, and the date/time. Other tables are maintained accordingly. The SDG will also update objects in the database tables that have been changed.
The SDG is also preferably responsible for cleaning up the database tables. The SDG periodically queries the DSA to determine whether all of the objects in each table managed by the DSA still exists. For example, for a DSA that retrieves documents, the SDG will pass the XML metadata to the DSA Web service and query whether the object still exists. The DSA attempts to open the URL for the document. If the document does not exist anymore, the DSA will indicate this to the SDG. Individual DSAs, and not the SDG, are responsible for object validation to avoid security restrictions that are data source specific. For example, there might be data source restrictions that prevent remote access to local resources. In such a case, only the DSA XML Web Service (which is preferably running locally, relative to the data source) will have access to the data source. Alternatively, some DSAs might run on the Agency server, alongside the SDG and other server components, and retrieve their data remotely. Having the DSAs handle object validation also provides additional efficiency and security in that the DSA prevents the SDG from knowing the details ofhow to open each data source to check whether an object still exists. Since the DSA needs to know this (since it retrieves the XML data from the data source and therefore has code specific to the data source), it is more appropriate for the DSA to handle this task.
The SDG preferably maintains a gather list that will point to DSA XML Web Service
URLs. The KIS administrator is able to add, delete, and update DSA entries from the SDG gather list. Each gather list entry is preferably configured with:
1. The name and XML Web Service reference of the DSA. This essentially will refer to a combination of the data source, the object type, and a reference to the XML
Web Service that implements the DSA (e.g., via a WSDL web service URL). Examples include: a. Microsoft Exchange 2000 Email DSA. This DSA will gather email XML metadata from a Microsoft Exchange 2000 Inbox or Public Folder b. Microsoft Exchange 2000 Calendar DSA. This DSA will gather event
XML metadata from a Microsoft Exchange 2000 Calendar c. Microsoft Exchange 2000 Users DSA. This DSA will gather users/people XML metadata from a Microsoft Exchange 2000 Directory d. Microsoft Exchange 2000 Email Distribution List DSA. This SDA will gather email distribution list metadata from a Microsoft Exchange 2000
Directory e. Lotus Notes Inbox. This DSA will gather email XML metadata from a Lotus Notes Inbox or Public Folder f. Siebel CRM Database. This DSA will gather customer XML metadata - from a Siebel CRM system
. g. Web site. This DSA will gather document XML metadata from a Web site h. File Directory or Share. This DSA will gather document XML metadata from a file directory or share i. Saba E-Learning LMS Repository. This DSA will gather E-Leaming XML metadata from a Saba Learning Management System (LMS) repository j. Microsoft Sharepoint Document DSA. This DSA will gather document XML metadata from a Microsoft Sharepoint server workspace k. Reuters News Repository. This DSA will gather News Article XML metadata from a Reuters news article repository 2. The description of the DSA gather entry.
3. A string indicating initialization infoπnation for the DSA.
4. The gather schedule - this indicates how often the SDG should 'crawl' the DSA to gather XML metadata.
In a prefened embodiment, the Agency is initialized with a user directory domain and group name. In this case, the SDG preferably automatically enters a gather list entry for the user directory DSA. For example, if the Agency is configured with a Exchange 2000 User Directory with Domain Name "Foo" and Address Book or group name "Everyone," the SDG creates a gather list entry with the Exchange 2000 Users DSA (initialized with these parameters). Alternatively, the Agency can be configured to obtain its user directory from any email application server (e.g., Microsoft Exchange or Lotus Notes). The SDG initializes gather list entries with an Email Inbox and Calendar DSA for the system user (and Email Knowledge Agent, described below). These three gather list entry DSAs (Users, Inbox, and Calendar) are initialized by default. The Inbox is preferably used to store Agency email postings and annotation and the Calendar DSA is used to store events posted to the Agency by users. Other custom DSAs can be added by the Agency administrator.
The SDG also keeps track of the last time the SDA reported to it that objects have been added or deleted to or from the data source. This date/time information is preferably based on the SDA's clock. Each time the SDA reports that there is new or deleted data, the SDG will update the date/time information in its entry for the SDA and gather all the new or deleted information in the SDA. The SDG will then update the database tables.
The SDG preferably maps the XML infoπnation it receives from the SDAs to the Semantic Network of the present invention. The SDG stores all the XML metadata in the database tables in the SMS. hi addition, the SDG parses the XML it receives from the SDA and, where necessary, maps semantic links to specific XML fields. The SDG adds or updates semantic links in cases where the XML includes infonnation that "links" objects together. For example, the schema for an email object preferably includes fields including "From," "To," "Cc," "Bcc," and "Attachments." In the case of the "From," "To," "Cc" and "Bcc" columns, the fields in the XML refer to email addresses (separated by delimiters such as ";" or "," or a space). In the case of the "Attachments" column, this field will refer to the file paths of the files that are attached to the email message (separated by delimiters such as ","). This raw XML is stored in the EMAIL database table, along with the other columns. In addition, the SDG parses the fields of the email object and adds semantic links to other objects that are identified by the contents of those fields. For example, if the "to" field contains "john@foo.com" and the attachments field contains the string "c:\foo.doc, c:\bar.doc," the SDG will process the email as follows:
1. Find any object in the USERS table with the email address "john@foo.com." Also, search for other USER objects with email addresses in the FROM, TO, CC, and BCC fields.
2. If any objects are found, add a semantic link entry to the SEMANTICLINKS table with the email object id as the subject and the appropriate predicate type id. In this case, the predicate PRED1CATETYPEID_CREAT0R refers to the originator of the email message. The predicate PREDICATETYPEID_SENTTO is used to link the email object and the USER objects refened to by the contents of the "to" field in the email XML metadata. The predicate PREDICATETYPEID_COPIEDTO and PREDICATETYPEID_BLΓNDCOPIEDTO are used to link objects in the "cc" and "bcc" fields in similar fashion.
In the case of attachments, the SDG extracts the XML metadata for the attached documents. If an XML object with the file path already exists in the SMS (or, in other words, the Semantic Network), the SDG will update the metadata. If the XML object does not already exist, the SDG creates a new document object with the XML metadata. The SDG will adds an entry to the SEMANTICLINKS table with the email object ID as the subject, the new document's object ID as the subject, and the predicate PREDICATETYPEID_ATTACHEDTO. This allows the user to be able to navigate from an email message to its attachments and then use the attachments as pivots to continue to browse the Semantic Network, for example using semantic tools like the Smart Lens (discussed below).
The SDG does not create any objects in the event for which it does not find user objects that match the entries in the XML fields. Preferably, the SDG gathers information from a Directory SDA when a user is manually added to the Agency. The Agency administrator preferably adds users to the Agency via the user group on the Agency properties.
The following illustrates an example of mapping raw email XML metadata to the
Semantic Network.
<email from- 'john@foo.com" to="nosa@nervana.ner' cc="steve@nervana.net" bcc- patrick@nervana.net" subject- 'Meeting this Friday" body="Let us meet on Friday at 2pm" attachments- 'cΛfoo.doc; c:\bar.htm" >
</email> is converted to the object graph illustrated in FIGURE 46. c. Semantic Network Consistency Checker
The Semantic Network Consistency Checker (CC) complements the consistency checking that is performed by the SDG. As described above, the SDG maintains the integrity of the database tables by precluding the addition of redundant entries into the Semantic
Network (from various data sources). The CC also ensures the consistency of the OBJECTS and SEMANTICLINKS tables. The CC periodically checks the OBJECTS table to ensure that each object exists in the native table (preferably by checking the OBJECTID field value). For example, a document object entry in the OBJECTS table preferably also exists in the DOCUMENTS table (with the same object ID). The CC removes any object in the OBJECTS table without a conesponding object in the native table (DOCUMENTS, EVENTS, EMAIL, etc.) and vice-versa.
The CC is also responsible for maintaining the consistency of the SEMANTICLINKS table. The semantics of this table are preferably as follows: A semantic link cannot exist if either its subject ("linked from") or its object ("linked to") do not exist. To illustrate this, if object A links to object B with predicate P, and either A or B is deleted, the link should be deleted. The CC periodically checks the SEMANTICLINKS table. If any of the subjects or objects has been deleted, the CC deletes the semantic link entry.
Consistency checks may be implemented in code in the KIS itself or as stored procedures or constraints at the database level. d. Inference Engine The Inference Engine is responsible for adding semantic links to the Semantic
Network. The Inference Engine employs Inference Rules, which consist of a set of heuristics, to add semantic links based on ongoing semantic activity. The Inference Engine is preferably allowed to remove semantic links. Decision Agents (described below) use the Inference Engine to assist knowledge-workers in making decisions. The Inference Engine operates by mining the Semantic Network and adding new semantic links that are based on probabilistic inferences. For example, the Inference Engine preferably monitors the Semantic Network and observes patterns in how email is sent, the type of email sent and by whom. The Inference Engine infers from this infoπnation background infoπnation, such as the expertise of the user, related to various subject matter categories within the monitoring purview of the Inference Engine. For example, the Inference Engine adds semantics links with the predicate PREDICATETYPEID_EXPERTON to indicate that a user is an expert in a particular category. The subject in this case will be a user object and the object will be a category object. To infer this, the Inference Engine is preferably configured to observe semantic activity for at least a certain period of time (e.g., two weeks), or to only infer links after users have sent at least a certain predetennined number of messages or authored a certain number of documents. The Inference Engine infers the new link by keeping statistics on the PREDICATETYPEID_CREATOR and
PREDICATETYPEID_CONTRIBUTOR links. *
By way of example, the Inference Engine may infer that users are an expert on a category if: • Of all categories of email messages they have written, this category is one of the top N (configurable). o They have written email messages on the same category an average of M times or more per week (configurable). • They have written at least O email messages (configurable) in the past P months (configurable).
More sophisticated inference models with which to accurately infer this data are contemplated. For example, probability distributions as well as statistical conelation models may be employed. Preferably these models will be developed on a per-scenario basis over time. The Inference Engine is also responsible for removing links that it might have added.
For example, if an employee changes jobs, he or she might "cease" to be an expert on a specific category (relative to other employees). Once the Inference Engine detects this (e.g., by observing email patterns), it removes semantic links that indicate that the person is an expert on the category. Infened semantic links are important for scenarios that involve probabilistic semantic queries. For example, in one embodiment of the present invention, using the Information Agent, users may drag and drop a document from their file-system onto an Agent (say, People.Research.All). In this case, users will want to know the people in the Research department that are experts on the document. The browser will then invoke an SQML query with the Agent as resource (or subject), the predicate nervana:experton, and the document path as the object. The Presenter will then retrieve the XML metadata for the document and call the XML Web Service, residing on the Agency that hosts the Agent, with the predicate ID and the document's XML metadata as arguments. The server-side semantic query processor on the Agency processes this XML Web Service call and translates the call to a SQL query consistent with the data model of the Semantic Network. In this example, the call is preferably resolved as follows: 1. For all semantic domain entries in the KDM, call the conesponding KBS to categorize the document.
2. Map the returned categories to category objects in the Semantic Network (by comparing URLs)
3. Invoke a query using the query of the People.Research.All Agent as a sub-query. In this example, the final query appears as follows:
SELECT * FROM USERS WHERE DEPARTMENT LIKE "RESEARCH" AND OBJECTID LN (SELECT OBJECTID FROM SEMANTICLINKS WHERE OBJECTTYPEID = 32 AND PREDICATETYPEID = 98 AND SUBJECTID IN (SELECT OBJECTID AS SUBJECTID FROM- CATEGORIES WHERE OBJECTID IN (34, . 56, 78)) AND
LINKSCORE >.90 ) >
This query assumes that the object type ID for the user object type is 32, the predicate type
ID value for PREDICATETYPEID_EXPERTON is 98, the document belonged to categories with the object ID 34, 56, and 78 and that the semantic link score threshold is 90. e. Ser-ver-Side Semantic Query Processor
The server-side Semantic Query Processor (SQP) responds to semantic queries from clients of the KIS. The SQP is preferably the main entry point to the Semantic Network on the KIS (or Agency). The SQP is exposed via the Agency's XML Web Service. The SQP processes direct Agent semantic queries and generic (client-generated) semantic queries with semantic link filters (see below). For queries with server-side Agent filters, the Infonnation Agent passes the Agent name and object index arguments to the SQP to be invoked. For example, the browser may ask for objects 0-24 on Agent Documents.Teclinology.Wireless.All. In this example, the SQP looks up the Agent query in the Agents table and invokes the query onto the database that hosts the Semantic Metadata Store (SMS). The Agent query is preferably stored as SQL or another well-known query foπnat like XQuery or XQL. The SQP may convert the query format to a foπnat that the database (that holds all the tables) understands. Because most commercial databases understand SQL, it will preferably operate as the default Agent query foπnat.
The Agent query preferably follows the query rales described above. Therefore, the query returns the object ID rather than the schema fields for the Agent's object type. In the above-described example, Documents.Technology.Wireless.AH invokes the Agent query "SELECT OBJECTID FROM DOCUMENTS WHERE ..." The SQP is responsible for issuing a query that is filtered with the Agent query, but which returns the actual metadata for the object type (in this case, the "document" object type). In this example, the query appears as follows:
SELECT * FROM DOCUMENTS WHERE OBJECTID IN (SELECT OBJECTID FROM DOCUMENTS WHERE ...) This query returns the data columns for the "document" schema for all the objects with an object ID that matches those in the original Agent query. The SQP reviews the metadata results of the database query and translates them to well-formed XML using the appropriate schema for the object type of the Agent (in this case, "document"). In the event that the database supports raw XML retrieval, the SQP optimizes the query by asking the database to give it XML results. This results in better perfonnance since the SQP does not have to perfonn the extra translation step. The SQP passes the XML back to the caller via the
Agency's XML Web Service.
■•, The SQP preferably handles more complex queries that are passed by the semantic browser (or other client of the XML Web Service). By way of example, such queries may take the form of the following XML Web Service API:
String InvokeSemanti cQu ery(
Integer Beginlndex,
Integer Endlndex, String AgentName, '
Integer NumberOfLinks,
String OρeratorNames[],
String LinkPredicateNames[],
String LinkTypeNames[] String LinkObjectsf]);
In this example, the "[]" symbols refer to arrays. The API takes a zero-based begin index, a zero-based end index, an optional Agent name, an integer indicating the number of semantic links, an anay of operator names, an anay of link predicate names, an anay of link type names, and an anay of strings that refer to the link objects. If the Agent name is NULL (""), the SQP processes the query "as is"; without any preconceived Agent filter. This will be the case with queries that are wholly generated form the client. The anays are variable sized because the "NumberOfLinks" parameter indicates the size of each anay. The operator names include valid predeteπnined operators, including logical operators, which can be used to qualify queries in SQL or other query formats. Examples include teπn:or and term: nd. The link predicate names may include one or more predefined predicates (e.g., teπn:relevantto, term:reportsto, terrmsentto, terrmannotates, ternuannotatedby, ternuwithcontext, etc.).. The link type names indicate the type of link objects. Common examples include terrmurl and ternυobject. In the case of termmrl, the link object string refers to a well-foπned URL comprising objects://... or Agent://.... In the case of temv.object, the argument will be a well-fonned XML metadata instruction refening to a object defined within the present invention. This object is preferably resolved from the client or from another Agency. The API returns a string that contains the XML results (in addition to the return value for the XML Web Service method call itself).
By way of example, the SQML with the data:
<resource type="term:uri"
Agent://all.criticalpriority.all@abc.com/Agency.asp>
<link predicate- 'ternr.relevantto" type- 'teπn, :obj ect" object://4576 >
</link> <link operator- 'or" predicate- 'temr.intersects" iype- 'teπn:url"
Agent://email.wireless.all@abc.com/Agency.asp>
</link>
</resource> is resolved on the Agency located at the Web service on abc.com/Agency.asp to: InvokeSemanticQuery(
0, 24,
"all.criticalpriority.all", 2, { "teπn:and", "term:or" },
{ "teπn:relevantto", "ternr.intersects" },
{ "term: object", "teπnmrl" },
{ "object://4576", "Agent://email.wireless.all@abc.com/Agency.asp" } );
This is preferably resolved to a SQL query: SELECT TOP 25 * OBJECTS WHERE OBJECTID IN (SELECT
OBJECTID FROM OBJECTS WHERE CREATIONDATETIME='02/26/02' AND (OBJECTID [RELATEDTO] [OBJECT WITH ID 4576]) AND OBJECTID LN (SELECT OBJECTS FROM EMAIL WHERE CATEGORY [IS] 'WIRELESS')
This SQL example uses shorthand to illustrate the type of query that will be generated by the SQP. The SQP retrieves the XML and returns it to the caller. This XML is in the form of
SRML (or Semantic Results Markup Language), which is the XML meta-schema definition for semantic query results in the prefened embodiment of the invention. Sample A shown in the Appendix hereto is a sample SRML semantics results buffer or document. This is a sample of the XML that an Agency returns in response to a semantic query. The client Skin takes these results and generates presentation form them (using XSLT and/or script), based on the properties of the Skin and the Agent (object Skin/Context Skin/Blender Skin), the amount of display area available, disability considerations and other Skin attributes. f. Natural Language Parser
The Natural Language Parser (NLP) preferably converts natural language text to either an API call that the SQP understands or to raw SQL (or a similar query format) that can be processed by the database. The Natural Language Parser is passed text directly from the semantic browser or by email via the Email Knowledge Agent (see below). g. Email Knowledge Agent
The KIS preferably includes one primary publishing component, refened to as the Email Knowledge Agent (or Enteφrise Infonnation Agent (EIA)). This Agent functions, in essence, as a digital employee, and preferably includes a unique email address (e.g., a custom name selected by the Agency administrator). The Email Knowledge Agent complements existing publishing tools such as Microsoft Office, SharePoint, etc. by adding a "Fire and Forget" method of publishing infomiation and sharing knowledge. This is especially useful in cases where the person publishing the infonnation does not know who might be interested in it; In a prefened embodiment of the present invention, users send email to the Email
Knowledge Agent to publish comments, annotations, documents, attachments, etc. The Email
Knowledge Agent extracts meaning from the email and properly adds it to the Semantic
Network. Other users are able to access published infonnation via Agents of other platform presentation tools such as drag and drop, the Smart Lens, etc. (discussed below).
The Email Knowledge Agent is a system component that is created by the Agency administrator. The system user name is indicated when the server is first installed. The system user preferably conesponds to an email user in the enteφrise email system (e.g., Microsoft Exchange, Lotus Notes, etc.) In this embodiment, the Email Agent has its own mailbox, calendar, address book, etc. These in turn conespond to the objects on the Email Server for the system user. When the server is installed, the KIS installs the appropriate DSA for the system inbox (depending on the email application). The KIS preferably automatically adds a gatherer list entry in the SDG indicating that the system inbox should be periodically crawled for email. Because the Email Knowledge Agent is a first-class email address, it also serves as a notification source and a query source (for natural-language and instant messaging). Notifications from an Agency are preferably sent by the Email Knowledge Agent (indicating that there is new and relevant infomiation the user might be interested in, etc.). The Email Knowledge Agent may also receive email from users as natural language queries. These messages are parsed by the SQP and processed. The XML results are preferably sent to the user as an HTML file (with the appropriate default Skin) generated with XSLT processed over the XML results of the natural-language query.
Because the Email Knowledge Agent is a regular familiar component or "employee," the Agency administrator preferably adds the address to distribution lists. This step allows the SDG to semantically index all the email in these distribution lists, thereby populating the Semantic Network by seamlessly integrating the Email Knowledge Agent into distribution lists useful to users. This is a very seamless way of integrating the digital Infonnation Nervous System of the present invention with the way people already work in an organization.
Annotations. The Email Knowledge Agent is preferably used to publish annotations. In the present invention, annotations are preferably email messages. In the prefened embodiment, the annotation object type is a subclass of the email object type. This allows users to use email, typically the most common publishing tool, to amiotate objects in the semantic- browser. Users are able to amiotate objects and add attachments to the annotations. These attachments are semantically indexed by the SDG on the KIS. This makes possible scenarios where a user is able to navigate from, say, a document, to an annotation, to its document attachment, to an article on Reuters, to an industry event that starts next week.
The process described for semantically indexing email (by mapping the email XML schema to the Semantic Network) also applied to annotations. However, in the case of annotations in a prefened embodiment of the present invention, additionally processing is desirable. Specifically, when the user clicks "Annotate" on an object in the Presenter window in the semantic browser (described below), the browser loads the registered email client on the local machine (e.g., Microsoft Outlook, Microsoft Outlook Express, etc.). The "to"' field is populated with the address of the system user for the Agency that hosts the object. The subject field is populated with a special string, for example, "annotation: object=[objectid]". When the email anives in the Email Knowledge Agent's inbox, the DSA for the email inbox will pick it up (e.g., via a server event). The SDG retrieves the new email XML metadata from the DSA by receiving an event, or from the DSA the next time it asks the DSA for more data. In a prefened embodiment, this polling process occurs frequently. The DSA returns the XML metadata of the email object, oblivious to the fact that the email object refers to an email object type or an annotation object type. The SDG processes the email XML metadata, and examines the "subject" field. If the SDG "sees" the "annotation:" prefix, it knows that the email is actually an annotation, and proceeds to extract the object ID argument from the subject text. The SDG updates the Semantic Net ork for remaining email messages (adding each message to the OBJECTS and EMAIL tables, adding semantic links for the "from," "to," "cc," "bcc," and "attachments" fields, where necessary, etc.). .-In the prefened embodiment, the SDG perfomis an extra step. Specifically, it adds a semantic link entry that links the email object with the object indicated by the object ID argument in the subject text (with the PREDICATETYPEID_ANNOTATES predicate).
With the present invention, an annotation is treated as another semantic link with a special predicate. As a result, all the semantic features apply to annotations, such as semantic navigation via semantic links, semantic queries, etc. For example, a user can query for all annotations written by every member of his of her team in the last six months. This can be accomplished in the semantic browser by dragging, for example, the Agent Annotations.All on top of the Agent People.MyTeam.All and then sorting the results, or by creating a Smart Agent, which in turn invokes the "Create Smart Agent" wizard to create the query. h. Knowledge Domain Manager
The Knowledge Domain Manager is the component on the KIS that is responsible for adding and maintaining domain-specific intelligence on the Semantic Network. The KDM essentially "annotates" the Semantic Network with domain-intelligence. The KDM is initialized with URLs associated with one or more instances of the Knowledge Base Server (KBS), which in turn effectively stores "knowledge" for one or more semantic domains. The KBS has ontology and categories conesponding to taxonomy for each semantic domain that it supports. In addition, an Agent with a semantic domain (connected to a KBS) respqnds to semantic queries. If an Agent does not belong to a semantic domain, it cannot conespond to semantic queries (that' require an ontology or taxonomy). Rather, it only responds to keyword-based queries (albeit it will still provide context and time-sensitive retrieval services, but the available contexts will be limited). Each entry in the KDM is a semantic domain entry. The semantic domain entry has the URL to the KBS and a semantic domain name. The semantic domain name maps to a specific ontology on the KBS. In the prefened embodiment of the present invention, semantic domain names follow the convention: <Top Level Domain Name>\<Secondary Level Domain Name>
Examples of semantic domain names include:
• Industries
• Industries\Phannaceuticals\LifeSciences
• IndustriesMnformationTechnology • General\Sports.Basketball\NBA
■ • General\Sports.Basketball\CBA
Alternatively, semantic domains names can be refened to as "domain paths" as long as they are fully qualified. Full qualification is achieved by adding an Internet domain name prefix to the beginning of the path. This indicates the "owner" or "source" of the semantic domain. For example, "Nervana.NET\Industries\Pharmaceuticals" refers to "IndustriesVPhaπnaceuticals" semantic domain according to the "NERVANA.NET" Internet domain name. In another example, "Reuters.com\Sports\Basketball" refers to "Sports\Basketbalι" on "Reuters.com." Using this approach, domain names and paths are maintained globally unique. The Knowledge Domain Manager (KDM) periodically requests each KBS in its domain entry list for the categories in the knowledge domain. The KDM is preferably implemented as an XML Web Service on the KIS. The KDM includes configuration options for each semantic domain entry. One of these options may include the schedule with which the KDM will update the Semantic Network with domain-specific intelligence conesponding to the semantic domain entry. For example, the Agency administrator may configure the KDM (via the KIS) to crawl a semantic domain on a KBS every day at 1pm. The update schedule .should be. consistent with how often the administrator believes the ontology or taxonomy on the KBS changes.
The KIS preferably invokes the KDM periodically and asks it to update the CATEGORIES table. In the prefened embodiment, the KDM calls the KBS (via an XML Web Service API call) to obtain updated categories for the semantic domain name in the semantic domain entry, which conesponds to a particular taxonomy. An example of an API call follows: GetCategoriesForSemanticDomain (String SemanticDomainName). The KBS returns an XML-based list of all the categories in the semantic domain refened to by the semantic domain name. This XML list is consistent with the CATEGORIES schema shown above (category URL, name, description, the KBS URL and the semantic domain name). The KDM updates the CATEGORIES table with this infoπnation. For category entries that already exist in the table, the KDM updates the name and description. For new entries, the KDM requests a new object ID from the object manager and assigns that to the category entry. Since, in the prefened embodiment, a category is an "object," it inherits from the Object type and therefore has an object ID.
The KDM synchronizes the CATEGORIES table to the CATEGORIES list on the KBS (for a particular semantic domain) by deleting entries in the CATEGORIES table not present in the new list after examining the URL of the category entries and obtaining the relevant KBS URL and semantic domain name. If a semantic domain entry is deleted from the KIS, the KDM deletes all category entries with a conesponding semantic domain name and KBS URL. Essentially, this will be akin to ridding the Agency of existing knowledge.
The KDM periodically categorizes all "knowledge objects" in the Semantic Network based on its semantic domain entries. When new objects are added to the Semantic Network by the SDG, the SDG requests that the KDM categorize the objects. The KDM enumerate all KBS instances in its semantic domain entries and invokes XML Web Service calls with the XML of the object as the argument. In the prefened embodiment, the KBS returiis a result in an XML buffer similar to:
<results>
<result categoryurl="category://foo" score="91" >
<result categoryurl="category://bar" score="93" > <result categoryurl="category://foobar" score="100" > </results>
This infoπnation indicates the semantic categorization weights of the XML object for the categories in the semantic domain on the KBS. In a prefened embodiment of the present invention, the semantic domain entry is initialized with a threshold (0-100) indicating the minimum weight that the KDM should request from the KBS. The KBS returns scores that exceed the predetermined threshold. The KDM annotates the Semantic Network based on these categorization results. This is preferably accomplished by adding or updating a semantic link with the predicate type ID of "belongs to category" with the object ID of the category in the result. The KDM will update the SEMANTICLINKS table. Assuming by way of example that the object that is categorized has an object ID value of 56, the update query appears as follows:
UPDATE SEMANTICLINKS SET LTNKSCORE = 91 WHERE OBJECTJD=56 AND PREDICATETYPEID = 67 AND SUBJECTID IN (SELECT OBJECTID AS SUBJECTID FROM CATEGORIES WHERE URL LIKE "CATEGORY://FOO") The KDM periodically scans and categorizes all the "knowledge objects"
(documents, news articles, events, email, etc., preferably not including objects like people).
This process preferably occurs even if an object in the Semantic Network has previously been categorized as the KBS might have become "smarter" and therefore provides superior categorization, hi such a case, the results could change even if the same categorization request is repeated. This will occur, for example, if the ontology on the KBS has been updated. Thus, in the prefened embodiment, categorization will be performed both when an object is added to the Semantic Network by the Semantic Data Gatherer and periodically to ensure that the Semantic Network has the most up-to-date domain knowledge. . Other Components
The Favorite Agents Manager. On Agencies that support User States, a Favorite
Agents Manager manages a list of per-user favorite Agents, hi the prefened embodiment, the
Favorites Agent Manager stores a mapping of user names to favorite Agents in a UserFavoriteAgents table.
Compound Agent Manager. A Compound Agent Manager manages the creation, deletion, and update of compound Agents. As described above, compound Agents are Agents that are comprised of other Agents in the system, and are initialized to return the union or intersection of the query results in the contained Agents. The Compound Agent Manager manages all compound Agents in the system and maps compound Agents to the Agents they contain via the CompoundAgentMap table.
The Compound Agent Manager exposes functions to create compound Agents, delete, rename, add to and remove Agents from them, and indicate whether a union or an intersection is desired. Compound Agents can be added to other compound Agents. On invocation, the semantic query processor asks the Compound Agent Manager for its compound query. The Compound Agent Manager navigates through its Agent map graph and returns a complex query of all the queries of all Agents that it contains. If Agents are deleted, compound Agents "pick up" the new state when they are invoked, ignoring the Agent query. In other words, the compounding of queries is only done for Agents that still exist. If the compound Agent observes that one of its Agents has been deleted, it will delete the entry from its map. User Profile Manager. The User Profile Manager (UPM) preferably uses the Inference Engine to infer the user's profile on an ongoing basis. The UPM annotates the Semantic Network based on feedback from users as to their explicit preferences. In the prefened embodiment, this process involved use of the PREDICATEID SLNTERESTEDIN predicate. The UPM infers semantic linlcs and amiotate the Semantic Network with the PREDICATEID SLIKELYTOBELNTERESTEDIN predicate. All query results to the user will be qualified (out-of-band) with a query to the Semantic Network for the PREDICATEID SLIKELYTOBEINTERIESTEDIN predicate. Query results are based on the user's habits, as the Inference Engine leams them over time. Alternatively, the UPM may be configured with user profile information stored in the
User State Store (USS). This is infoπnation manually entered at the client indicating the user's preferences. This infoπnation is transferred and stored at the server that the user is interacting with. These preferences are tied to different schema. For example, for documents, the schema may be based on the prefened categories. For email messages, the schema may be based on prefened categories, authors, or attachments. These are two of many possible examples. The UPS annotates the Semantic Network based on the manually entered infonnation in the USS.
Server Notification Manager. The Server Notification Manager (SNM) is responsible for batching server-side notifications and forwarding them to users. In. the prefened embodiment, users register for server-side notifications at the Agent level. Each Agent is capable of firing notifications of its query results. The Server Notification Manager deteπnines how to filter the query results and format them for delivery via email, voice, pager or any other notification mechanism, e.g., the Microsoft .NET Alerts notification services. The Server Notification Manager maintains information on the last time users "read" the notification. This is preferably indicated from the client via a user interface. The SNM preferably only notifies a user when there is new infomiation on the Agent since the last "read" time for the particular user.
Agent Discovery. Using midticast-based Agent discovery, each Agency sends multicast announcements indicating its presence on the local multicast network. The Agency administrator sets the multicast TTL. The present invention preferably uses either use the
Session Announcement Protocol (SAP) with a well-known port of 9875 and a TTL of 255, or a proprietary announcement port with a customizable TTL. For details on SAP, see http://sunsite.cnlab-switch.cli/ftp/doc/standard/rfc/29xx/2974, which is incoφorated by reference. The Infonnation Agent preferably includes a listener component that receives SAP announcements. In the prefened embodiment, the announcements are sent as XML and will include the following infonnation
• The server ID (this is a unique identifier)
• The server URL (this is the HTTP URL to the Agency's XML Web Service) o The announcement period (T) - this indicates the time between each announcement o Whether there are any new Agents in the Agency since the last announcement and the last Agent creation time (on the Agency's clock)
Each Agency sends the XML announcement and uses Forward Enor Conection (FEC) or Forward Erasure Conection to encode the packet. This makes the system robust to dropped packets. Alternatively, the Agency can be configured to send the XML announcements several times in succession (per announcement).
The Information Agent multicast listener exposes directory-like semantics to the
Semantic Environment Manager. The listener aggregates all the XML announcements from the Agencies from which it receives announcements. It will also cache the last time it received an announcement from each Agency. The listener flags Agencies that it thinks might be dead or inactive. It does this when it has not heard from the Agency for a time longer than the Agency's announcement period. The listener might be configured to wait for several periods before flagging the Agency as inactive. This will handle the case of dropped announcements (due, perhaps, to traffic congestion). The listener will update the Agency list in the Semantic Environment Manager each time it receives announcements. The Semantic Environment Manager periodically inquiries of the listener whether there are any new Agents. The Semantic Environment Manager checks the Agency list and asks each Agent that is active whether it has new Agents. The Semantic Enviromnent Manager qualifies this request with the Agency's last Agent creation time maintained locally and the current time based on the Agency's clock. The Agency responds and also sends the new value of the last Agent creation time. The Semantic Environment Manager caches this value in the Agency entry. If there are new Agents, the browser infoπu the user via a dialog box and asks the user whether he or she wants to view the new Agents.
The present invention_also supports Agency announcements using a peer-to-peer Agent discovery. In this model, announcements are sent either to a directory server that all clients check or directly to the clients via a standard peer-to-peer publishing protocol.
FIGURES 47-53 are exemplar screenshots showing aspects of Agent management by the KIS. FIGURES 47-50 illustrate a sample KIS Agency administration manager showing server-side Agent views and server-side Agents. FIGURE 51 further illustrates sample administration user interface elements for managing SDG (crawl) tasks, system tasks (e.g., the Inference Engine), the system Agent Email (e.g., inbox), calendar and contacts DSA and all the SMS data tables (objects, semantic links, categories, etc.). FIGURE 52 illustrates a sample, of the "Server "Properties" dialog of the present invention in the KIS Agency administration manager. The dialog illustrates how the server administrator can set server properties such as the server name, the display name, the SMS Data Store, properties, the KDM properties (e.g., the knowledge domain path) arid the user DSA properties. FIGURE 53 illustrates a sample of the "Server Statistics" dialog in the KIS Agency administration manager of the prefened embodiment. The dialog illustrates the display of statistics such as the total number of server-side Agents (Standard Agents and Blenders), the total number of server-side Standard Agents, the total number of server-side Blenders, the total number of server-side Agent-views, the total number of server-side Agent subscriptions, the total number of infoπnation objects stored on the server, the total number of semantic links, the total number of users on the server (Agency) and the total number of user groups.
3. KNOWLEDGE BASE SERVER
The Knowledge Base Server (KBS) is the server that hosts knowledge for the KIS. In most applications, many instances of the KIS will be deployed, but only few (or one) KBS will be deployed for any given organization. This is because KBS can be reused (they are domain-specific but data-independent). For example, a phaπnaceutical fiπn might deploy one KBS initialized with a pharmaceuticals ontology, but have several KIS installations; perhaps per employee division or per employee group. The KIS preferably includes the following components: 1. One or more ontologies that conespond to one or more semantic (knowledge) domains. A semantic domain is refened to using a semantic domain name. This is a name that refers to a domain path within a semantic hierarchy. Examples are Industries. Technology, Industries.Pharmaceuticals.LifeSciences, and
General. Sports.Basketball. These names or paths may also be globally and uniquely qualified (e.g., with Internet domain names) as previously discussed.
2. One or more taxonomies that conespond to the supported semantic domains. These taxonomies contain a hierarchy of category names.
3. A categorization engine that take a piece of text or XML and the semantic domain name with which, the categorization is to be performed, and returns the categories in that domain that the text or XML belong to, along with the categorization scores (on a scale of 0-10 or, preferably, 0-100).
4. An XML Web Service that exposes APIs to add new supported semantic domains (and conesponding ontologies and taxonomies), to enumerate the categories for a given semantic domain, and to categorize a text or XML data blob. 5. An XML Web Service reference to another KBS from which the KBS gets its knowledge. In this mode, the KBS acts as a proxy. The KBS can be initialized to act as a proxy and to get its supported semantic domains, ontologies, and taxonomies from another KBS. > As explained above, the KIS (via the KDM) periodically sends XML objects to the
KBS to categorize them for a given semantic domain.
4. INFORMATION AGENT (SEMANTIC BROWSER PLATFORM) a. Overview
The system client, in the prefened embodiment the Information Agent of the present invention, includes the semantic browser components and user interface that provide a semantic user experience. In the prefened embodiment, the Information Agent provides the following high-level services:
• Allow users the power of context and time-sensitive semantic information retrieval via local and remote Infoπnation Agents. • Allow users to discover infonnation on local and remote Agencies that are exposed via Agents tlirough the XML Web Sendee of the present invention. This infomiation is preferably classified into well-known semantic classes such as documents, email, email distribution lists, people, events, multimedia, and customers. • Allow users to browse a semantic view of infomiation found via Agents of the present invention.
• Allow users to publish infoπnation to an Agency.
• Allow users to dynamically link infomiation on their hard-drive, local network or a specific Agency with infoπnation found on Agents from another Agency. This facilitates dynamic e-linking and user-controlled browsing.
An advantage of the Infoπnation Agent of the present invention is that users open up
Agents similar how users open up documents from their file-system namespace. The
Information Agent will have its own environment that opens up semantic "worlds" of infoπnation. For example, ABC company may have an internal KIS Agency that has Agents for internal documents, email, etc. In addition, third-parties may host Agencies on the Internet to hold infomiation on industry reports, industry events, etc. In a prefened embodiment of the present invention, ABC company employees open Agents to discover information on the Internet that relates to their work as well as to semantically relate infomiation that is internal to ABC company to infoπnation that is external but relevant to ABC company. b. ' Client Configuration
In the prefened embodiment, the system client is able to semantically link infoπnation found locally as well as on remote Agencies. This is preferably accomplished through the use of an exposed Semantic Environment comprised of Agencies from a Global Agency Directory, Agencies on the local area network (published via multicast or a peer-to- peer publishing system) and Agencies from a custom Agency Directory using Agent Discovery. The prefened client configuration is based on a framework having Agents and local Agencies, and includes a Semantic Enviromnent Manager, which manages locally saved Agents and Favorite Agents, essentially integrating the history and favorites metaphors. The Semantic Environment Manager uses Semantic Query Documents within the Semantic Environment to present knowledge to users via the Semantic Environment Browser. The client configuration will also include the Agent Discovery infoπnation (e.g., Agency lists, Agency directory infoπnation, etc.). c. Client Framework Specification Overview. The client framework specification provides the service infrastructure for the Infomiation Agent user interface, and defines basic services and interfaces, includes core user interface components, and provides an extensible, configurable environment for the main building blocks of the user interface of the Infomiation Agent. This section described the client framework specification according to a prefened embodiment of the present invention. The Framework Core defines base services, configuration, preferences and security mechanisms. The Core User Interface Components define the user interface services and modules that support server and Agent configuration, control and invocation, and some configuration for the Semantic Browser Framework. The Core User Interface Components are implemented as a Windows Shell extension and associated user interface (described
I, below). The Semantic Browser Framework provides base query and results management services, and the framework for results presentation. The specifics of the user interface related to semantic object presentation are preferably configurable and extensible; even default presentation support is provided as a pre-installed "extension." The Semantic Browser Framework is preferably implemented as a set of behavior extensions to existing platfom s used in Today's Web (e.g., Internet Explorer), and leverages the supported XML, XSLT, HTML/CSS and DOM functionality.
Context. The. client framework builds upon semantic services components of the present invention including semantic query support, context and time-sensitive semantic processing and linking of information, etc. The client framework is preferably built as a shell extension and platfomi (e.g., Internet Explorer) extensions, which provides functionality to users in the context of their existing tools and environment. For example, the Information Agent may be implemented as a Shell Extension (which extends the Windows Shell and. employs the standard Explorer view and user interface models). In an alternative embodiment, the present invention is equally applicable in a standalone semantic browser application. Requirements. The preferred requirements for the client framework relate to flexibility and extensibility. This ensures that the user interface can be easily and quickly adapted as there are more information object types, user profiles, etc. Included are the following:
• Provide support for Skins to manage the entire set of query results. • Allow for a wide range of approaches, include lists, tables, timed slides, etc.
• Provide a screen-saver (or equivalent) mode.
• Provide support for Skins that can be associated with an object class. • Ensure that there is a default Skin that can handle all classes.
• Skins should be as simple as XSLT, but should allow script support, and possibly even code (with appropriate security restrictions).
• Provide support for browsing the Semantic Environment in the results view (to complement the Agent Tree View), including Agents (Smart, Dumb, and
Special), Agencies, and Blenders.
• Provide well-defined interfaces between components, and ensure that all . communication must occur via the framework.
• Provide a solid security model throughout the framework Framework Core
Semantic Environment Manager (SEM). The SEM manages the creation, deletion, updating and browsing of Agents, Blenders, and Agencies on users' local machines. In addition, the SEM is responsible for listening to Agency multicast announcements, browsing Agencies on the enteφrise directory (e.g., via LDAP), browsing Agencies on a custom directory, and browsing Agencies on the Global Agency Directory.
The SEM includes a storage layer that stores the metadata of every Agent on the system, including all the Agent attributes (such as the Agent name, description, creation time, last usage time, the Agent type (Smart, Dumb, Special, etc.), the infoπnation object type the Agent represents (for Agents created based on infomiation type), the context type the Agent represents (for Special Agents or Agents created based on a Context Template), the attributes of the Agent, a reference to the XSLT or other script file that represents the Agent's Skin (including filter/sort preferences and other presentation schemes), the notification information and method (if requested for the Agent), and the buffer or file-path/URL to the Agent's SQML query. The Information Agent (semantic browser) may store this Agent metadata in a local database, a store like the Windows registry, or in an XML file store on the local file-system. The SEM also uses the Agent attribute to indicate whether an Agent is a Favorite Agent. In addition, the SEM automatically deletes Agents that are not favorites and which are older than a configurable age limit (e.g., two weeks).
The Infomiation Agent's Shell Extension and other components (such as the toolbar and the Open Agent dialog) employ the SEM to provide Agent creation, deletion, browsing, updating, and management of Agents via its user interface.
Preferences Manager. This component manages all client-side preferences, providing services to persist the preferences, communicates with servers as needed to share preferences or support roaming, and supports setting and obtaining preference values from other components. This component has associated user interface as well as some more specific preferences user interface components. The preferences are divided into sub-components, and may abstract the preferences for associated client classes. These include:
• Core Preferences. This includes basic configuration such as user profile and persona infoπnation. o Skin Preferences. This also associates prefened Skins with object classes, as well as the prefened list Skin and screen saver Skins. There may be additional
Skin-related preferences settings.
This component also manages the set of locally available Skins. Downloadable Skins are preferably managed through this component.
Notification Manager. Notifications provide a means to indicate to users that there is new information available on a given Smart Agent. Users optionally configure a specific Smart Agent to support or provide notifications (it will be OFF by default for most Smart Agents), and will also configure how to present notifications to users. These notifications are presented by the Notification user interface component. The Notification Manager is responsible for managing background, polling queries for the appropriate set of Smart Agents. The Live Infonnation Manager is a parallel component that provides similar services to the Results Browser.
The Notification Manager gathers the list of Smart Agents marked for notification, andperiodically polls the associated servers for new infonnation. "New" is defined as "since the last poll [or query]." Each time the poll responds, it includes a timestamp indicator that the Notification Manager must persist, associated with the Agent.
The user interface associated with configuring the Notification Manager is preferably implemented in coordination with the Agent Tree View. This enables notifications (e.g., a "Notify" popup menu option of each Smart Agent). The Notification Manager may also support alternatives for notifying the user when there are new results available. Some options include a display style (e.g. bold, colored, etc.) for the Agent in the Agent Tree View, a reminder dialog, audio notification, or more exotic actions like email, IM or SMS notification. Client-Side Security. Client-side security issues relate to extension code and Skins.
The Skins are preferably XSLT, but may also support script. In addition, the generated
HTML may include references to ActiveX components and behaviors. The presentation sandbox may include security ' restrictions that prevent Skins from running potentially malicious code via script. For example, the implementation may completely disallow any unsigned code (including ActiveX and DHTML behaviors).
All client-server communication with Agencies are preferably hidden from the published interfaces (for Skins), which third parties will customize to provide custom Skins. By isolating the functionality outside of the primary client runtime, the risk of security compromise can be reduced. Core User Interface Components Agent Tree View. This is a Shell Extension Tree View that supports much of the core user interface for controlling and invoking Agents.
Semantic Environment Browsing User Interface. This provides user interface to allow users to browse the Semantic Environment. An example of this is the "Open Agent Dialog." This complements the Agent Tree View, which also displays a hierarchical view of the namespace (see screenshots).
Agent Inspector. This provides user interface to view the properties or edit (in the case of user-created Smart Agents) an individual Agent, Blender or Agency. Browser Host. This is preferably a "wrapper" on the semantic browser core (e.g., the
Internet Explorer browser runtime), which allows the presentation of a custom view of the Agents, Agencies, and Blenders in the Agent Tree View. It preferably does not have any user interface itself, but is a bridge component between the Shell Extension and the Browser Framework. This component is also preferably responsible for coordinating certain browser functionality with the Windows Shell user Interface, including in particular the navigation ("back/forward") mechanism, in order to provide a seamless "back/forward" user experience (wherein the user only has to deal with one "back/forward" history list).
Core Preferences UI. This provides a user interface for preferences related to
Semantic Environment, server, persona and Agent management, as well as any other miscellaneous preference settings. This preferably includes primitive property sheet dialog, possibly divided up into separate sheets by functional area. In the prefened embodiment, this should be a tabbed dialog user interface. . ,
Skin Preferences UI. This provides a user interface for preferences related to Skin management. This is preferably a property sheet dialog. The list of available Skins should be presented as a list, for selection. This user interface allows users to set the cunent Skins, as distinct from the default Skins. It preferably allows users to make the cunent Skin be the default. For per-Agent Skin preferences, this preferably allows users to select a Skin for the cunently selected or opened Agent.
Notification UI. The user interface associated with configuring the Notification Manager is preferably implemented in coordination with the Agent Tree View. The Notification Manager may also support alternatives for notifying users when there are new results available. Some options include a display style (e.g. bold, colored, etc.) for the Agent in the Agent Tree View, a reminder dialog, audio notification, or more exotic actions like email, LM or SMS notification. In the preferred embodiment, the user interface should include a tabbed dialog (or equivalent) to allow users to select out of the aforementioned notification schemes (and the like).
Screen Saver. The user interface preferably provides a special modality to the Results Browser that function like a screen saver, filling the screen in a theater-mode display. In the prefened embodiment, special Skins should be used for the screen-saver mode. These Skins could emphasize a dynamic display that can leverage a larger screen area, but could also use larger fonts and more widely spaced layout.
Browser Framework Results Browser. The Results Browser is responsible for displaying the results of queries, and the information on any local resources opened. The Results Browser preferably, obtains one or more XML files from the Query Manager and merges these into a single XML file that represents a list of objects. The list itself may be filtered or sorted as an initial step. The list as a structure is transformed by a special class of Skin (an XSLT transform sheet, possibly including some script) that handles lists. The list-Skin creates the primary DHTML (or the like) structure, e.g., a list, a table or perhaps a timed sequence. Object Skins manage the individual DHTML items that present the information for each object instance. List-Skins may handle the dispatch of individual object Skins (mapping object class to Skin), but the Results Brower preferably provides default mappings of class to Skin for simplicity. Users may prefer a given form of presentation, and may choose default Skins (both for the list as well as for object classes). The original query (i.e. the SQML) may also include parameters that indicate which Skins should be used (especially which list-Skin). These will be passed to the Results Browser along with the results. The Results Brower uses the facilities of the Skin Manager to select the right Skin to apply. Different rules may be employed for how user preferences and Agent (author) preferences are combined and prioritized.
When query results are composed of multiple distinct XML files, the Results Browser must merge these into a single XML document to provide a seamless user experience. The prefened embodiment provides for handling additional results dynamically. This dynamic update mode is preferably implemented by using a different template or perhaps a script method within the XSLT template. Alternatively, the list Skins may require a behavior (or local runtime component) to manage the logic of adding to the document without disturbing user context. Query Manager (or Client-Side Semantic Query Processor). The Query Manager is responsible for handling the communication with the server(s), executing the requests for information and gathering the XML results. The resulting XML is passed to the Results Browser for presentation to users.
The Query Manager preferably provides the services to support the Smart Lens functionality. When a Smart Lens request is made, the results are returned as XML and are passed to the Results Browser, preferably marked to indicate that they are Smart Lens results for a given object. The Query Manager preferably includes the. following sub-components that provide individual services to fulfill the query requests.
• SQML Interpreter. This component must decompose passed SQML into a set of requests, possibly with linked resources. Each request or resource link resolves to a resource with an associated protocol (e.g. HTTP, or one of a number of local pseudo-protocols like outlook: or document:), and is dispatched to the associated protocol handler. A given SQML file may include a mix of network and local resource types.
• Resource Handler Manager. This is preferably a central registration mechanism for resource handlers. It is a minimal layer that associates protocols and pseudo- protocols with handlers, and simplifies the dispatch of resource requests.
• Resource Handlers. These are components that encapsulate the specifics of accessing the resources from a given "server." A resource handler does not resolve any linked resources. This is preferably the responsibility of the SQML Inteφreter (i.e. the SQML Inteφreter will have already resolved linked resources and provided the associated meta data as part of the resource request to this handler). When the resource is a Semantic Web service, the component preferably bundles up the request and issues it via http. When the resource is a local resource (e.g. a document: or Outlook: resource), the resource handler handles the resource directly. For documents, the resource handler passes the document (a file: URL) to the semantic meaning extraction, summarization, and categorization engine to extract meta-data. For email, the resource handler extracts messages from the exchange server, or local .PST files. Note that when there are links on a local resource, the local resource handler must perfoπn the processing that filters results for semantic relatedness. This may be custom to the handler for efficiency, but a central, generic Relatedness Engine will provide services for most cases. o Relatedness Engine. This provides a place to gather the logic for comparing objects for relatedness. The comparison is preferably dependent on the; mix of . schemas involved, but is otherwise a simple operation — iven two objects, provide a measure of relatedness. Filter/Sort Manager. The Filter/Sort Manager supports the application of filters and sorts to the lists of results provided to the Results Browser. The Filter/Sort Manager leverages the services of the Filter/Sort Preferences component to obtain user preferences for cunent settings. The main function of this component is to resolve general preferences, per- Agent preferences, and any settings defined in the actual results (this may or may not be supported). This component is notified by the Filter/Sort Preferences component when users change the cunently applied filters and sorts. Because the associated user interface is part of a tool bar associated with the Shell Extension (i.e. its right-pane View), but the application of the functions happens in the Results Bro ser space, the control is typically indirect.
Lens Mode. When a Smart Lens is invoked, the Results Browser must generate Lens requests (queries) for objects that users choose. The queries are asynchronous so that users can select Smart Lens queries for various objects and view the results as they are returned. A suggested user interface for this is to reserve some real-estate for a Smart Lens icon. When in Smart Lens mode and the user clicks (or hovers) over the Smart Lens icon, a query is issued, and the icon changes to indicate that the query is in progress. When results are returned, they are handled by the Results Browser and dedicated Smart Lens templates in the Skins, and the Smart Lens icon for an object changes to indicate that results are available. Clicking or hovering over the icon again will display the Smart Lens results in a Skin specific manner (see sample Smart Lens pane user interface). If the query is returned quickly enough, then the whole function preferably feels like a popup activated by a hover or single click.
Deep Info View. If Deep Information is not available in the original results, this component generates the associated query. The query is preferably asynchronous. When results are returned to the Results Browser, they are processed tlirough the appropriate Skin (using a special Deep Infomiation template for each Skin), and the resulting HTML is incoφorated into the results document under the associated object. The primary Skin for the schema inserts a Deep Infoπnation element in the HTML for the object so that the Results Browser knows where to incoφorate the results. When Deep Information is available (whether as part of the original results or in response to a Deep Information query), the Skin either displays it directly or will indicate that it is present, and some Skin-defined user interface will allow users to enable the display (e.g. as a popup window).
Context Info Manager. For objects cunently displayed in the Results Browser, certain notifications are preferably provided by default. Two classes of new or additional info will be provided to users: 1. Additional results that were added to the server since the user made the original request. This is especially useful for things such as headlines or active email threads. The results are handled by the Results Browser, by inserting the new objects into the view. 2. Context Templates and related infoπnation that would be of interest to the user.
This is generated by additional queries to a specific Agent (Smart Agent, Special Agent, Blender or Agency), using a particular object as context. The results are handled similarly to the way that Deep Infomiation View and Smart Lens Mode results are handled, by processing the XML returned from the query, and inserting the resulting HTML into the existing HTML for the object. The Skin controls the display mechanisms and UI. An example of related information is "Breaking News" associated with the object.
Skin Manager. Maintain user preferences for list Skins, object Skins, and dependencies between list and object Skins (certain Object Skins may only make sense for a given list-Skin). The Skin Manager also maintains parameters for each Skin that indicate constraints for the Skin, e.g. how much screen real-estate it requires, or modalities it best applies to. Considerable intelligence is preferably built in that assists the Results Browser to choose Skins for a range of screen and window size constraints, as well as for modalities, accessibility, language and other constraints. Initial versions will likely be much simpler. Skin Templates. This describes the structure of a Skin and how it is applied from within the Results Browser. A Skin is preferably XSLT templates that convert the'results XML to XHTML (and/or other languages like SVG) or proprietary presentation platforms like Flash MX and ActionScript. The templates can also insert styling information, e.g. for CSS styling. The resulting presentation code (e.g., XHTML) can restrict the inclusion of code, for security reasons. Framework code in the Results Browser invokes the Skins. The prefened embodiment includes the following classes of Skins:
• List Skins (or layout Skins). A list Skin is used to transform a list of objects returned from a query into some overall presentation structure. This may be a simple list, a table, or a timed sequence of slides. List Skins are not schema or object specific, although they may only support certain Skins, which can work within the constraints that the associated presentation form defines. E.g., a list Skin that defines a table layout may require, or prefer, object Skins that can produce infomiation in a small rectangular format.
• Object Skins. Object Skins are schema specific, and generate the presentation for an individual object of a given infomiation object type (or infoπnation class). It is possible to define a Skin for the generic super-class (or any other super-class) that can serve as a default Skin for a range of derived classes or subclasses (presumably by omitting some details).
• Context Skins. Context Skins are tied to a particular Context Template, and generate the presentation that will most effectively convey the context indicated by the template.
• Blender Skins. Blender Skins are designed to present the results from Blenders. These Skins should allow the user to view the results via the Agents contained in the Blender, via information object type, or via a merged view that displays all the results as though they came from one source.
Skins preferably model constraints such as modality and presentation display area by handling the constraints (passed as parameters either statically or dynamically by events within the browser core itself). This is preferably supported by imposing a restriction that list Skins must specify only acceptable object Skins. In an alternative approach, object Skins may be designed for a given list Skin, and the Results Browser/Skin Manager chooses object Skins for the cunent list Skin.
List Skin Details. Users may choose a single list Skin for the cunent view and make it the default. List Skins may also be associated with individual Agents, in which case the generic default is overridden. The Results Browser invokes the list Skin to process the list of results, although the list Skin preferably does not actually handle the individual objects. It creates some per-object instance in the framework presentation (e.g., a timed entry in a sequence, or a cell in a table, or an item in a list), and then the object Skins will fill in the details.
Object Skin Details. The object Skins convert a particular schema to XHTML. Support for asynchronous query results for things like Deep Infoπnation and Context Template infomiation are provided by invoking associated templates from the Results
Browser (through the DOM) on the query results XML, and then inserting the resulting
XHTML into thβi results document through DOM interfaces. There are preferably several individual templates within an object Skin, including: ; • . Primary schema template. This is the main piece that generates XHTML, for default display. This must create the wrappers for Deep Infoπnation, Smart Lens information, Context Template infoπnation content, and any script that provides user control over the associated display.
• Deep Information template. This template handles the meta-information for Deep Infomiation. It may be called for inline deep info provided with original results, or it may be called to handle asynchronously requested Deep Infoπnation. Either way, it preferably generates XHTML in some form, which is inserted under the wrapper element for Deep Infoπnation. The insertion probably happens in XSLT for inline deep info, and is effected tlirough DOM insertion for Deep Information query results.
• Context information template. This template handles the results-information for context infomiation query results. It generates XHTML in some form, which is inserted under the wrapper element for live info. The insertion is effected through DOM insertion for Deep Infoπnation query results. ° Smart Lens information template. This template handles the results-information for Smart Lens query results. It generates XHTML in some form, which is inserted under the wrapper element for live info. The insertion is effected tlirough DOM insertion for Deep Infoπnation query results.
In the prefened embodiment, the template cannot modify the other contents of the XHTML (even for the same object), so it will be up to the Results Browser to coordinate the user interface changes that indicate when Deep Information, live infomiation or Smart Lens results are available. The framework requires certain icons to be used (also for consistency), and for- these to have regular names or element types, which will allow the Results Browser to find and modify them as needed. In addition, the Results Browser can create and raise events to indicate the state changes. The template-generated script can respond to these events, and display the associated information as desired. Default Skins. In the preferred embodiment, a set of default Skins is provided. This preferably includes Skins for the basic object classes and a small set of list-Skins that allow a variety of views of query results. Preferable list-Skins include:
• A detailed list display (like the Windows Explorer details view) • A tabular Icon view (again, like the Windows Explorer Icon view, but somewhat richer)
• A timed presentation view. e. Client Framework
In the prefened embodiment, the system client includes Shell Extensions, a Presenter, and Skins used by the Presenter to display infoπnation with context and meaning.
Shell Extension. An Explorer Shell Extension is a Microsoft Windows software component that extends the Windows Shell with custom code. Shell Extensions allow applications to use the Shell as a custom client, and also provide services such as clean integration with the desktop, the file-system, Internet Explorer, etc. Examples of default shell extensions include "My Documents," "My Computer," "My Network Places," "Recycle Bin," and "Internet Explorer."
The use of a Shell Extension in the prefened embodiment of the present invention has several advantages:
1. It provides a very clean way to provide a user experience that seamlessly integrates with how knowledge-workers cunently browse for information. In turn, this obviates the need to develop a proprietary client and allows for non-standard integration with Microsoft's Internet Explorer, "My Documents," etc.
2. It embraces Today's Web and provides a migration path for the transfer of content in Today's Web to the Information Nervous System of the present invention. For example, users preferable drag and drop documents from, their hard drive (via
Microsoft Explorer) or from the Internet (via Internet Explorer) into remote Agents on the Shell Extension of the present invention. This is difficult and non-intuitive with a proprietary client. Nevertheless, the present invention contemplates portability to a proprietary client or to the equivalent of Shell Extension on non-Windows operating system and operating systems for non-personal computer devices.
The Shell Extensions of the present invention provide a view of users' Semantic
Environment (e.g., history, favorites and other views). In the prefened embodiment, the Shell Extension provides for the following:
1. Allows users to open an Agent, a document, a folder, or an address on the semantic browser's Semantic Environment. For an Agent, the client displays a custom "Open Agent" dialog box that allows users to browse the semantic browser's Semantic Environment. This preferably includes Agents in users' My Agents list, Agencies on the Global Agency Directory, Agencies on the local area network (announcing via multicast), 'and Agencies on any custom .Agency Directory that users have configured. [INSERT RELEVANT SCREEN SHOTS ON UI] Opening an Agent results in the client displaying the results of the query of that Agent. Opening a document opens the XML metadata for that document, consistent with the schema for the document object type. Opening a folder opens the XML metadata for a file-system folder., Users are able to open the immediate or deep contents of the folder via the folder itself. Opening an address allows users to enter any address to be opened by the client framework. This includes URLs (which open the XML metadata for the document), documents on the file- system, Agents, or objects (see "URL Naming Conventions" below). In the case of Agents, the Agent URL is preferably entered as follows: Agent://<Agent name>@<Agency name>.<domaϊn name>. This is analogous to the http://<URL> naming convention for HTTP URLs. The Agent:// prefix is required in this case because the Open Address option can open any address. In the case of the "Open Agent" option, users preferably do not need to add the prefix; the client framework automatically canonicalizes the URL to include the prefix. This is similar to how users are able to enter "www.foo.com" into Today's browser without the qualifying http:// prefix. It is anticipated that the client allows users the ability to open other objects, for example, Microsoft Outlook .PST files.
2. Allows users to browse, subscribe, and unsubscribe to or from Agents on a given Agency that supports User State. 3. Allows users to save invoked Agents or semantic query results into the My Agents list.
4. Allows users to create Blenders and to add and remove Agents to and from Blenders (including via drag and drop). 5. Notifies users when there are new Agencies on any of the Agency directories (for example, the Global Agency Directory, the Local Area Multicast Network or any custom Agency Directories) since the last time they checked 6. Notifies users when there are any new Agents on any particular Agency since the last time they checked 7. Provides drag and drop access to relational semantic queries for objects in the
Semantic Environment. The Shell Extension allows users to drag and drop a document from the Semantic Environment (either on a local drive, the network neighborhood, the Intranet, or the Internet) to a shell folder representing an Agent. This triggers a remote procedure call to the XML Web Service for the given Agency with the document metadata as the argument.
8. Provides "paste" access to objects copied to the system clipboard. The present invention uses the system clipboard to allow users to copy any object for later access. In addition, the clipboard allows users to copy objects from other applications, for example, Microsoft Office applications (e.g , email items from Outlook), from multimedia applications, and to copy data from any application.
9. Allows users to select an Agent as a Smart Lens. A Smart Lens allows users to view objects in the results view based on context from an Agent or any object that can be copied to the system clipboard. For example, ordinarily, if a document object is in the results view and users hover over the link representing the object, the object metadata is displayed. If, however, a Smart Lens is selected (for example by pasting it onto the results sheet), and users hover over the object, information that relates the object in the Smart Lens and the object underneath the cursor is displayed. For example, if users copy "People.Research.All" to the clipboard and paste it as a Smart Lens, then hover over a document, metadata may be displayed in a balloon popup as follows: "Found 15 people in
People.Research.All that are experts on this document." Other examples are "Found 3 people that might have written this document" and "Found 78 email messages relating to this object posted by people in People.Research.All". Users decide whether to invoke any of the links in the metadata in the balloon popup. In an alternative embodiment, the popup may be displayed in a sidebar and does not require a balloon. When a Smart Lens is pasted onto the clipboard, the Shell Extension preferably communicates with the system .and changes the mouse cursor to reflect the name of the selected Agent. The Smart Lens preferably has global scope because it is copied from the clipboard. In other words, for example, all instances of Windows Explorer and Internet Explorer "see" the Smart Lens and respond to its actions. In the prefened embodiment there is a Smart Lens tool in the Infonnation Agent toolbar that applies to the ctinent object on the clipboard (e.g., Agent or other object). By default the Smart Lens tool will be deselected once a link is clicked in the system. Users are preferable able to "pin" the Smart Lens. When the Smart Lens is pinned, the Smart Lens remains active until users explicitly deselect* it. In the prefened embodiment, to pin a Smart Lens, users select the "Paste as Smart Lens and Pin" tool on the toolbar.
10. Allows users to "tear-off the results of an Agent from the Shell Extension and display it in docked view on the desktop. In this view, the Agent results browser window acts as a semantic ticker. This feature allows users to continuously display the semantic information while continuin to do other work.
11. Allows users to enable an Agent to be used as, a screen-saver.
12. Allows users to browse and invoke available Skins on the Global Agency Directory. Presenter. The Presenter is a set of local components (e.g., browser plug-ins) that take semantic queries from scripts (or other plug-ins) and pass them off to a KIS Agency XML Web Service. The present invention translates the results of semantic queries and passes XML to other behaviors or scripts for eventual presentation to users.
In the prefened embodiment, the Presenter is invoked by the Shell Extension with an SQML file. The system preferably communicates with the XML Web Service directly. The system resolves the SQML file and invokes calls to open XML infomiation sourced locally or remotely (via XML Web Services on Agencies refened to in the SQML file). Alternatively, if an Agent URL is passed to the system, the Presenter directly opens the URL by invoking it via a call to the XML Web Service of the Agency on which the Agent is hosted. In the prefened embodiment, the system calls the appropriate method with the appropriate semantic object type. Examples of default semantic object types are SEMANTICOBJECTYPEΓD_EVENT, SEMANTICOBJECTTYPEΓD_EMAILMESSAGE, etc, which are defined in the header file (semanticruntime.h). The prefened embodiment allows registration of new semantic object types via the Registers em anti cObjectType API. This semantic query processor on the Agency returns the appropriate XML results using the semantic object type as a filter.
In the prefened embodiment, a Skin according to the present invention (see below) uses XSLT (and/or script) to transform the XML returned from the framework (en-route from the XML Web Service) into DHTML. The Shell Extension allows users to select a new Skin for the cunent query. Skins are preferably object-type specific, Context Template specific (for Special
Agents) or Blender specific (for Blenders). Skins can also be customized based on the semantic domain name/path or ontology of the Agent, and based on other attributes such as the user's persona, condition, location, etc. Each Agent is configured on an Agency with a default Skin. The present invention further contemplates custom Skins that may be published onto the root Agency (e.g., on the Global Agency Directory). The client preferably downloads the Skin either from the Agency for the declared Agent or from a central server (e.g., the Global Agency Directory), and applies it to the cunent presentation. The client optionally includes user preferences to ignore Agent Skins or to confine them to a portion of the user interface. Aside from the Skin type (e.g., object Skin, list/layout Skin, Context Skin, Blender
Skin, etc.), in the prefened embodiment, Skins are categorized as follows:
• Design template Skins
• Color template Skins
• Animation template Skins Semantic Skins are preferably required to be interactive, except when they are displayed as part of a tear-off (see above) or Screensaver. Each Skin allows users to seek to a particular point in the "semantic presentation." For example, if the Skin initially displays only the first 25 items, the Skin must have a seek-bar (or other user interface mechanism) to allow the user to seek to the next 25 items, to fast-forward, to rewind, etc. Some Skins have a "Real-Time Mode" option. In this mode, the Skin continuously fetches new objects from the XML Web Service (via pull). Skins are responsible for polling the XML Web Service for new infomiation based on the schema of the desired objects. In the prefened embodiment there are no notifications to the client since the Agency does not maintain any client-specific state for scalability reasons.
Skins optionally include a real-time mode. These Skins are required to be intelligent in that they must cycle through (i.e., present, order or highlight) objects based on priority. For example, if the Presenter relays information indicating that a new object is posted on the Agency, the Skin immediately displays/reorders/highlights this and continues the presentation of the remaining objects. The Presenter determines the ordering and the Skin deals with dynamism given various sort and filter settings. This creates the perception that the semantic presentation is occuning in real-time. In the prefened embodiment, this occurs when there is new data that users are allowed to access using Skins. If the list is time-sorted, the real-time presentation may confuse users due to jumping the user interface into an interactive mode. A user preference option in some modes (e.g., screen saver mode) automatically resets the Skin to display the new data (e.g. scrolling to the top of a sorted list when new data is inserted at the top of the list).
In an alternative embodiment, Skins are designed to customize their presentation based on the amount of available presentation window. For example, a Skin may change from static mode to dynamic mode by displaying infomiation using fade-in and fade-out if, for example, the presentation window is relatively small. Skins are preferably modal depending upon the expected level of user interaction. For example, a screen saver works differently from a browser; a docked view is similarly different (not only because it is smaller, but because it is assumed to be a kind of background view rather than a focus of user interaction). When a view is minimized or hidden, an alternate mode may be used (especially to indicate new infomiation). Examples are audio notification, reminder-like alerts, start-bar show and blink (like outlook reminders). Agents may be used to send email, telephony or Instant Messenger (LM) notifications. In an alternative embodiment, the present invention contemplates an Agent that posts to a Web site (e.g., automatic HTML content generation for event, calendars).
Alternatively, Skins may generate audio-visual infoπnation. For example, a text-to-speech Skin may read out an email object. This feature has great potential value for disabled users and for users of auto-PCs, etc., as well as other uses.
In the prefened embodiment, the Skins framework exposes the following services:
1. Methods to open an SQML-based semantic query. This can be a local SQML document, an Agent, etc.
2. Methods to open an Agent URL directly. 3,. Methods to browse the Infoπnation Agent Semantic Environment.
4. Methods to interface with the system clipboard using customizable clipboard formats.
5. Methods to persist the cunent Skin for a given query or for a given semantic class ID. Skins. As introduced above, Skins are presentation templates that are used to customize users experience on a per-Agent basis. In the prefened embodiment, Skins are XSLT templates and/or scripts that are hosted on a centralized server. Skins according to the present invention preferably generate XHTML+TIME code (e.g., for Presenter display, text- to-speech, Structured Vector Graphics (SVG) via a plug-in, etc.) and access various system services. In the prefened embodiment, Skins support the following features:
1. Display some or all of the fields conesponding to the XML schema of the object(s) being displayed. The Skin optionally provides users a way to uniquely distinguish objects in a returned set or provides users with any conventional access means, for example, filename, URL or personal name (for people).
2. Display a user interface indicating whether the object is understood by the host Agency. Each object preferably includes an "understood" field that indicates this information.
3. For the semantic object type SEMANTICOBJECTTYPE_OBJECT, the Skin optionally displays the raw object metadata or displays the metadata for the XML schema for the class-specific objects that the raw objects represent. For Skins that display class-specific XML schema for queries that refer to raw objects, the Skins must be "smart" to display the class-specific information in different panes.
Prefened ways of accomplishing this uses frames, tabbed boxes, or other user interface techniques. Since every semantic query points to raw objects, the Skin preferably either loads the" query with the filter
SEMANTICOBJECTTYPE_OBJECT (which simply returns raw objects) or the required object type ID. In the prefened embodiment, in order to prepare the presentation of an object list with raw objects of many classes, the Skin should first:
• Get the object query o For each semantic object type, determine how many objects exist in the Agent resource for the given object type. This is preferably obtained by calling the
Agency XML Web Service method GetNumObjectsOfClassInAgent with the
Agent URL and the object type ID name (email, document, event, etc.) as argument. The XML Web Service returns the number of objects in the Agent, satisfying the object type ID filter. • Depending on how many object types there are in the Agent query, the Skin displays frames or other user interface that are appropriate for the number of object types. In the prefened embodiment, when the Skin is ready to load the object type-specific metadata, it calls the Agency's XML Web Service method ExecuteSemanticQuery with the Agent URL and the semantic object type as the arguments
4. When users hover over an object, more metadata for the object is displayable.
5. If a Smart Agent Smart Lens is selected, the Infomiation Agent of the present invention displays contextual metadata that maps the object in the Smart Lens with the object underneath the mouse. In one embodiment, the Smart Lens applies to objects displayed within the Presenter. In alternative embodiment, the present invention allows the Smart Lens to be invoked in other applications (e.g., Microsoft Office applications, the desktop, etc.). This involve installing system hooks to track the mouse and invoke a Smart Lens application when the mouse moves anywhere in the system. The "hook" is called on all mouse events and the hook will also capture the mouse. The Smart Lens may alternatively be invoked asynchronously. In this embodiment, anytime the Presenter displays new results, it checks the clipboard to see if there is any semantic Smart Lens infomiation present. In the asynchronous embodiment, the Presenter automatically caches all the Smart Lens results for all objects in its view. It displays an icon beside each object it presents indicating that there is context-specific related information therein. In a prefened embodiment, users are able to invoke a Smart Lens for any object in the view.
6. Breaking Information. Each object preferably displays a user interface indicating whether there is "breaking infonnation" relating to the object. This is the semantic equivalent of "breaking news." The user interface is preferably presented to indicate the criticality of the infoπnation, yet must not be too intrusive in case users do not want to see the infoπnation. For example, the user interface may be shown as an icon that slowly blinks at a comer of the object display window. When users hover over the icon, metadata on the "breaking infoπnation" is displayed. In the prefened embodiment, "breaking information" is implemented by an implicit Special Agent that invokes calls to all Agents using the Breaking News Context Template.
7. Each object is preferably displayable with a user interface indicating whether the object has any Annotations. This information is included as a field in all query results for all objects.
8. Preferably, each object is displayable with a user interface indicating whether there is related infomiation on any predefined Context Template or Special Agent on the client. This preferably includes Special Agents created by users, as well as default Special Agents (e.g., installed by the client). In the prefened embodiment, Context Palettes for the Context Templates are displayed with the user having the option of displaying one or more of the Context Palettes, hiding them, scrolling them (in order to navigate the Context Palettes), etc. Context Templates and Context Palettes are discussed in further detail below. In an alternative embodiment, Agency priorities preferably include the following: • Critical priority. This is the highest priority. For example, for a given document, this flag will be TRUE (on the Agency) if a related email message was just posted (in this example with a few minutes) or if there is an
J upcoming event that is imminent. • High Priority. This is the next highest priority. The user interface feedback preferably makes it clear that the priority is high enough to wanant users' attention, albeit the feedback must not be very intrusive. The priority is optionally different for different Users, e.g., if there is an event that is local to users the priority might be higher than if the event is remote (particularly if there is no way for the remote user to participate in the event).
• Medium Priority. This may merely indicate that there is infomiation that users should look at if they have the time. The user interface feedback must make this clear.
• LOΛV Priority. This may indicate that there is related information that is germane but not recent.
The four priority virtual Blenders are preferably installed by default on the client. These Blenders automatically aggregate infoπnation from conesponding priority Agents on each Agency in the My Agencies list. There is preferably default priority Agents on every Agency. In the prefened embodiment, relational semantic queries take the context and the user into consideration.
In the prefened embodiment for each Context Template (or the cunently selected Context Template), the Presenter enumerates the Agencies that users add to their My Favorite Agencies list or the recent Agencies, and queries appropriate Agencies using dynamically generated SQML to find out if there are any objects that relate to the cunent object based on the Context Template. If any of the
Agencies in the favorites or recent lists are not accessible, the user interface preferably transparently handles this by ignoring the Agency. In the prefened embodiment, by default, the dynamically generated SQML is created by indexing the SQML of the cunently selected object's SRML and inserting the resource in the SQML as a link filter in the SQML of the Context Template (preferably using the default predicate "relevant to"). This intelligently handles the mapping of the object type of the currently selected object to the semantics of the displayed Context Palette. For example, if the cunently selected object is a document, the Headlines Context Palette uses the SQML based on a derivation of the SQML for the Headlines Context Template. Each Agency in the Semantic Enviromnent semantically processes the resulting SQML appropriately using the default predicate. In another example, if the selected object is a person, the Headlines Palette shows the Headlines relevant to the person, e.g., the "Headlines" authored or a iotated by the person, etc. Altematively, if the cunently selected object is a document or email message, the SQML (with the default predicate) produces semantic results that represent semantically related Headlines on each Agency. These results are preferably displayed in the Context Palette. The same applies to other Context Palettes (e.g., Classics, Newsmakers, etc.). For a person object, the priority flag preferably refers either to objects the person has posted or to objects the person authored or is hosting. In this example, only metadata fields with semantic uniqueness' are preferably used to make this determination (e.g., the person's email address).
9. Each object preferably displays a user interface including a number of manipulation options. By way of example only, a sample user interface illustrating an infomiation object displayed in the Information Agent (semantic browser) Results Pane is shown in FIGURE 54. FIGURE 54 shows a balloon popup (for the object's metadata) and user interface icons on the object allowing the users to invoke tool options such as a Recommendations context pane, a Breaking News context pane, a verbs popup menu, etc. Additional and other user interface options include the following:
• Intrinsic Semantic Links. These are links that are intrinsic to the semantic class of the object. If there are no Intrinsic Semantic Links, nothing needs to be displayed. By was of example, an email object of the prefened embodiment includes the following Intrinsic Semantic Links:
1. From List ->
1. Person A
2. To List ->
1. Person B 2. Person C
3. Cc List ->
1. Person D
2. Person E
4. Bcc List -> 1. Person F
2. Person G 5. Attachments ->
1. Document 1 2. Document 2
3. Document 3
In the prefened embodiment, when any of these semantic links are invoked by users, the client fetches the metadata for the associated object (and not the object itself). This allows users to explore the semantic information for aspects of the original object. The Skin preferably calls the XML .Web Service of the Agency that hosts the object with the appropriate method. In the preferred embodiment, the form of this method is ISemanticRuntimeService::LoadNativeSemanticLink. This embodiment includes the semantic class ID, the name of the semantic link, the name of the argument, and the string form of the argument. For example, to "navigate" to the third attachment (with a zero-based index), the Skin should call LoadNativeSemanticLink( SEMANTICCLASSJ3MAILMESSAGE,
"Attachments", "Index", 2 ). This preferably generated the SQML that represents this relational semantic query, creates a new temporary Smart Agent that has this SQML and loads the Smart Agent. This illustrates prefened semantic navigation. The process is optionally recursive. The user can navigate off the new results using any of the new objects and pivots, etc. An example of a balloon popup associated with an Intrinsic Semantic Link showing an email sample according to the present , invention is shown in FIGURE 55. In this sample user interface, the popup menu is displayed when users selects the "Intrinsic Links" icon on an information object in the Results Pane. This illustration shows what Intrinsic Semantic Links users see for an email object. In the prefened embodiment, the popup menu items invoke a new SQML query (what the proper resource and predicate links) when users hit the menu option. A new temporary Agent is created (with the SQML) showing the results of the query. Users are able to save the Agent in their favorites list. Also, the new results display the Intrinsic Semantic Links, Context Templates, etc., thereby support user-controlled browsing where in users can navigate infonnation semantically. An alternative configuration and functionality for native verbs follows: ALL INFORMATION:
Find Related Infonnation on Agency (only if this came from an Agency)
Find Possibly Related Information on Agency (only if this came from an Agency)
Open Annotations -> All
Annotation 1 Annotation 2 Annotation 2
EMAIL: +=
From List ->
Person A To List -> Person B
Person C Cc List ->
Person D Person E Bcc List ->
Person F Person G Attachments -> Document 1 Document 2
Document 3 PERSON:
Reports To -> Direct Reports -> Member of Distribution Lists ->
Information Authored By -> Infomiation Annotated By ->
Information with categories of which this person is an expert -> CUSTOMER: Information Authored By -> • Annotations. This preferably allows users to navigate to a summary view for all the Annotations for the cunent object. In the prefened embodiment, the Skin displays all the Annotations by calling the ISemanticRuntimeService::EnumAnnotations (with the object metadata as argument). This returns an XML representation of the property table containing the metadata for the Annotation objects. The Skin preferably displays some representation of the Annotation summary being displayed (e.g., names or titles of the Annotations). When an Annotation link is invoked by users, the Skin displays metadata for the Annotation object. These functions preferably come from filters applied on the client. Alternatively, these functions can be created as an Agent. This aspect of the present invention further illustrates semantic navigation. The Annotations are preferably loaded using an SQML representation of the "Annotations" query. This creates a new Smart Agent with this SQML. The Smart Agent is then added to the "recent" list and loaded (or navigated to). The process is optionally recursive. The user can navigate using the newly displayed Anno tation(s) as pivots, etc.
• Related Objects. In the prefened embodiment, this optionally allows users to find related information on each Agency included in the users' My Agencies list using the cunent object as an Information Object Pivot. This is preferably accomplished without resorting to a copy and paste or reliance on the Shell Extension user interface). In the prefened embodiment, the user interface popup shows infonnation in the following format: Find Related Objects -> All my agencies ->
Agency Foo -> A11.A11 All.Understood.All
All.CriticalPriority.All All.HighPriority.All
All.MediumPriority.All All.LowPriority.All
Al 1.MyFavorites . All All.Recommended.All Agencies that understand this object -> Agency Bar -> A11.A11
All.Understood.All All.CriticalPriority.All All.HighPriority.All
All.MediumPriority.All All.LowPriority.All
All.MyFavorites.All
All.Recommended.All The "All my agencies" list is obtained by the Presenter simply by enumerating the Agencies users have registered locally. The Presenter returns the
"Agencies that understand this object" list by "asking" each locally registered
Agency whether it understands the object in question. The Presenter passes the XML representation of the object to the Agency, which attempts to semantically process the XML representation. The Agency returns a flag indicating whether it understands the object. The Presenter optimizes the returned list by excluding the Agency on which the object itself is hosted since each object has a field that indicates whether the Agency understands its contents. o Verbs. This allows users to invoke any actions that relate directly to the cunent object. For example, a document or an email message can have an
"Open" verb. This opens the word processor or email client and displays the infomiation. An event can have an* "Add to Outlook Calendar" verb. In the prefened embodiment, verbs, preferably class-specific, are invoked on the client by the system framework. The Agency need know nothing about verbs.
In the prefened embodiment of the present invention, there are several verbs for every object. These verbs are preferably displayed first in the popup menu.
In the prefened embodiment the verbs include:
1. Annotate. When the user invokes this verb, the Skin preferably communicates with the client runtime and calls the Annotate
• method. This method initiates the default mail client with the appropriate subject line (which the Agency parses to inteφret the Annotation). Users send a regular email message as an Annotation for the object. Email Annotations optionally include attachments that also constitute semantic links. This allows users to navigate from an object (e.g., a document) to its Annotation to its attachment and then to an external content source (e.g., via a Smart Lens). Alternative embodiments are also supported for Annotations, e.g., simple form-based or dialog-based annotations. But email provides the most semantic richness.
2. Copy. This copies the object XML to the system clipboard.
3. Hide. This indicates that users have no interest in viewing the object.
4. Open. This is qualified with the link of what is being opened. In the example of a document, "Open Document" may be displayed. For an email message, "Open Email" may be displayed. The client opens the object with the default application registered in the system for the link's MIME type. In an alternative embodiment, the present invention support other related open verb form, such as "Open with...", .which allows users to open the object with a-; specific application.
5. Mark as Favorite. This is preferably displayed if the Agency supports User State and if the object is not a favorite.
6. Unmark as Favorite, This is preferably displayed if the Agency supports User State and if the object is a favorite.
An example of a balloon popup associated with a Verb user interface according to the present invention is shown in FIGURE 56. In this sample user interface, the popup menu is displayed when users hit the "Verbs" icon on a displayed infomiation object in the Results Pane. The menu shows the. relevant and supported actions for the information object based on the object type (e.g., document, email, person, etc.). An alternative configuration and functionality for native verbs follows: ALL INFORMATION:
• Annotate (Opens Outlook; if the object is from an Agency, the Agency's Email Agent address is filled in the "to" field; if not, the "to" field is left blank so the user can indicate the Agency for object annotation association). If the object is not from an Agency, the object should be attached to the email message either as an URL or as a full-blown attachment). • Copy • Open
• Mark as Favorite (stored on the client)
• Unmark as Favorite
PERSON AND CUSTOMER: += "Send Email" 10. When a Skin loads a new query or the metadata for one or more objects, the Skin preferably calls the framework with the query or the metadata. In the prefened embodiment Skins do not perfomi queries, but passes queries to the Presenter runtime which then managers the results. 11. Deep Information (or Presentation) Mode. An alternative embodiment the iresent invention provides Skin support for Deep Presentation Mode. In this embodiment, the Skin displays a user interface indicating whether there is related infomiation for the cunent object. The Skin also displays text describing the information. For example, for a given document object, the Skin may display a popup with the text "Jane Doe posted the most recent email message that relates to this object: <summary of email message>" In this embodiment, the Skin shows details for specific information, such as the last recently posted related object or the most imminent upcoming object. The Skin may optionally display other "truths" or infened data that might be interesting to users. Examples include: o Lisa Heilbom recently posted a related document: <summary> ° The most likely author of this document is <foo> o Steve Judkins reports to Patrick Schmitz. Patrick has posted 54 critical priority objects that relate to this one.
• This document has 3 likely experts: <names>
• Yuying Chen appears to have the most expertise on this document. The present invention framework exposes several "semantic depth" levels that
Skins use to obtain information. Smart Lenses may also be configured to support Deep Presentation Mode. In other words, in the prefened embodiment, invoking a Smart Lens on an object returns the deep infoπnation similar to what is shown above. The Skin shows an icon at a comer of the object display window. Users are able to click that icon to display the "deep information." Metadata for the
"deep information" can optionally be fetched asynchronously.
An example of a balloon popup associated with a Deep Information Mode user interface according to the present invention is shown in FIGURE 57 as presented in the contexts Results Pane. In this sample, users have the option of selecting a template for the Deep Infonnation that filters what kind of Deep Infoπnation to display, of viewing the "stories" of the Deep Information, along with semantic (SQML) links to objects that are in the Semantic Enviromnent (for example, the "Steve Judkins" person object, the "experts" Context Template results objects, the "direct reports" objects using the "direct reports" predicate filter), etc. In addition, users have the option of previewing the results of the semantic queries in-place using the Preview Player/Control. e. Semantic Query Document
From the client's perspective, every thing it understands is a query document. In the present invention, the client opens "query documents" in a way analogous to how a word processor opens "textual and compound documents." The client is primarily responsible for processing a Semantic Query Document and rendering the results. A Semantic Query
Document is preferably expressed and stored in form of the Semantic Query Markup
Language (SQML). This is akin to a "semantic file format." In the prefened embodiment, the SQML semantic file foπnat consists of the following: o Head. The head tag includes tags that describe the document. o Head: Title — This indicates the title of the document. © Filters. The Presenter filters all returned objects using the entries in the "filters" tag. These entries, optionally contain object type names (documents, events, email, etc.) If no filters are specified, no objects are filtered. The tag has a qualifier that indicates whether the entries are to be included or excluded. In the event of redundant entries (indicated with both "include" and "exclude" tags), the inteφreter excludes the entries (i.e., in the event of a tie, "exclude" is presumed). • Attributes. This tag indicates the attributes of the document. • Skins. This is the parent tag for all Skin-related entries
• skin:<objecttypename>. This contains infoπnation for the Skin to manage objects of the object type indicated in "object type name." The Presenter uses default and Agent Skins for objects that do not have conesponding Skin entries in the SQML document. Options preferably include the following: • skin:<objecttypename>:color. This has information on the color template to be used with this document. The primary entry is an XSLT URL. • skin<objecttypename>:design. This has infoπnation on the design template to be used with this document. The primary entry is .an XSLT URL.
• skin:<objecttypenarne>:animation. This has infoπnation on the animation template to be used with this document. The primary entry is an
. XSLT URL. • Query. This is the parent tag for all the main query entries of the query document, and may include:
• Resource. The reference to the resource being queried. Examples include file paths, URLs, cache entry identifiers, etc. These will be mapped to actual resource manager components by the inteφreter.
• resource:typ,e. The type of resource reference, qualified with the namespace. Examples of defined resource reference types are: nervana:url (this indicates that the resource reference is a well-foπned standard Internet URL, or a custom URL like "agent://...") and nervana:filepath (this indicates that the resource reference is a path to a file or directory on the file-system).
• resource:arg. This indicates an optional string which will be passed to the resource when the inteφreter converts the resource references to actual resources. It is the equivalent of a command line argument to an executable file. Note that some resources might inteφret the arguments as part of the nef, and not as part of the nef argument. For example, standard URLs can pass the nef argument at the end of the URL itself (prefixed with the "?" tag)
• resource:version. See below
• resource:link. All link tags. • resource:Iink:predicate. This indicates the type of predicate for the link. For example, the predicate nervana:relevantto indicates that the query is "return all objects from the resource R that relate to the object O," where R and O and the specified resource and object, respectively. Other examples of predicates include nervana:reportsto, nervana:teammateof, nervana: from, nervana:to, nervana:cc, nervana:bcc, nervana: attachedto, nervana: sen tby, nervana:sentto,- nervana":postedon, nervana ontainstext, etc.
• resourcerlink. This indicates the reference to the object of the semantic link.
• resource:link:type. This indicates the type of object reference indicates in the "oref ' tag. Examples include standard XML data types including xmkstring, xmltinteger; custom types including nervana:datetimeref (which may refer to object references like "today" and "tomonow"), and any standard Internet URL (HTTP, FTP, etc.) or system URL (objects://, etc.) that refer to an object that the present invention can process as a semantic XML object. • resource:link:version. This indicates the version of the resource semantic link. This allows the Agency's semantic query processor to return results that are versipned. For example, one version of the semantic browser can use VI of a query, and another version can use V2. This allows the Agency to provide backwards compatibility both at the resource level (e.g., for Agents) and at the link level. • Query Type. This indicates the type of query (or Agent) this SQML buffer file represents. In the prefened embodiment, this includes Agents, Agencies, Special Agents and Blenders. • Query Return Type. This indicates the type of objects the query returns (e.g., documents, email, Headlines, Classics, etc.). Alternatively, this may indicate names of infomiation object types, Context Templates, etc.
By way of example, SAMPLE B of the Appendix hereto illustrates a Semantic Query
Document in accordance with the present invention.
In the prefened embodiment, the Presenter includes an SQML inteφreter. When the
Presenter opens an SQML file, it preferably inteφrets it by first parsing it, validating it, creating a master entry table, and then executing the entries in the entry table. Effectively, it "compiles" the SQML file before "executing" it, not unlike how a language compiler compiles soμrce code into an object module before it is then linked, with other modules and executed. In the case of the SQML inteφreter, this process optionally involves loading other SQML files via references. This process is preferably not cyclical. The client uses the XSLT templates in the "<skin>" tags (if available and if not overridden by default or Agent Skins) to display the information for each declared object type. Any returned objects that do not have a declared Skin are displayed with the default Skin of the object type or, in the case of a single Agent entry, that of the Agent (if one is specified).
In an alternative embodiment, the client may load a new Skin to display each object type even after the Semantic Query Document is opened. In this embodiment, the "<skin>" tags preferably inform the client which Skin to load the query with initially. In this embodiment, the specified Skin is preferably appropriate for the declared object type.
In the prefened embodiment, the framework executes the document in two phases: the validation phase and the execution phase. For the validation phase, the inteφfeter first builds a master semantic entry table. The table is keyed with the resource .URL and also has columns for the operator, the resource, the resource type, the predicate, the predicate type, and the link. The inteφreter excludes all redundant entries as it adds entries into the table. Also, inteφreter preferably canonicalizes all URLs before it adds them into the table. For example, the URLs "http://www.abccoφ.com" and "www.abccoφ.com/" are inteφreted as being identical since they both share the same canonical form. The inteφreter builds and maintains a separate SQML reference table. This table includes the canonical path to the SQML file. When the interpreter loads the original SQML file, it adds the canonical file path to the reference table. If the SQML file points to itself, the inteφreter ignores the entry or returns an enor. If the. SQML file points to another SQML resource, it adds the new file to the reference table. It then recursively loads the new resource and the process repeats itself. If, during the process, the inteφreter comes across an SQML entry that is already in the reference table, the inteφreter returns an enor to the calling application (indicating that there is a recursive loop in the SQML document). As the inteφreter finds more resources in the document graph path, it adds them to the master entry table for the given resource. It dynamically adds links for a given resource to that resource's entry in the entry table. As a result, the inteφreter effectively flattens out the document link graph for each resource in the graph.
. The inteφreter then proceeds to the execution phase. In this phase, the inteφreter reviews the semantic entry table and executes all the resource queries asynchronously, or in sequential fashion. Next, it processes each resource based on the resource type. For example, for file resources, it opens the property metadata for the file and displays the metadata. For HTTP resources that refer to understood types (e.g., documents), the inteφreter downloads the URL, extracts it, and displays it. For Agent resources, it calls the XML Web Service for each Agent and passes the links as XML arguments, qualifying each link with the operator.
In the prefened embodiment, operators for links that cross document boundaries are always AND. In other words, the inteipreter will AND all links for identical resources that are not declared together because recursive queries are assumed to be filters. The interpreter issues as many calls to a component representing the resource as there are Agent resources. For each link, the inteφreter resolves the link by converting it into a query suitable for processing by the resource. For example, an Agent with a link with the attributes: <predicate>nervana:relevantto</predicate>
<oref>c:\foo.doc</oref> ' <oreftype>neιvana:filepath</oreftype> is resolved by extracting the XML metadata of the object (e.g., c:\foo.doc) and calling the
XML Web Service for the Agent resource with the XML as argument. This illustrates how local context is resolved into a generic (XML-based) query that the server can understand and process.
In order to optimize the query, the Agency XML Web Service exposes methods for passing several arguments qualified with operators (and, or, etc.). The inteφreter preferably issues one call to the XML Web Service for the Agent resource with all the link arguments. Semantic Query Implementation Scenarios. The following are exemplar scenarios illustrating the implementation and operation of Semantic Query Documents according to a preferred embodiment of the present invention.
Scenario 1: Loading an SQML Document. The client creates a temporary file and writes into it a buffer containing the attributes of simple, local HTML page. This page includes the client framework component (e.g., an ActiveX control, a Java applet, an Internet
Explorer behavior, etc.). The page is initialized with this component opening the SQML file and a unique ID identifying the Infomiation Agent instance. The component itself opens the SQML file. In other words, the client framework tells the plug-in what SQML query document to open. The plug-in opens the Semantic Query Document by inteφreting it as described above. Scenario 2: Open Documents. The client opens the standard dialog box, which allows users to select files to be opened. The dialog box is initialized with standard document file extensions (e.g., PDF, DOC, HTM, etc.). When users select the documents, the dialog box returns a list of all the opened documents. The client creates a new SQML file and adds resource entries with the paths of the opened documents. The new SQML file is given a unique name (preferably based on a globally unique identifier (GUID)). Because this is a temporary file, the name is preferably not exposed to users. The methodology proceeds to Scenario 1 as described above.
Scenario 3: Open Folder in Documents. The client creates an SQML file (as described above) and initializes it with one resource entry: file://<foldeφath>?includesubfoIders=(true|false . The SQML file is loaded (as in Scenario 1) by enumerating all the documents in the folder and displaying the metadata for the documents.
Scenario 4: Save as Agent. The client opens a dialog box allowing users to set the Agent name. The client renames the Agent in the Semantic Environment (see below) to the new name. The Agent being saved may be temporary or may already have been saved under a different name. The Information Agent preferably suggests an Agent name.
Scenario 5: Save into Blender. The client opens a dialog box that allows users to select a Blender. The dialog box preferably allows users to create a new Blender. When the Blender is selected, the client opens the Blender's SQML file into the SQML object model and adds the new entry (the cunently loaded SQML file). It then increments the reference count of the cunent entry. Scenario 6: Drag and Drop. The client creates and opens an SQML file with a single resource entry, for example, similar to the following:
<resource type="nervana:url"> agent://documents.all@abccoφ.com <link predicate="nervana:relevantto" type="nervana:filepath" c:\foo.doc </link> </resource> This example assumes that an icon representing "c:\foo.doc" is dragged and drop over an icon in the hifomiation Agent refening to the Agent "agent://documents.all@abccoφ.com."
Scenario 7: Multiple Drag and Drop. The client creates and opens an SQML file with a single resource entry, for example, similar to the following:
<resource type="nervana:uri"> agent://documents.all@abccoφ.com
<link predicate="nervana:relevantto" type="nervana:filepath" •c:\fool .doc </link> <link type="nervana:filepath" operator="or" predicate="nervana:relevantto" c:\foo2.doc </link> <link type="nervana:filepath"> operator="or" predicate="nervana:relevantto" type "nervana: filepath" </link> </resource> This example assumes that multiple icons representing "c:\fool.doc," "c:\foo2.doc" and "c:\foo3.doc" are dragged and dropped over an icon in the Information Agent referring to the Agent "agent://documents.all@abccoφ.com." Also, this example assumes, that users indicate that they want the UNION of the semantic queries targeted at the Agent resource. Scenario 8: Smart Lens. When a Smart Lens is selected in the Infomiation Agent, the Infomiation Agent indicates to the Semantic Environment Manager (see below) that a Smart Lens has been selected for the Infomiation Agent identifier. When the Skin notices that the mouse is over an object (e.g., via the "onmouseover" event in the document object model (DOM)), it calls the Presenter first to find out whether the Infonnation Agent is in Smart Lens mode. The client framework determines this by asking the Semantic Environment Manager if an Information Agent with the identifier is in Smart Lens mode. Because the Semantic Environment Manager caches this information from the Information Agent itself, it can answer the question on behalf of the Information Agent. If the Information Agent is in Smart Lens mode, the client framework preferably obtains the SQML buffer from the system clipboard via the Semantic Environment Manager. This is because a Smart Lens is a virtual "paste" in that it obtains its information from the clipboard. In other words, any object or Agent that is copied to the clipboard can be used as a Smart Lens (even regular text). The framework obtains the SQML buffer and instantiates resource components for every resource in the SQML buffer. The client framework calls the resource API GetlnformationForSmartLens passing the XML information for the cunently displayed object to the resource. All resources preferably return Smart Lens metadata to the client framework. Each resource preferably returns metadata in the fonn of a list of Smart Lens infomiation nuggets. Each nugget contains a text entry and a list of query buffers (in SQML). The text entry contains simple text or a custom text foπnat, for example, similar to the following: Steve reports to <A>Patrick</A>. Patrick posted <A>54 critical-priority messages</A> relating to this one.
Each "<A>" tag pair preferably includes a conesponding SQML query buffer in the infomiation nugget. The client framework foπnats the text into DHTML (or similar presentation foπnat) for display in the Infoπnation Agent (e.g., as a balloon popup or other user interface, preferably not to block or conceal the object that the mouse is over). The client framework displays a user interface for links (analogous to HTML links) where the containing "<A>" and "</A>" tags are found. When a link is invoked, the client framework ι calls the Semantic Enviromnent Manager to create a new cache entry. The Semantic Environment Manager indicates what file-path the entry should be stored in. The client framework writes the SQML buffer for the <A> tag that was clicked into the file. The client framework pushes the SQML document to the Semantic Environment Manager and loads the SQML into the Information Agent (via Dynamic HTML). Because the Semantic Environment Manager includes this SQML document as the cunent document, users are able to save the document via the "save" button in the Infomiation Agent (e.g., "save as Agent" or "save into Blender"). An example of information that a Smart Lens can display is as follows:
. The Agent Email.Technology.All@Marketing has a total of 300 objects that i relate to this object. Critical Priority: 5 objects, High Priority: 50 objects,"
Medium Priority: 100 objects, Low Priority: 145 objects. In the prefened embodiment, if users do not click any of the links in the balloon, no SQML document is created and nothing gets added to the Semantic Environment. This is because the Smart Lens preferably represents only a "potential query."
In the prefened embodiment, any infomiation that can be contained in SQML can be invoked as a Smart Lens (e.g., Agents, people, documents, Headlines, Classics, Agencies, text, HTTP URLs, FTP URLs, files from the file-system, folders from the file-system, email
URLs from an email application such as Microsoft Outlook, email folder URLs, etc). For example, users are able to copy regular text from text-based applications to the clipboard. If users enter the Infoπnation Agent and select the Smart Lens, the SQML version of the text will be invoked as a Smart Lens (via a "document" resource). If the "text Smart Lens" is then hovered over a document object, the document resource representing the text Smart Lens optionally displays the similarity quotient, indicating to users similarities between the Smart Lens object and the object underneath the mouse. If the object underneath the mouse is a person object, the document resource may decide to "ask" the Agent representing the person object whether the Agent is an expert on the information contained in the text. Alternatively, the Smart Lens might display links to similar documents or email messages the person has authored that relate to the text. Scenario 9: Copy and Paste.
Copy: On invocation of a Copy command from within the Semantic Environment, the client framework copies an SQML buffer to the system clipboard with a custom clipboard format. This ensures that other applications (e.g., Microsoft Word, Excel, Notepad, etc.) do not recognize the format and attempt to paste the infomiation. The SQML buffer is preferably consistent with the semantics of the object being copied. For example, a copy operation from an object being displayed in the Presenter is copied as a resource with the appropriate resource type and URL from whence the metadata came. Copying an icon representing an Agent copies the URL of the Agent or the cache entry refening to the t Agent's entry in the Semantic Environment. Copying infomiation from a desktop application (e.g., Microsoft Outlook) copies SQML with a resource type refening to the source application and URLs pointing to the objects within the application. These URLs are preferably resolvable at runtime by the inteφreter to objects within that application. For example, copying an email message from Outlook to be copied into the Semantic
Environment may create a resource entry as follows: <resource type- 'nervana:outlookemailmessage"> outlook://file://c:\temp\foo.html </resource>
Paste: On the invocation of a Paste command, the client framework creates an SQML file based on the clipboard foπnat of the information being pasted. For example, if the clipboard contains a file path, the SQML file contains a link (from the resource on which the Paste was invoked) to an object with the file path. This file is opened as described above. If the clipboard format is an URL, the object is of the URL object type. If the format is regular text, the object contains the actual text with, in this example, the resource type nervana:text.
Alternatively, the client framework creates a temporary cache entry, stores the text there
(e.g., as a .TXT file), and stores the SQML object with a reference to the file path and the object type, in this example, nervana:filepath. When the inteφreter is invoked, it creates an XML metadata version of the text and invokes the resource with the XML link argument. If the clipboard fonnat is the SQML clipboard format of the present invention, a similar process is perfonned, except that if a file is created, the extension will be .SQM (or .SQML). This . indicates to the inteφreter that the object is an SQML file and not just a regular text file.
Semantic Environment
A prefened embodiment of the Semantic Environment of the present invention provides a view of every Agent and Agency available to user via the hifomiation Agent. This preferably includes Agents that have been saved locally into the favorites "My Agents" list, recently used Agents, Agents on local Agencies, and Agents on remote Agencies. Remote Agencies include Agencies that announce their presence via multicast on the local area network, Agencies available on a Global Agency Directory, and Agencies available on a u custom Agency Directory. Agents can be dynamically added to the Semantic Environment by invoking their URL. In the prefened embodiment, the Semantic Environment hierarchy has the pattern shown in SAMPLE C of the Appendix hereto. "Recently Used," "Recently
Created" Agents are preferably collapsed to "Recent Agents." Optionally, "All Agents,"
"Deleted Agents," and "Custom View" may be added.
The Agencies view allows users to see the Agents in the main view by Agency. The object type view allows users to see the same Agents, but filtered by object type. Other views operate in similar fashion, e.g., "By Context" (based on Context Templates) and s'By Time."
The Semantic Environment merges the notion of "favorites" with the notion of "history." The
Semantic Environment optionally adds dynamically managed views such as "recently used
Agents," etc. These views are preferably updated by code ranning within the Semantic
Environment Manager (see below). Exemplar Semantic Environment according to the present invention is shown in
FIGURES 58 and 59. Icons incoφorated into the Semantic Environment may include the following:
Application
All container object types All document file types
Breaking News Agent Icon Qualifier (e.g., an exclamation point) Special Agent Icon Qualifier (e.g., a halo) ., Standard Agent for each object types Agency
Agent View Containers
My Agents
Breaking News Agents Favorite Agents
Special Agents
Recently Used Agents
Snapshots. Users are preferably able to save a snapshot of the Semantic
Environment. A Semantic Enviromnent snapshot essentially is a time-based cache of the state of the Semantic Environment. In the prefened embodiment, a snapshot includes locally stored state with the following infomiation:
• All the Agencies at the snapshot time that have new Agents. o The last Agent creation time of each Agency (based on the Agency's clock).
« The cunent time of each Agency (based on the Agency's clock). Snapshots are preferably accessible to users. The Information Agent filters the Semantic Environment to show only Agencies in the snapshot list, and the Agents in each of those Agencies created between the last Agent creation time and the snapshot time for each Agency. g. Semantic Environment Manager The present invention provides a Semantic Environment Manager that exposes APIs to manage the Semantic Environment objects. In the prefened embodiment, the managed Semantic Environment objects are comprised primarily of Agent references via SQML buffers. The Semantic Environment Manager also exposes APIs to navigate the Semantic Environment. In the prefened embodiment, the Semantic Environment Manager allows instances of the Information Agent to:
1. Register itself at the Semantic Environment Manager. The Semantic Environment Manager preferably maintains infoπnation on all open Infomiation Agent instances. It does this because a number of services (e.g., clipboard access, Smart Lens access, etc.) are performed across applications such as the shell extension application and the Presenter component miming inside a browser control.. For example, when the Presenter loads a new SQML document into the display area, it needs to get a cache entry from the Semantic Environment Manager. It asks the Semantic Environment Manager to create a new cache entry for a given SQML buffer. The Semantic Environment Manager creates the cache entry, writes the SQML buffer to the file- path conesponding to that entry, creates a temporary HTML file initialized with an ActiveX control, Dynamic HTML Behavior, Java applet (or an equivalent client rantime engine) pointing to the cache entry, and returns the cache entry identifier and the file-path to the temporary HTML file to the Presenter. For example, in the prefened embodiment, the temporary HTML file may be named as follows: c:\windows\temp\nervana_39fc54bc-81e5-4954-8cef-3dla54935a0d.htm ii where 39fc54bc-81e5-4954-8cef-3dla54935a0d is the cache entry identifier. The containing Infoπnation Agent automatically detects new documents being loaded (via events in the contained Infomiation Agent control). The containing Infomiation Agent is able to respond when users hit "save" (e.g., "save as Agent" or "save into
Blender"). The Information Agent accomplishes this by getting the cunent document file path, getting the cache entry identifier from the file path (since the file-path is partially named with the identifier), and displaying the metadata for the cache entry (name, description, etc.) when users hits "save as." The Information Agent optionally asks the Semantic Environment Manager to resave the cache entry with a new name. .
The Infomiation Agent registers itself (preferably at startup) with the Semantic Environment Manager with the process ID of its instance. The Semantic Enviromnent Manager allocates a new identifier for the Information Agent and stores metadata for the Infomiation Agent instance (for example, whether it is cunently in Smart Lens mode). The Infomiation Agent stores this identifier. The Information Agent preferably passes the identifier to the Semantic Environment Manager each time it makes a call. The Infoπnation Agent initializes the Presenter with the identifier. In the prefened embodiment, the client framework calls the Semantic Environment Manager with the identifier each time it needs cross-application services. The Semantic Environment Manager stores the process identifier of the Information Agent instance in order to garbage collect all Infomiation Agent entries when the Information Agent processes have terminated. The Semantic Environment Manager preferably accomplishes this in order to remove the Infonnation Agent entry because the Information Agent may not "know" when it is temiinated. Add new Agent references to the Semantic Environment. Agent reference entries are preferably stored in a database, the file-system or a system store (e.g., the Windows registry). In the prefened embodiment, each Semantic Environment entry contains: a. Identifier. This uniquely identifies the Agent in the Semantic Environment. b. Name. This indicates the name of the Agent. The Infoπnation Agent sets a default Agent name when a new Agent is created. This Agent name is set based "on the manner of creation. For example, if document "foo" is copied and pasted over Agent "bar," the Information Agent may create a temporary Agent named "bar" related to "foo" (cunent time). The cunent time is stored to uniquely name the Agent (in the event that users reissue the same query. Users are able to rename the Agent as desired. c. Query Buffer. This indicates the buffer containing the SQML for the Agent. d. Type. This indicates the Agent type (e.g., Standard Agent, Blender, Search Agent, Special Agent, etc.) e. CreationTime. This indicates when the Agent entry was created f. LasfModifiedTime. This indicates when the Agent entry was last modified g. LastUsedTime. This indicates when the Agent entry was last used h. UsageCount. This indicates the number of times the Agent has been used either standalone, as a filter, or as a Smart Lens. i. Attributes. These are the Agent attributes (e.g., noπnal, temporary, virtual, and marked for deletion). If the entry is temporary, it means users have not explicitly saved it as a local Agent. Temporary entries are preferably used in cases where users compose complex queries using drags and drops, but without saving any of the inteπnediate queries as Agents. When users save a query as an Agent, the Information Agent resets the temporary flag indicating that the query entry is now permanent. j. ReferenceCount. This indicates the number of references to the Agent by other Agents and Blenders. The count is initialized to 0 when a new Agent entry is created. : Delete Agents from the Semantic Environment. This is preferably accomplished in two phases. Agents can be marked for deletion, in which case the Semantic
Environment Manager sets a flag indicating that the Agent entry is in the "trash can."
The Agent entry can also be permanently deleted, in which case the entry is removed from the cache all together. 4. Change the properties of an Agent in the Semantic Environment (e.g., reset the temporary flag for an Agent when users save the Agent).
5. Rename Agents in the Semantic Environment.
6. Enumerate the cache to retrieve entries preferably conesponding to: a. All Agents b. Deleted Agents c. The most frequently used Agents d. The most recently used Agents e. The most recently created Agents f. Filters for each object type underneath the aforementioned views (e.g.,
Documents, Email, Events, etc.) g. Filters for Agencies that host Agents in the aforementioned views, filters for object types on the Agencies, and the Agents that fit those views (Documents, Email, etc.) h. Filters for Special Agents based on the Context Template (e.g., Headlines,
Classics, Newsmakers, etc.). For samples of these enumerations and views, FIGURES 12-14 and 17-19 showing the Semantic Environment Tree View.
7. Filter the Agents lisl based on counters updated via invocations from instances of the Information Agent. Each instance of the Infomiation Agent preferably communicates with the one Semantic Environment Manager. That way, updates are user-oriented rather than session-oriented. For example, if users open an Agent in one Infomiation Agent, the Agent entry will show up in the recently used Agents view in another Information Agent. The Semantic Environment Manager maintains information on the number of times each Agent has been used, the last time each
Agent has been used, etc. It filters the Agents. For example, the most frequently used Agents are filtered based on the N Agents with the highest usage counts, where N is configurable and where the filter is only applied after some stabilization wait period (e.g. after the total usage count is at least Y, where Y is also configurable, for example, based on simple heuristics such as the expected number of Agent uses in a two week period). The recently used Agents are filtered based on the usage time (which is stored on a per-Agent basis and which is updated by instances of the Information Agent each time the Agent is used). The recently created Agents are filtered based on the Agent creation time. The deleted Agents are filtered by examining the "marked for deletion" flag on each Agent. The Favorites Agents are filtered by examining the "marked as favorites" flag on each Agent. For each of the aforementioned parent views, the underlying views are populated using simple filters. The Agencies view is populated by examining each Agent returned in the parent view and extracting unique Agencies there from. The object type views underneath each of the Agencies displayed therein are then populated by filtering the Agents based on the Agent object type (e.g., document, email, event, etc.). The Blenders view is filtered by displaying only Agents that have the "Blender" type. The object type Views are directly filtered using the Agent object type. The "My Agencies" view displays local Agencies. Each view underneath this is preferably an object type view filtered using each available Agent on the Agency. The "By Context" view is populated by filtering only for Special Agents (preferably created with a Context Template) and checking for the context name (e.g., Headlines, Classics, etc.).
8. Maintain a reference count for gents in the Semantic Environment. It is the responsibility of the calling component (the Information Agent) to increment and decrement a document entry's reference count. The Information Agent preferably accomplished this by way of a drag and drop, copy and paste, etc. In other words, actions that create new queries that refer to existing Agents. 9 Empty the Semantic Environment. This deletes all Agents 10. Perform garbage collection. The Semantic Environment Manager automatically deletes all old (and temporary) Agents. The cache may be configured to keep a history of Agents up to a certain age. For example, if the cache is configured to only maintain information for two weeks worth of Agents, it periodically checks for temporary Agents that are older than two weeks. If it finds any, it automatically deletes Agent entries that have a reference count of zero. This preferably occurs in cases where the Information Agent creates a new cache entry but does not create another entry (Agent or Blender) that refers to it. In other words, the Information Agent perfomis link-tracking for the immediate link (to avoid complexity). The Semantic Environment Manager optionally performs deep garbage collection. This occurs periodically on a configurable schedule. This applies to entries that have a reference count greater than zero but have no actual references because links were not maintained when other entries were deleted. This feature is incoφorated into the prefened embodiment to minimize complexity because the Information Agent preferably does not track references between Agents and Blenders when Agents and Blenders are saved or edited. In an alternative embodiment, the Presenter performs lazy Agent link-tracking when an Agent is invoked. The client framework ignores all references that have been deleted from the Semantic Environment, analogous to how a Web page returns a 404 (file not found) enor when oiie of its links. has been deleted. In other words, the present invention provides for the situation of incomplete queries. By way of example, a possible scenario may be as follows:
Blender BI -> refers to Blender B2 -> refers to Agent Al -> refers to Agent A2
In this case, the reference count of each entry will be 1, even though the reference count of the chain is 4. As such, it is possible to have stale entries even though the reference counts are greater than zero. For each entry being garbage collected, the garbage collector searches for any reference to the entry in all SQML documents. If no reference is found, the entry is removed (if it is temporary and older than the age limit).
11. Handle notification management. Users are able to register for notifications from any Agent in the Semantic Environment (e.g., saved or local Agents, Standard
Agents, Blenders, etc.). In the prefened embodiment, notification methods include sending email, instant messages, pager messages, telephony messages, etc. The Semantic Enviromnent Manager includes a Notification Manager (see below), which will manage notification requests from users via the Infoπnation Agent. The Notification Manager stores a list of notification requests. A notification request preferably includes the Semantic Environment object ID (which identifies the Agent), the type of notification (email, FM, etc.) and the destination, e.g., the email address, etc. The Notification Manager periodically polls each Agent in the notification request list to "ask" if there are any new objects. The Notification Manager also passes the "last requested time" (based on the destination Agent's clock). The Agent
• responds with the number of new objects (by invoking its stored query and passing back the number of objects in the query results that were created since the "last requested time"). The Agent responds with the cunent time (on its clock). The
Notification Manager stores the Agent's time to avoid time, ■'synchronization problems. Alternatively, the client and all Agencies use the same time server (a time
Web service) to get their time to ensure that all time comparisons will be on the same scale. Agency Directories. In the prefened embodiment, the Semantic Environment Manager preferably maintains an Agency list for each Agency "directory." The multicast network preferably looks to the Semantic Environment Manager as a directory of Agencies. In the preferred embodiment, there is a default Global Agency Directory configured with the URL to an XML Web Service on a public system. This XML Web Service stores a cache of all registered Agencies (preferably with the infomiation described above, including ID, URL, etc.). The XML Web Service exposes methods to allow Agencies to register their presence on the Agency Directory. The XML Web Service filters redundant entries. The XML Web Service also exposes methods to allow users to enumerate all Agencies on the Agency Directory. The Semantic Environment Manager enumerates the directory in this manner. Preferably, the Infonnation Agent considers the Agency Directory as an extension of the Semantic Environment, and allows users to browse and open Agents on the Agencies listed on the Agency Directory. Users are preferable able to add URLs to custom Agency Directories that may be installed on the internal network. The present invention contemplates the creation and integration of customizable Agency Directories. This essentially is an alternative to using multicast for discovery in cases where multicast may not be enabled on the network (for bandwidth conservation reasons) or where certain subnets on the wide area network do not support multicast. h. Environment Browser (Semantic Browser or Information Agent™) The Environment Browser, or Information Agent, hosts a regular Web browser component (such as the Internet Explorer ActiveX control), and is primarily responsible for taking an SQML file and rendering the results via the Presenter. In the prefened embodiment, it does this by opening a local HTML file initialized with a reference to the SQML document cache entry of the SQML file. The HTML file loads the Presenter through a control (e.g., ActiveX, Java, Internet Explorer behavior, etc.). This control retrieves the SQML document from the cache (via the Semantic Environment Manager) and loads the SQML file as described above. The control adds objects to the Web browser document object model (DOM) as it received callbacks from resources indicating that objects are available to be converted to XHTML (or equivalent presentation foπnat, preferably via the cunent XSLT and/or script-based Skin, and pushed into the DOM for presentation. The Infoπnation Agent allows users to open an SQML file or an entry in the cache (via the" cache ID). The Infoπnation Agent also allows users to navigate back and forward, and to navigate the first document in the stack (analogous to the "back," "forward," and "home" options in Today's Web" browsers, the difference being that in this case SQML documents are being opened for inteφretation and display (of the results) as opposed to HTML and other documents). FIGURES 60-68 provide exemplar screenshots of an Information Agent according to a prefened embodiment of the present invention. FIGURE 60 illustrates the Semantic Environment showing a toolbar popup menu option having tools allowing users to import local search results into the Semantic Environment, e.g., via a Dumb Agent, to create a new Special Agent, a new Blender, or a new local Agency. Alternatively, these tools can be collapse into one tool button that invokes a wizard from which users can select the kind of Agent (Dumb, Smart, Special) or Agency they wish to create. FIGURE 61 shows a sample dialog that allows users'to search the Semantic Environment using keywords. This creates a new Smart Agent (with the appropriate SQML). Users are preferably able to customize the name of the new Smart Agent and add an optional description. FIGURE 62 shows the "Save" tool popup menu options of the toolbar that allows users to save a newly created or opened Agent peπnanently into the Semantic Environment (e.g., into the "favorites" list), or to save the Agent into a Blender. FIGURE 63 shows Smart Lens tool menu options of the toolbar that allows users to invoke the Smart Lens (based on the Smart Agent or object1, that is cunently on the clipboard). This communicates to the Presenter that the user wishes to use " the clipboard contents as a Smart Lens. The Presenter preferably automatically invokes the Smart Lens functionality for any object users hover over (e.g., with the mouse). The menu also shows a "Paste as Smart Lens and Pin" option that keeps the Smart Lens turned on (even across Agent navigations) until the user explicitly turns off the Smart Lens. FIGURE 64 illustrates a sample view of the "Open Agent" dialog, showing how users can open server-side Agents from the Semantic Environment and change the "view" of the Environment (e.g., Large Icons, Small Icons, List, etc.). FIGURE 65 illustrates the standard Windows "Open" dialog showing how users can import a "regular" document from the file system into the Semantic Enviromnent of the Infonnation Nervous System. A Dumb Agent is created that refers to the document(s). When the Dumb Agent is invoked, the document(s) is opened in the Infoπnation Agent and all of the semantic tools (e.g., smart copy and paste, Context Templates, etc.) are enabled with the document(s). This illustrates how the browser can make a regular, "stupid" document on the file system semantically "smart." FIGURE 66 shows a custom "Open Documents in Folder" dialog that allows users to search for documents on a folder on the local file system and import them into the Semantic Environment. This makes the documents "smart" by "exposing" them via the semantic tools of the Infomiation Nervous System (e.g., smart copy and paste, Context Templates, etc.). FIGURE 67 shows the "Browse for Folder" dialog box that is shown when users select a browse option. This allows users to select a folder to open (from the local file system). FIGURE 68 shows a page from the "Add Blender" wizard that allows users to select .whether they want to create a standard Blender or a virtual Blender . Additional Application Features
Application Menu Extensions and other Framework Features. The system client preferably installs a menu extension to applications that support programmatic extensions but that do not already support copying data to the clipboard. These include applications such as Microsoft Windows Media Player and Microsoft Outlook (for email message headers). In the prefened embodiment the menu extension reads "Copy." The system copies the selected object as an XML object to the Windows system clipboard. For example, the system plug-in for an email Microsoft Outlook copies a selected email object as an Email XML Object. For applications that already support the clipboard, no extension is needed.
Server-Side Favorite Objects. On Agencies that support User State, users are able to mark objects as "favorites." When an object is marked as a favorite, the Presenter invokes a method on the Agency's XML Web Service. The XML Web Service adds a semantic link i between the user object and the object in question. In the prefened embodiment, users are able to view favorite objects via the All.MyFavorites.All Default Agent. This Agent returns all objects that have been marked as favorites. The Agency administrator is able to create sub-Agents such as All.MyFavorites.Technology.XML.All. The Presenter allows users to mark and unmark favorites, which is also a means of redefining the structure that the servers and Agencies export. The use of "favorites" scenario is especially valuable in cases where users may see objects of interest and not want to navigate them immediately. The favorites feature may optionally be also used by the Agency to recommend objects to users. In the prefened embodiment, these recommended objects are retrievable via the All.Recommended.AU Agent. The Agency recommends objects based primarily on objects that users have marked as being favorites. Server-side favorites will also preferably be used with the "favorites," Classics and Recommendations Context Templates.
Agent Screen Savers. A prefened embodiment of the present invention allows users to select any subscribed Agent as a screen-saver. Users are preferably warned that Agents may expose sensitive data and given an opportunity to determine whether it is safe to use a particular Agent as a screen-saver. In the prefened embodiment, the system client is capable of loading any subscribed Agent as a screen-saver. In an alternative embodiment, users may combine Agents to provide a desired screen-saver presentation. Alternatively, a screen-saver may be a stractured Skin that includes displayed parallel Agents, for example, in four quadrants of the screen. Agent-Agent Smart Lens. In an alternative embodiment, the system client supports the use of a Smart Lens (invoked either tlirough an Agent or a Blender) as' context to invoke another Agent or Blender. For example, users may select All.CriticalPriority.All and want to use that Agent as a Smart Lens to browse All.Understood.All in order to find out all objects that are critical priority and which are also understood by the destination Agency.
Smart Lens Sample User Interface Illustrations. FIGURES 69-71 provide exemplar balloon popup menus associated with the Smart Lens feature of an Information Agent according to the present invention. FIGURE 69 shows a sample of a balloon popup menu in the context Results Pane with a Smart Agent as the Smart Lens. This shows a popup window that is displayed when users select the Smart Lens icon on an infomiation object. This sample shows a case where the Smart Agent titled "Documents on Reuters Related to [My Nervana UI Specification] is on the clipboard and is "posted" as a Smart Lens over an email object titled "Yuying's Thoughts on the Nervana UI.". FIGURE 70 shows a sample of a balloon popup menu in the context Results Pane with an object as the Smart Lens (and "hovered" over an Agent). This sample illustrates that the Smart Lens is connotative (A [SMART LENS] B = B [SMART LENS] A). The results section of the context pane is identical to in the example shown in FIGURE 69, indicating that the Smart Lens in the prefened embodiment is connotative. FIGURE 71 shows a sample of a balloon popup menu in the context Results Pane with an infonnation object as the Smart Lens and an infoπnation object as the item being "lensed over." In this sample, an object titled "My Neryana UI Specification" has been copied to the clipboard (its SQML representation) and pasted as a Smart Lens over another object (in the Results Pane) titled "Yuying's Thoughts on the Nervana UI" (an email object). In this sample, the user has the option of selecting a predicate that is semantically consistent with the combination of a document to an email message. FIGURE 72 shows a sample of a variant of the balloon popup menu of FIGURE 71 showing the relatedness measure of the two objects (the Smart Lens object and the "lensed over" object), both as a percentage and graphically, in this example as a bar chart.
' FIGURES 73-75 show sample tables illustrating the behaviors and relational contains objects types predicates when using Smart Lenses. FIGURE 73 shows the Agent-Object scenario for all infomiation wherein the Smart Lens behavior is commutative, for example, A [Smart Lens] B = B [Smart Lens] A. FIGURES 74-75 show the Object-Object scenario for document and email, respectively, also wherein the Smart Lens behavior is commutative, for example, A [Smart Lens] B = B [Smart Lens] A.
Blender Skin User Interface Illustrations. FIGURE 76 is a user interface sample illustrating semantic results Player/Preview Control. The Information Agent Presenter preferably attaches this control to each Results Pane. The Player/Preview Control allows users to navigate the results in the Results Pane, to animate the results (play, stop, pause, change, speed up, etc.) and to filter the results (e.g., in the case of the results of a Blender). FIGURE 77 is a user interface sample showing the semantic results of a Blender. In this sample, the Blender Skin has reserved parts of the display area as separate frames for each Agent in the Blender, and attached a Player/Preview Control to each frame, thereby allowing users to individually navigate, control and animate the results of each Agent in the Blender. Alternatively, a Blender Skin can display the merged results from all the Agents in the Blender (with one Player/Preview Control attached), can display the results in frames according to information object type, etc.
Multiple Drag and Drop. In an alternative embodiment, the system client allows users to select multiple documents or folders from the desktop and use them as the basis of relational queries on an Agent or Blender. This allows users to further refine a query using multiple documents as the refining tool. For example, the user may optionally indicate whether they want the union or intersection of the results (using each of the documents as a filter). This creates an SQML file with one resource (the object over which the links were dragged) and multiple links (one per document or dragged object). The client's SQP preferably interprets this by retrieving the XLM metadata for all the object filters and calling the destination Smart Agent's XML Web Service with the XML arguments, hi the prefened embodiment, the Agency's XML Web Service categorizes the XML metadata arguments, forms the proper SQL representation of the query and returns the results.
URL Shortcut Conventions. Agencies of the present invention may share the
Internet Web since they are optionally installed as Web applications. As a result, Agencies can be refened to using the Web's naming scheme (e.g., a regular HTTP URL). In the prefened embodiment, the present invention exposes shortcut naming conventions and URLs that are specific to the Infoπnation Agent's Semantic Environment. o Agent Shortcut URL Convention. The Agent shortcut URL convention is: agent://<agentname>@<agencyurl>?start=<start>&end=<end>&skin= <skin urlL>
When invoked, this is preferably mapped to a fully-qualified HTTP URL, for example: http://<path to Agency ASP; or
CGI script>?agentname=<agentname>& start=<start>&end=<end>&skin=<SkinUrl>.
An example of an Agent shortcut URL convention is as follows: agent://email.technology.wireless.all@marketing.abccoφ.com?start=0
&end=25&skin=http://www.nervana.net/skins/email/abcemailskin.xslt
This URL is resolved by the client as follows: Start the Web service proxy, open the WSDL file http://abc.com/nervanaroot/webservice.wsdl and ask the Web service for the statistics of the Agency named "Marketing." For HTTP access, this will be resolved to a path to the ASP or CGI. For example: http://abccoφ.com nιarketingagency.asp?urltyρe=agent&agentname= email.technology .wireless. all& start=0&end=25&skin::=http://www.ι ervana.net/skins/email/abccoφe mailskin.xslt
The start argument indicates the zero-based starting index of the object to return first. The end argument indicates the end index. The Skin URL is optional. If no Skin URL is specified, the client loads the Agent with the Agent's default Skin.
A locally saved Agent may be accessed with agent://<agentname>@localhost. For example: agent://Documents. [Related to My Business Plan]@localhost will load the locally saved Agent (in My Agents) named "Documents. [Related to My Business Plan]". Agency URL Convention. An example is as follows: agency://<agencyname>.<domainname>?query=getproperties|getstats] getagents@agentviewfilter=<agentviewfilter>&agentnamecontaiiisfilt er=<agenthamecontainsfilter>&agenttypefiltei-<agenttypefilter>&age ntobj ecttypefιlter=<agentobj ecttypefilter> In this example, the query argument is "getproperties". The URL retrieves the properties of the Agency itself (e.g., the name, the display name, whether it is local or remote, etc.). Alternatively, if the property is "getstats," the URL retrieves the statistics of the Agency (total number of Agents, number of Standard Agents, number of Compound Agents, number of Domain Agents, total number of objects, number of document objects, number of email objects, etc.). In the prefened embodiment, the getproperties flag is the default, meaning that the properties are retrieved if no other argument is specified. If either the getproperties or getstats arguments are specified, preferably no other arguments are specified alongside. The agentviewfilter argument is optional and allows the caller to specify an Agent view within with to restrict the search. For example, an Agent view "Reuters News" may be installed on the server to only return Agents that manage news objects from Reuters. The agentnamecontainsfilter argument is optional and allows users to filter the results by a search string for the Agent name. The agenttypefilter is optional and allows users to filter Agents based on Agent type
(Standard Agent, Compound Agent, or Domain Agent). The agentobjecttypefilter. argument is optional and allows users to filter the results with the object type the Agent manages (e.g., email, documents, people, etc.). Examples include the following: f agency://sales.boeing.com?query=:getstats (conesponding to the HTTP URL http://boeing.com/salesagency.asp?urltype=agency&query=getstats)
agency://sales.boeing.com?agenttypefilter=standard&agentobjecttypei dfilter=events (conesponding to the HTTP URL
http://boeing.com/salesagency.asp?urltype=agency&agenttypefilter=st andard&agentobjecttypeidfilter=events • Objects URL Convention. Agency objects can be accessed directly from a client. The URL convention is:
I objects://<querystring><agencyname>.<domainname>?querytype=<o bjectid|searchstring>&objecttypefilter=<objecttypefilter>
The objecttypefilter argument is optional and can be used to filter the returned
;ι objects by object type. It is an enumeration of known object types (e.g., document, email, event, etc.). Examples include the following: objects://34547848@suppbrt.attwireless.com?querytype=objectid will return the object with the objectid 34547848.
objects://80211@support.attwireless.com?querytype=searchstring&obj ecttype=email will return the email objects matching the query string
"80211"
• Category URL Convention. The URL convention is: category://«categoryname>@<kbsurl>?semanticdomaiιmame=<sema nticdomainname> The semanticdomainname argument is optional. In the prefened embodiment, if it is left out, the default domain of the KBS will be selected. An example is as follows: category://tecluιology.wireless.all@abccoφ.com/marketingknowledge .asp
This conesponds to the "Technology. Wireless.All" category for the default domain on the knowledge-base installed on the abccoφ.com/marketingknowledge.asp web service. This will be resolved to the following HTTP URL: http://abccoφ.com/marketingknowledge.asp?category="technoldgy. wireless, all. An example of a fully qualified version of the category URL may be: category://technology.wireless.all@abccoφ.com/marketingknowledge .asp?semanticdomainname- 7InformationTechnology"
Sharing and Roaming Client Information. In the prefened embodiment, users are able to share Agents (including Blenders) with others by sending them via email, instant messaging, etc. Local infoπnation users are preferably able to either store Agent information locally or have the infonnation roam with them (e.g., via AbccoφliMinor support in Windows 2000 for department-wide roaming, via a proprietary XML Web Service on a Global Agency Directory (using passwords for identity), or via integration with Microsoft .NET My Services, which employs Microsoft's Passport identity service).
Local Agencies. The system client preferably also allows users to create and add local Agencies that run a local instance of the KIS to the "My Agencies" list. In this embodiment, the client also allows users to delete a personal Agency.
User-Experience Consistency and Non-Disruptiveness. The Infoπnation Agent (semantic browser) of the present invention provides a consistent and undisruptive user experience. In other words, the hifomiation Agent seamlessly coexists with Today's Web browser. Tools such as "Back," "Forward," "Home," "Stop," "Refresh," and "Print" preferable work as they do with Today's Web browser so as not to confuse the user. Many of the tools remain the same albeit the functionality is different. In addition, new tools are preferably added to the toolbar and menu options reflecting the new functionality in the semantic browser (these can be seen by obseiving the toolbar in the screenshots). FIGURES 78 and 79 illustrate exemplar functionality mappings of the present invention demonstrating prefened mappings for introducing new functionality to users while maintaining metaphor consistency. FIGURE 78 is a comparison of default user interface toolsets for Today's Web browser and a prefened embodiment of the Infonnation Agent of the present invention. FIGURE 79 is a comparison of default user interface toolsets for the file system Microsoft Explorer/Document Viewer and a prefened embodiment of the Infomiation Agent of the present invention.
5. PROVIDING CONTEXT IN THE PRESENT INVENTION a. Context Templates The present invention provides Context Templates, or scenario-driven infonnation query templates that map to specific semantic models for infonnation access and retrieval. Essentially, Context Templates can be thought of as personal, digital semantic infomiation retrieval "channels" that deliver infonnation to a user by employing a predefined semantic template. In the prefened embodiment, the semantic browser 30 allows the user to create a new "Special Agent" using Context Templates to initialize the properties of the Agent.. Context Templates preferably aggregate infoπnation across one or more Agencies.
By way of example only, the present invention defines the following Context
Templates. Additional Context Templates directed towards the integration and dissemination of varied types of semantic infomiation are contemplated within the scope of the present invention (examples includes Context Templates related to emotion, e.g., "Angry," "Sad," etc.; Context Templates for location, mobility, ambient conditions, users tasks, etc.).
"Headlines" Context Template. The Headlines Context Template (and its resulting
Special Agent) can be analogized to a personal, digital version of CNN's "Headline News" program in how it conveys semantic infoπnation. The Context Template allows a user to access infoπnation headlines from one or more Agencies, sorted according to the infoπnation creation or publishing time and a configurable amount of time that defines information "freshness." For example, CNN's "Headline News" displays headlines every 30 minutes
(around the clock). In a prefened embodiment, the hifomiation Agent 30 of the present invention allows users to create a Headlines Special Agent using the following filters and parameters: • Information Object Pivots. The resulting Blender shows result that relate to these object. This is an optional parameter. If it is not specified, headlines are displayed for the entire Agency (without any object-based filter).
• Predetermined "freshness" period. For example, 30 minutes, 1 hour, etc.
• Predicate. This will define how the Information Object Pivot links to the information to be retrieved. Examples are: "related to," "possibly related to" (uses a text-based search), "authored" (in the case of a person object), "possibly . authored," "has expertise on,"' etc. The default predicate "relevant to" is preferably used by default. This default predicate is resolved by the Agency by intelligently mapping it to specific predicates. • Agency(ies). This includes the Agencies on which to check for headlines. At least one Agency must be specified and there is no limit to the number of Agencies that can be specified. The user may indicate whether all Agencies in the "recent" and/or "favorites" lists should be used. o Category list. For example "Technology. Wireless.AH". This acts as an additional filter for the query.
In addition to freshness, the Headlines Context Template preferably incoφorates how
"hot" the result items are in order to determine the ranking of the results. This may be accomplished by querying the Agency to find out the number of semantically related objects on the Agency, which is a good indicator of whether an object's topic is "hot." In addition, returned objects (or items) are preferably sorted by freshness or as new.
By way of example, SAMPLE D of the Appendix hereto illustrates an SQML output from a Headlines Context Template of the prefened embodiment. In this example, the Context Template retrieves all infomiation from four different Agencies (marketing, research, sales, and human resources), with a freshness time span of 30 minutes, and with a "relevant to" predicate (indicating a semantic query). In the prefened embodiment, the SQML of this example, as for all Context Templates, can optionally fomi the basis of a Smart Lens, smart copy and paste, drag and drop and other tools in the semantic toolbox.
"Breaking News" Context Template. The Breaking News Context Template (and its resulting Special Agent) can be analogized to a personal, digital version of CNN's "Breaking News" program inserts that intenupt regularly scheduled programming in how it conveys semantic infoπnation. Like CNN's "Breaking News" inserts, this Context Template allows users to access "breaking," time-critical information from one or more Agencies, preferably sorted by the infonnation creation or publishing time or the event occunence time (in the case of event), and with a configurable amount of time that defines freshness and a configurable "deadline" for events to define time-criticality. For example, the Context Template can be defined to filter information objects posted in the last one-hour, or events holding in the next one day.
In the prefened embodiment, the Breaking News Context Template is different from Breaking News Agents. The Context Template is a template that defines static query parameters that are passed to one or more Agencies. A Breaking News Agent is any Smart Agent users may have created and is essentially user-created and user-customizable. By way of example, a Breaking News Special Agent based on the Breaking News Context Template may inform users of information objects posted in the last hour or events holding in the next day that relate to a local document (or any other local context, if specified). But a Breaking News Agent gives users the flexibility of receiving alerts for "Events on wireless teclinology being given by a member of my team and holding either Seattle or Portland in the next 24 hours and which relate to this document on my hard drive." The Breaking News Agent provides users much greater flexibility and personalization than the Breaking News Context Template. An advantage of the Breaking News Context Template is that it preferably forms the basis for intrinsic alerts by using parameters that qualify as "breaking" for typical users. "Conversations" Context Template. The Conversations Context Template (and its resulting Special Agent) can be analogized to a personal, digital version of CNN's
"Crossfire" program in how it conveys semantic infomiation. Like "Crossfire," which uses conversations and debates as the context for infoπnation dissemination, in the prefened embodiment, the Conversations Special Agent tracks email postings, annotations, and threads for relevant infonnation. The Conversations Context Template may be thought of as the
Headlines Context Template filtered with email object type. In addition to the "Headlines" parameters, the Conversations Context Template preferably (but optionally) contains the following parameters: ' • Minimum thread length to return. The user optionally indicates that he or she only wants email threads with at least one reply, two replies, etc. In many instances, the number of threats provides an indication of semantic significance. The default is zero.
• Distribution list filter. The user optionally restricts the returned email to those that have members of one or more distribution lists on the "from," "to," "cc," or
"bcc" lines. This allows the user wants to monitor debates from prefened groups, divisions, etc.
• Distribution line filter. The user optionally restricts the returned email to those that have the filter email addresses on the "from," "to," "cc," or "bcc" lines. The returned items are optionally sorted based on freshness or based on the depth of the conversation thread.
"Newsmakers" Context Template. The Newsmakers Context Template (and its resulting Special Agent) can be analogized to a personal, digital version of NBC's "Meet the
Press" program in how it conveys semantic infomiation. In this case, the emphasis is on "people in the news," as opposed to the news itself or conversations. Users navigate the network using the returned people as Infomiation Object Pivots. The Newsmakers Context
Template can be thought of as the Headlines Context Template, preferably with the "People" or "Users" object type filters, and the "authored by," "possibly authored by " "hosted by,"
"annotated by," "expert on," etc. predicates (predicates that relate people to information). The "relevant to" default predicate preferably is used to cover all the germane specific predicates. The sort order of the relevant infonnation, e.g., the newsmakers, is sorted based on the order of the "news they make," e.g., headlines. In addition to the Headlines Context Template parameters, the Newsmakers Context Template preferably contains the following optional parameters:
■•. • Distribution list filter. The user optionally restricts the returned email to those that have members of one or more distribution lists on the "from," "to," "cc," or
. "bcc". lines. This allows the user wants to monitor debates from prefened groups, divisions, etc. • Distribution line filter. The user optionally restricts the returned email to those that have the filter email addresses on the "from," "to," "cc," or "bcc" lines.
"Upcoming Events" Context Template. The Upcoming Events Context Template
(and its resulting Special Agent) can be analogized to a personal digital version of special programs that convey information about upcoming events. Examples include specials for events such as "The World Series," "The NBA Finals," "The Soccer World Cup Finals," etc. The equivalent in a knowledge-worker scenario is a user that wants to monitor all upcoming industry events that relate to one or more categories, documents or other Infoπnation Object Pivots. The Upcoming Events Context Template is preferably identical to the Headlines Context Template except that only upcoming events are filtered and displayed (preferably using a semantically appropriate "context Skin" that connotes events and time-criticality). Returned objects are preferably sorted based on time-criticality with the most impending events listed first.
"Discovery" Context Template. The Discovery Context Template (and its resulting Special Agent) can be analogized to a personal, digital version of the "Discovery Channel." In this case, the emphasis is on "documentaries" about particular topics. Unlike in the case of "Headline News," the primary axis for semantic information access and retrieval is not time. Rather, it is one or more category with an intelligent aggregation of infomiation around those categories. In a prefened embodiment of the present invention, the Discovery Context Template simulates intelligent aggregation of information by randomly selecting infoπnation objects that relate to a given set of categories and which are. posted within an optionally predetermined, configurable time period. While there is an optional configurable time period, the semantic weight as opposed to the time is the prefened consideration for deteπnining how the infomiation is to be ordered or presented. The present invention allows for different axes to be used, for example, the semantic weight for the category or categories being "discovered," time, randomness, or a combination of all axes (which would likely increase the effectiveness of the "discovery"). The Discovery Context Template preferably has the same parameters as the Headlines Context Template, except that the freshness time span is replaced by an optional maximum age limit, which indicates the maximum age of infomiation (posted to the Agency) that the Agent should return.
"History" Context Template. The History Context Template (and its resulting Special Agent) can be analogized to a personal, digital version of the "History Channel." In this case, the emphasis is on disseminating infoπnation not just about particular topics, but also with a historical context. For this template, the prefened axes are category and time. The History Context Template is similar to the Discovery Context Template, further in concert with "a minimum age limit." The parameters are preferably the same as that of the Discovery Context Template, except that the "maximum age limit" parameter is replaced with a "minimum age limit" parameter (or an optional "history time span" parameter). In addition, returned objects are preferably sorted in reverse order based on their age in the system or their age since creation.
"AH Bets" Context Template. The All Bets Context Template (and its resulting Special Agent) represents context that returns any information that is relevant based on either semantics or based on a keyword or text-based search. In this case, the emphasis is on disseminating infoπnation that may be even remotely relevant to the context. The primary axis for the All Bets Context Template is preferably the mere possibility of relevance. In the prefened embodiment, the All Bets Context Template employs both a semantic and text- based query in order to return the broadest possible set of results that may be relevant.
"Best Bets" Context Template. The Best Bets Context Template (and its resulting Special Agent) represents context that returns only highly relevant infomiation. In a prefened embodiment, the emphasis is on disseminating infomiation that is deemed to be highly relevant and semantically significant. For this Context Template, the primary axis is relevance. In essence, the Best Bets Context Template employs a semantic query and will not use text-based queries since it cannot guarantee the relevance of text-based query results. The ' Best Bets Context Template is preferably initialized with a category filter or keywords. If keywords are specified, categorization is performed by the server dynamically. Results are preferably sorted based on the relevance score, or the strength of the "belongs to category" semantic link from the object to the category filter.
"Favorites" Context Template. The Favorites Context Template (and its resulting Special Agent) represents context that returns "favorite" or "popular" infomiation. In this case, the emphasis is on disseminating information that has been endorsed by others and has been favorably accepted, hi the prefened embodiment, the axes for the Favorites Context Template include the level of readership interest, the "reviews" the object received, and the depth of the annotation thread on the object. In one embodiment, the Favorites Context Template returns only infoπnation that has the "favorites" semantic link, and is sorted by counting the number of "votes" for the object (based on this semantic link).
"Classics" Context Template. The Classics Context Template (and its resulting Special Agent) represents context that returns "classical" infonnation, or information that is of recognized value. Like the Favorites Context Template, the emphasis is on disseminating infoπnation that has been endorsed by others and has been favorably accepted. For this Context Template, the prefened axes includes a historical context, the level of readership interest, the "reviews" the object received and the depth of the annotation thread on the object.. The Classics Context Template is preferably implemented based on the Favorites Context Template but with an additional minimum age limit filter, essentially functioning as an "Old Favorites" Context Template. "Recommendations" Context Template. The Recommendations Context Template
(and its resulting Special Agent) represents context that returns "recommended" infomiation, or infomiation that the Agencies have infened would be of interest to a user. Recommendations will be inserted by adding "recommendation" semantic links to the "SemanticLinks" table, and by mining the favorite semantic links that users indicate. Recommendations are preferably made using techniques such as machine learning and collaborative filtering. The emphasis of this Context Template is on disseminating information that would likely be of interest to the user but which the user might not have already seen. For this Context Template, the primary axes preferably include the likelihood of interest and freshness. In the prefened embodiment, the Context Template is implemented by generating SQML that has the PREDICATETYPEΓDJSLIKELYTOBEINTERESTEDL predicate as the primary predicate filter on the Agencies in the Semantic Environment.
"Today" Context Template. The Today Context Template (and its resulting Special Agent) represents context that returns infonnation posted or holding (in the case of events) "today." The emphasis with this Context Template is preferably on disseminating information that is deemed, to be cunent based on "today" being the filter to detennine freshness. In the prefened embodiment, the Today Context Template results are a subset of the Headlines Context Template results wherein the results posted "today" or events holding "today" are displayed..
"Variety" Context Template. The Variety Context Template (and its resulting Special Agent) represents context that returns random information. The emphasis with this Context Template is preferably on disseminating information that is random in order for the user to get a wide range of possible infomiation items. In the prefened embodiment, the primary axis is randomness, albeit the "random" items will be semantically relevant to the query filter (using the "relevant to" predicate). b. Context Skins The present invention includes a special class of Skins called "Context Skins."
Context Skins include presentation infonnation that conveys the semantics of the context that they represent. For example, a Context Skin for the Today Context Template may display a background or filter effects with a clock pointing to midnight, or some other representation of "Today." In yet additional examples, a Context Skin for the Variety Context Template may show transfonn effects like bowling balls falling over randomly (indicating the randomness of the results); the Breaking News Context Skin may show effects and light animations with flashing text, ambulance red lights, etc. to indicate the criticality of the context; and the History Context Skin may show graphics that indicate "age"; for example, old cars, clocks, etc. Context Skins preferably "honor" the presentation template for object types being displayed. For example, email objects may be displayed with a background showing stamps or a post office truck in addition to graphics that indicate the Context Template. Because some Context Templates cut across Agencies — and therefore cut across ontologies — they need not display any information that indicates ontology (e.g., industry information). However, Context Skins that are initialized with a category filter preferably indicate the category or ontology of the Context Template. Typically this will be represented with graphics elements (and filters, transforms, etc.) that indicate the industry or genre of the ontology. For example, a Pharmaceuticals Context Skin may have filter effects showing laboratory equipment; an Oil and Gas Context Skin may show pictures of oil rigs; and a Sports Context Skin may show pictures of sports gear, etc. c. Skin Templates
The present invention allows a user to select different kinds of Skins, depending on the task at hand. The implication of having flexible presentation is that the user can select the best presentation mode based on the cunent task. For example, users may select a subtle Skin when working on their main machine and where productivity is most critical and where effects are not. Users may select a moderate Skin in cases where productivity is also important but where effects will also be nice to have as well. Users may select an exciting Skin for scenarios like second machines, for example where users are viewing infoπnation in" their peripheral vision, and features such as text-to-speech to alert them on breaking news is important. Exciting Skins may feature animations, storyboard like effects for deep infomiation, objects displayed on motion paths, and other effects. Exciting Skins are most likely going to be used with screensavers. The choice of Skins is preferably user-definable. d. Default Predicates
In the prefened embodiment, each object type includes a default predicate that links it with other object types. This provides users with an intuitive method of dynamically linking objects together without requiring a separate evaluation of the predicate to use for the semantic link. For example, a drag and drop operation from a document object to an Agent that returns documents can have the predicates "Related To" and "Possibly Related To."
When a document object is dragged on top of a document Agent, the semantic browser of the present invention displays a popup menu option that allows users to select the predicate to use for the semantic query. In an alternative embodiment, other related popup menus may be incoφorated, e.g., a first popup menu that allows users to select the link or predicate template; child popup menus that display the actual predicates for the selected template. The default predicate is preferably inserted in the dynamically generated SQML from which the query will be invoked. By way of example, a default predicate may be "relevant to." This predicate maps to a query that returns infonnation in the document Agent that is relevant to the object being dragged. The advantage of having a default predicate in this case is that the semantic browser of the present invention may display a popup menu option named "Open" that in turn invokes a query using this predicate. The semantic browser may also display a popup menu option named "Open with Link" that has submenu options with specific predicates. The default predicate makes the system easier to use because users are able to browse the system using dynamic linking, knowing that the default predicate will be the sensible option giving the source object and that target Agent or object. In addition to being used in drag and drop scenarios, Default Predicates are optionally used in Smart Lenses, smart copy and paste, etc. Default Predicates may be analogized to degenerate smart links that return "the right thing" given the context. Preferably the default predicate will be "relevant to," which may in turn produce "The right thing" as the appropriate query result for a semantic distance of one. In an alternative embodiment, the Default Predicate may be a merger of several specific predicates. For example, the Default Predicate for a document-to-people drag or drop, copy or paste, or Smart Lens may be "relevant to" and may be inteφreted by the KIS Agency XML Web Service as, for example, a cascaded query involving "authored," "expert on," and "annotated" predicates. In other words, "relevance" is inteφreted smartly by the present invention and may involve merging together different predicates.
Default Predicates allow users to navigate the system quickly and efficiently and with little thought. Default Predicates provide the system with simplicity and make it intuitive to use. In addition, users are comfortable with Default Predicates because users are already used to invoking HTML links on Today's Web where there is only one predicate: "invoke". e. Context Predicates
Context Predicates are predicates that are defined at a high level of abstraction and which map to a relevant subset of the Context Templates. Context Predicates allow a user to select a predicate filter based on a Context Template, rather than on a low-level system predicate. When the query is invoked with the Context Predicate, filtering the containing SQML with the filter parameters of the Context Template generates a new SQML query. For example, he Context Predicate "Best Bets" maps to the Context Template of the same name and filters a query with those infomiation objects that are "best bets" (typically, these will be those items that are returned from a semantic query and not from a text-based query). Similarly, the Breaking News Context Predicate filters items based on whether they qualify with the filter conditions of the Breaking News Context Template. In general, Context Predicates are applied for object types that are consistent with the Context Template (for example, the Context Predicates "Experts" and "Newsmakers" will only be valid for queries that return "Person" objects). / Context Attributes
Context Attributes are "virtual attributes" that are cached as part of each XML object that an Agency returns to the client. These attributes are dynamic in that they reflect the cunent context in which the results are being displayed. For example, where relevant, the Context Attribute "Best Bet" is attached to each XML result that satisfies the semantic query filter in the SQML of the cunent query. The results of a semantic query with default predicates might include both semantic and non-semantic (text-based query) results. The Agency processing the query may cache Context Attributes for the XML results that are "Best Bets" by running a semantic sub-query on the SQML with the result object as a filter. In this case, the schemas for the "Object" and derived types should include attribute fields for each relevant Context Template (e.g., a "Best Bet" attribute, "Headline" attribute, etc.). This is the prefened implementation. Alternatively, the semantic browser calls the Agency, passes each XML object as an argument and "asks" whether the object satisfies the Context Attribute. Other examples are a Headline Context Attribute that indicates whether the object qualifies as a "Headline" in the context of the cunent query, a "Classics" attribute, etc. The semantic browser should display a user interface indicating whether the context attribute is set or not.
Context Attributes provides further benefits over the prior art systems in that they ' make the system easier to use. For example, a user can perfonn a drag and drop operation to generate a relational query that includes both semantic and non-semantic query filters (as processed by the Agency when it receives the SQML arguments from the client). In one embodiment, the browser "asks" the user whether he or she desires a broad query or a "Best Bets" query. In this mode, the user effectively applies for an additional filter before the query is issued. Alternatively, the Agency, in concert with the semantic browser, preferably returns the results of the broad query, and also qualifies each result with a context attribute and conesponding user interface indicating whether each result object is "broad" or a "Best Bet." The same applies to other object types like the "Person" object type. Rather than having the user indicate whether a relational query to a Person Agent should return "authors," "experts," or "annotators," the browser can issue a broad query and than qualify the results (with help from the Agency) with whether each returned "Person" object is an "author," "expert," or "annotator," for the cunent context. g. Context Palettes
Context Palettes are a very powerful feature of the present invention that involves invoking Context Templates dynamically for the cunently selected object within the semantic browser. Essentially, Context Palettes are preferably automatically invoked and displayed when users select any object in the Results Pane. Context Palettes enable users to always have the context for the cunently displayed results at their disposal. In addition, the ■ semantic browser constantly refreshes the palette for the currently selected object, thereby guaranteeing that the context for the object is always up to date. In a prefened embodiment, this is accomplished via a timer that triggers a refresh action or by querying the SQML query processor for the Context Palette for whether there is any new object since the last time the palette was refreshed. In the prefened embodiment, results displayed in Context Palettes are "first-class" infoπnation objects in the same way as the infonnation objects displayed in the main Results Pane. In other words, Context Palette results are preferably used with all of the present invention's semantic tools, e.g., smart copy and paste, Smart Lens, Deep Information, etc. The same preferably is true for results displayed in other context panes anticipated in the present invention.
The present invention preferably includes the following Context Palettes. In the prefened embodiment, users have the option to "scroll" through the different Context Palettes for a selected object. The incoφoration of additional and different Context Palettes is expressly anticipated, and may parallel the addition of Context Templates. "'Headlines" Context Palette. This uses the Headlines Context Template and employs SQML that has the SQML of the Headlines Context Template with an additional link to the cunently selected object, and the default predicate for the object-type combination. In particular, the SQML will be keyed off resources that map to all the favorite Agents or recent Agents in the Semantic Environment. The user configures whether he or she wants Favorite Agents, recent Agents, or both to be used when generating the Context Palette. In addition, the Headlines Context Palette is also configurable to show headlines without any filter for the number of objects to be displayed or the "freshness" time limit. In this case, the palette will allow the user to navigate all the relational results sorted by the publication or post time. "Breaking News" Context Palette. Contains relational results from every Breaking
News Agent in the Semantic Environment using the default predicate of the object-type combination, and linked with the cunently selected object. In addition, results for the default Breaking News Context Palette are displayed. The semantic browser of the present invention will dynamically generate SQML with as many (and identical) resource or link combinations as there are Breaking News Agents, witli additional links that have the default predicate and the resource qualifier of the cunently selected object (a file-path, folder-path, object://URL, etc.). The semantic browser of the present invention invokes the generated SQML query and loads the palette windows with the SRML results. The Breaking News Context Palette preferably contains navigation controls to allow users to navigate the results in the Context Palette. "Conversations" Context Palette. Similar to the Headlines Context Palette except utilizing the Conversations Context Template.
"Newsmakers" Context Palette. Similar to the Headlines Context Palette except utilizing the Newsmakers Context Template.
"Upcoming Events" Context Palette. Similar to the Headlines Context Palette except utilizing the Upcoming Events Context Template.
"Discovery" Context Palette. Similar to the Headlines Context Palette except utilizing the Discovery Context Template.
"History" Context Palette. Similar to the Headlines Context Palette except utilizing the History Context Template. "AH Bets" Context Palette. Similar to the Headlines Context Palette except utilizing the All Bets Context Palette.
"Best Bets" Context Palette. Similar to the Headlines Context Palette except utilizing the Best Bets Context Template.
"Favorites" Context Palette. Similar to the Headlines Context Palette except utilizing the Favorites Context Template. "Classics" Context Palette. Similar to the Headlines Context Palette except utilizing the Classics Context Template.
"Recommendations" Context Palette. Similar to the Headlines Context Palette except utilizing the Recommendations Context Template. "Today" Context Palette. Similar to the Headlines Context Palette except utilizing the Today Context Template.
"Variety" Context Palette. Similar to the Headlines Context Palette except utilizing the Variety Context Template.
"Timeline" Context Palette. This Context Palette preferably contains merged results from the Headlines, Best Bets, History, and Upcoming Events Context Templates. The Timeline Context Palette preferably allows the user to navigate all objects on the semantic timeline based on the cunently selected object. The timeline may contain infoπnation items based on their publish/post time, event items based on their appointment time, etc. Essentially, with the Timeline Context Palette, the user navigates relevant (and perhaps other semantically related) objects using time as the primary axis for infonnation conveyance.
"Guide" Context Palette. The prefened embodiment of the present invention includes a unified Guide Context Palette. This Context Palette combines all Context Palettes. In other words, each window in the Guide Context Palette conesponds to one result from each of the other system Context Palettes. The user interface for the Guide Context Palette allows the user to scroll through the results for each Context Palette in each window or to animate the results using animation techniques, for example, fade-in/fade-out techniques. A prefened use of the Guide Context Palette is to view context for the cunently selected object in a minimal viewing space. In the prefened embodiment, the use has the option of viewing all Context Palettes side-by-side (vertically, horizontally, diagonally, etc.), docked, or in other anangement formats. Context Palette User Interface. The user interface for Context Palettes is preferably configurable based on the layout Skin for the cunently displayed Agent, hi the prefened embodiment, Context Palettes may be docked on the left, right, top or bottom of the Results Pane. Context Palettes may be collapsed in order to minimize intrusion into the viewing area and dynamically re-expanded to full view. Skins may also allow Context Palette windows to be resized to variable sizes or preset, fixed sizes. Alternatively, some Skins may also animate Context Palettes results'.
By way of example, FIGURE 80 illustrates a user interface showing Agent results and conesponding Context Palettes. In the example, several Context Palettes are collapsed and the Context Palettes are skinned (or presented) to be vertically docked on the right side of the display, or Results Pane. h. Intrinsic Alerts
In a prefened embodiment, in addition to the Breaking News Agent, the present invention provides for Intrinsic Alerts. While conceptually similar to Breaking News Agents, Intrinsic Alerts are fundamentally different in operation. In the case of Breaking News Agents, the present invention signals the user as to breaking news notifications after polling each Breaking News Agent specified by the user and querying it to find if there is anything related to the cunent object that is breaking. An Intrinsic Alert does not require the user to specify a Breaking News Agent or otherwise perfonn any action in order to introduce breaking news notification. An Intrinsic Alert is automatically signaled in the user interface, (for all cunently displayed objects) when there is an event that relates to the object at issue in a fundamental, intrinsic way. For example, if the cunent object is a document, the present invention polls the Agency from whence the document came and asks the Agency if there is any recently posted infoπnation on the Agency that relates to the object. If the cunent object is a person, the present invention may poll the Agency and ask if the person recently sent email, recently posted a document, recently annotated a document, recently joined or exited a distribution list, etc. This allows the user to have in-place infonnation within the native context of the object in a time-sensitive manner.
In the prefened embodiment, the default implementation for Intrinsic Alerts will poll only the Agency from whence the object came. This has the advantage of simplifying the user interface; if the user wants to perfonn cross-Agency queries, he or she has the option to drag and drop, copy and paste, etc. in order to invoke relational queries. In alternative embodiments, hitrinsic Alerts will poll multiple Agencies, including Agencies other than from whence the object came, in an effort to locate breaking news notifications.
In an alternative embodiment, the present invention is configurable to maintain infonnation as to whether a user has accessed an object. This may be analogized to how an email server keeps track of what email messages a user has read. In an embodiment in which the Agency supports per-object, per-user server-side state, Intrinsic Alerts are always accurate because the Agency indicates that there is "intrinsic breaking news" only if there is infonnation on the Agency that relates to the object in question that has not been accessed or read by the user. This alternative is preferably accomplished means of an additional filter on the SQML query.
The alternative of a per-object, per-user server-side state required for this embodiment has disadvantages, especially for Agencies that will hold massive amounts of information and will have a huge number of users (e.g., Internet-based Agencies). In this situation, the system does not scale well if state is maintained per object and per user.
In an alternative embodiment where the Agency does not support per-object, per-user server-side state, the Agency 'may be configured with a static freshness time limit for Intrinsic Alerts. For example, the server may be configured with a freshness time limit of thirty minutes, in which case the server would respond in the affirmative if an Intrinsic Alert query is received within thirty minutes of the anival of a new object that relates to the object in the query. In a prefened embodiment, the KIS Agency maintains information on the average infonnation anival rate. This way, a busy server will have a lower freshness time limit than a server that seldom receives new information. This embodiment is not as accurate as if the server kept per-object, per-User State because the average anival rate produces only an approximation of whether an alert should be signaled. This embodiment will still result in reduced infoπnation loss, h the prefened embodiment, the present invention optionally signals Intrinsic Alerts in a non-intrusive manner that suggests their probabilistic nature (i.e., that an alert is only a best guess). . Smart Recommendations '
Smart Recommendations represent semantic queries to the Semantic Network, for infened semantic links, using an object as an Infomiation Object Pivot. For example, the Inference Engine may infer that users would like to attend a certain event, based on events they have attended in the past, the fact that they have been engaged in many email conversations with the presenter of the event, etc. By way of example, in the prefened embodiment, this infoπnation is available in a Smart Recommendations popup context Results Pane such as that shown in FIGURE 81. This is similar to what users see for a given object against the Recommendations Context Template.
In the prefened embodiment, each link is generated by the object Skin or a special recommendations infomiation pane Skin and will link to SQML containing the predicates for the inferred semantic links. 6. PROPERTY BENEFITS OF THE PRESENT INVENTION
The hifomiation Nervous System of the present invention provides proper context, meaning and efficient access to data and information to allow users to acquire actionable knowledge. Many of the advantages of the Information Nervous System over Today's Web and the conceptual Semantic Web are derived from its use of the technology layers shown in FIGURE 82. The various embodiments of the present invention demonstrate the advantages - as they relate to the properties required to produce an integrated and seamless implementation framework and resulting medium for knowledge retrieval, management and delivery, which include Semantics/Meaning; Context-Sensitivity; Time-Sensitivity; Automatic and intelligent Discoverability; Dynamic Linking; User-Controlled Navigation and Browsing; Non-HTML and Local Document Participation in the Network; Flexible Presentation that Smartly Conveys the Semantics of the Infonnation being Displayed; Logic, Inference, and Reasoning; Flexible User-Driven Infomiation Analysis; Flexible Semantic Queries; Read/Write Web; Annotations; "Web of Trust"; Infomiation Packages ("Blenders"); Context Templates; and User-Oriented Infonnation Aggregation. SEMANTICS/MEANING The present invention employs semantic links, ontologies, and other well-defined data models using XML. As a result, an Agency as described above has the power of a semantic Web site in that its information includes semantics. In addition, by providing meaning as an intrinsic part of the XML Web Service, it further provides context-sensitivity, time-sensitivity, etc. associated with the subject matter infoπnation CONTEXT-SENSITIVITY
Intelligent system Agents described above monitor the private context of users and automatically alert users when there is relevant information on an information source (or sources) related to the specific context. B.y way of example, these specific contexts may include the following: • My Documents
• My Web Portal
• My Favorite Web Sites
• My Email
• My Contacts • My Calendar
• My Customers
• My Music My Location
"This" document
"This" Web site/page
"This" email message
"This" contact
"This" event in my calendar
"This" customer
"This" music track, album, or play-list
The present invention provides a context-sensitive user experience via the use of information Agents associated with the server 10 and via the semantic browser 30 and associated XML Web Service. For example, users automatically connect infonnation in "My Documents," "My Email," etc. (from application islands such as the file system, Microsoft Outlook, etc.) to remote infonnation sources that have semantically relevant infomiation. Users have the flexibility to make these connections in real-time via application-level innovations that reside on top of the Semantic Network such as the new query tools described above, for example, drag and drop, Smart Lenses, smart copy and paste, etc. It is also contemplated that such application tools can be used independent of a Semantic Network, for example, integrated into an existing browser of Today's Web.
In, a prefened embodiment, the KIS of the. present invention pulls semantic infonnation. from the Semantic Web or other repository with semantic markup (preferably via RDF plug-ins) into its Semantic Network. Alternatively, the system 10 of the present invention exists without the Semantic Web. In this situation, the KIS builds its own Semantic Network (e.g., a private semantic web) from data sources that the system administrator selects (e.g., email, documents, etc.). The system 10 of the present invention is able to utilize the actual semantic applications with a semantic backend (which can optionally include the Semantic Web). The system 10 thus provides context-sensitivity via integration with client- side applications (including the proprietary semantic browser 30), location-tracking tools, etc. and the proprietary XML Web Service (which the Semantic Web does not describe). More specifically, while the conceptual Semantic Web describes architecture for semantic linking and knowledge representation, it does not address scenarios and innovations using XML Web Services to provide context-sensitivity, time-sensitivity, dynamic linking, Context Templates, Context Palettes, etc. In contrast, the present invention addresses semantic linking via the semantic data model and Semantic Network as well as provides software services for context sensitivity, time-sensitivity, semantic queries, dynamic linking, Context Templates, Context Palettes, etc. via integration with its proprietary XML Web Service. TIME-SENSITIVITY The present invention has an intrinsic notion of time-sensitivity. For example, by providing features related to time-sensitivity such as Breaking News Agents, Breaking News Context Templates, Breaking News Context Palettes and intrinsic alerts, the present invention demonstrates the importance of time as an element in semantics and presentation. While not universally true, generally speaking old information is usually not as relevant as new infomiation. For example, when CNN interrupts news broadcast to show breaking news, the interruption is based on a combination of semantics (the relevance of the breaking news about to be displayed) and the fact that the news is indeed breaking. Except is those rare cases where the Web author specifically builds in time-prioritized analysis, this time-sensitivity element as an axis for alerts and presentation is totally lacking in Today's Web and in the conceptual Semantic Web.
The present invention allows users to select Smart Agents as Breaking News Agents. Any information being displayed will show alerts if there is relevant breaking news on a breaking-news Agent. For example, with the present invention, a user is able to create an Agent as: "All Documents Posted on Reuters today" or "All Events relating to computer technology and holding in Seattle in the next 24 hours" as Breaking News Agents. Because these Agents are personal ("breaking" is subjective and depends on the user), the browser provides uniquely individual support. In yet another example, a user in Seattle would be able to schedule notification on events in Seattle in the next 24 hours, events on the West Coast in the next week (during which time he or she can find an inexpensive flight), events in the United States in the next fourteen days (the advance notice for most U.S. air caniers to obtain a competitively priced cross-continental flight), events in Europe in the next month (likely because he or she needs that amount of time to get a hotel reservation), and events anywhere in the world in the next six months.
The present invention further supports a Breaking News Context Template based on which users can create Breaking News Agents. In addition, the present invention supports a Breaking News Context Palette that allows users to view all displayed results in the context of a template-based definition of "breaking news," thereby seamlessly and intelligently integrating context and time-sensitivity.
The present invention further provides a powerful personal historian tool for performing historical analyses. Using browse history, past events, and document creation times, the system 10 can compensate for faulty memory by recalling details from an event, for example, showing results to the query "The coworkers who attended the design meeting from 6/1/98 through 6/1/99". Alternatively, the system may seek for a cluster of events. For example, investigators may ask for "All stock market transactions greater than $10M related to the airline stocks from 7/1/01 up to 9/11/01" or "Show all documents created within a ten day window of this event".
AUTOMATIC AND INTELLIGENT DISCOVERABILITY
The system 10 of the present invention has an intrinsic notion of discovery. In a prefened embodiment, the KIS automatically announces its presence on a local multicast network, an enteφrise directory (e.g., an LDAP directory or the Windows 2000 Active Directory), a peer-to-peer system or other system. Ideally, the semantic browser 30 periodically listens for multicast or peer-to-peer announcements and checks an enteφrise directory or a Global Agency Directory. The browser also allows the user to navigate the system in a hierarchical fashion to locate additional Agencies. This way, users are notified when new Agencies are available and when existing Agencies expire. The semantic browser of the present invention preferably notifies users instantly when new Agencies are available via namespace snapshots and periodic checks for announcements and directory presence.
The peer-to-peer aspect allows the system 10 to scale and automatically populate the enteφrise directory without any centralized maintenance (which is a large ongoing^cost for organizations). The system preferably uses programmatic queries for new classes of servers, thereby eliminating the needs for Web logs. DYNAMIC LINKING
The present system 10 provides fundamental advantages over Today's Web and the conceptual Semantic Web by employing smart objects having intrinsic behavior. The system embeds behavioral characteristics in each Agency's XML Web Service, thereby make each node in the Semantic Network much smarter than a regular link or node on Today's Web or the Semantic Web. hi other words, in the prefened embodiment, each node in the Semantic
Network of the present invention links to other nodes independent of authoring. Each node has behavior that dynamically links to Agencies. Smart Agents also allow for such additional features as drag and drop and smart copy and paste, creating links to Agencies in the
Semantic Environment, responding to lens requests from Smart Agents to create new links, including intrinsic alerts that will dynamically create links to time-sensitive information on its Agency, including presentation hints for breaking news (wherein the node can automatically link to breaking news Agents in the namespace), etc.. These features dramatically increase the user's ability to, for example, find and navigate new links. Once the user reaches a node in the network, the user has many semantic means of navigating dynamically and automatically using context, time, relatedness to smart Agencies and Agents, etc. By making each node in the network smarter, the entire Semantic Network becomes a smart, virtual, self-healing and self-authoring network.
The dynamic linking technology of the present invention allows users to issue queries across local/remote infonnation boundaries. For example, the present invention (preferably using SQML technology) allows a user to issue a query like: "Find me all email messages written by my boss or anyone in research and which relate to this specification on my hard disk." The client-side query processing teclinology (preferably via SQML) allows this flexible query because the processor links the metadata from the client with the remote XML Web Service that processes the relational query. Smart and Dynamic Information Propagation. Dynamic linking as provided for in the present invention provide for intelligent infomiation propagation. Because the Semantic Network can be navigated from many more axes than Today's Web or the Semantic Web, information sharing and propagation becomes much more efficient and infomiation loss is minimized. USER-CONTROLLED NAVIGATION AND BROWSING
The dynamic linking property of the present invention allows for continuous semantic browsing as opposed to with Today's Web and the Semantic Web, where static links result in browsing "dead-ends." With Today's Web and the Semantic Web, the user typically browses to the desired location or effectively reaches an impasse where no further links are available. With dynamic linking, the user can, depending on the nature of the infomiation space at that point in time, continue browsing indefinitely since the node itself includes intelligence to dynamically update links.
For example, via the seamless integration of linking and semantic XML Web Services provided for by the present invention, users drag and drop files, links, etc. to Smart Agents to create new Smart Agents. Preferably this occurs recursively. Smart Agents, in turn, can, where appropriate, be made Breaking News Agents. Other nodes in the presentation display presentation hints indicating whether there is breaking news on any Breaking News Agent. To continue the example, the results of the Breaking News Agent query can be used as a Smart Lens, which shows further results. These results preferably include intrinsic alerts that provide the user with a context and time-sensitive path through the network. Subsequent results can be copied and pasted to any Agency, as well as dragged and dropped on other Smart Agents.
In the prefened embodiment, the dynamic linking of the present invention is applied bόth..to objects within the semantic "sandbox" (objects that are in the system 10 environment and displayed within the semantic browser 30) as well as to external objects that can be dynamically added to the environment. This provides a seamless, dynamic migration path from existing documents (on the file system, Today's Web, or other environments) to the system 10 of the present invention.
FIGURE 83 illustrates dynamic linking and user-controlled navigation and browsing according to a prefened embodiment of the present invention. Note that for puφoses of this example, "Smart Links" refer to the dynamic, programmable semantic link of the present invention. NON-HTML AND LOCAL DOCUMENT PARTICIPATION IN THE NETWORK
The present invention does not require that documents be encoded as RDF or XML before inclusion in the network. Rather, the KIS (or Agency server) automatically extracts metadata from all sorts of documents and adds them to the Semantic Network. In addition, client-side dynamic linking, preferably via such features as drag and drop, smart copy and paste and Smart Lens, ensures that local documents of all types are linked to the network, thereby increasing the value and scope of the network. The present invention automatically extracts metadata from local documents and calls the KIS (via its XML Web Service) to retrieve semantically related infomiation. Thus, the local document is not excluded from the network. The present invention empowers a user to drag and drop a document from a dumb environment (e.g., Today's Web or file system) into the system 10, thereby providing it semantic intelligence., Once the metadata is in the system 10, semantic tools such as semantic lenses, smart copy and paste, etc. niay be performed to and with the object. Drag and drop is also supported directly from the user's file system and Today's Web into the system 10. FLEXIBLE PRESENTATION THAT SMARTLY CONVEYS THE SEMANTICS OF THE INFORMATION BEING DISPLAYED
The present invention empowers users with flexible presentation. Because the XML
Web Service sends back XML, rather than HTML, and because the presentation is dynamically generated on the client, the user selects different "skins" with which to view semantic information. Skins preferably convert XML to a fomiat suitable for presentation (e.g., XHTML+TIME, SVG, etc.), allowing the user to dynamically select Skins based op the capability of various display technologies. For example, SVG has many features that XHTML+TIME does not, and vice-versa. The user is able to select an SVG Skin for scenarios in which SVG is optimized. Alternatively, the user is able to select XHTML+TIME for other scenarios.
The flexibility of Skins as part of the present invention provide for application in additional situations. In various alternative embodiments, the use is empowered by text-to-speech Skins that may be running the semantic browser 30 on a second machine concunently with a first or main machine, for example to assist blind users; dynamically resizable Skins that adapt to the size of the cunent view-port (thereby allowing the user to resize the window and yet retain a pleasant user experience); Skins that check local state to display semantic hints (e.g., the user's calendar in the case of event infomiation, e.g., free/busy information); Skins that display inline preview windows that save user navigation time and increase productivity; Skins that display different customizable hints for intrinsic alerts, breaking news, deep infonnation, smart recommendations, intrinsic links, lens info, etc. Users are also allowed to select Skins to be used with smart screensavers, for example where users desire to view an Agent in screensaver mode. In an alternative embodiment, the system 10 supports Skins for Context Templates (described above), e.g., Headlines, Newsmakers, Conversations, etc.
By virtue of allowing for flexible presentation, the present invention allows the user to select the best presentation mode based on the cunent task. For example, users can select a subtle Skin when working on their main machine where productivity is a higher priority than aesthetic effect. Users can select a moderate Skin in cases where productivity is important but where effects are desired or allowed. Users can select an exciting Skin for scenarios like wherein secondary machines are utilized — or example, where users are viewing information in their peripheral vision and desires features such as text-to-speech to alert them of breaking news, etc. Exciting Skins may alternatively feature animations, storyboard like effects for deep information, objects displayed on motion paths, and other special effects.
In addition, Skins according to the present invention are optionally configured with include and exclude object type filters. For example, a Skin may be configured to include only "documents" but exclude "analyst reports." Because the Skin takes XML results to determine the ultimate presentation, the Skin can include or exclude objects in the XML (SRML) results based on an examination of the object type (or other attributes) of the returned objects. LOGIC, INFERENCE AND REASONING The present invention provides for logic, inference, and reasoning. The semantic data model on KIS Agency preferably offers support for logic via database processing of the Semantic Network, conversion of semantic queries to SQL and other database query languages for logic processing, etc. In addition, the system 10 of the present invention preferably includes an Inference Engine for inferring links such as the experts on a particular category or information item, recommendations, probabilistic links (e.g., the probability that a person wrote a document), etc. As described above, an Inference Engine according to the present invention preferably observes the Semantic Network, mines it to infer new semantic links and represents resulting links in the SemanticLinks table.
FLEXIBLE USER-DRIVEN INFORMATION ANALYSIS
The present invention provides native support for flexible infoπnation analysis on the client. The Presenter of the present invention preferably utilizes Smart Lenses to allow a user to preview the results of a semantic query prior to issuing the query. The user is able to
• change relevant predicates and other filters in order to preview the results. In an alternative embodiment, the user has the option of invoking the query and using that as the basis of a new sub-query, if desired. FLEXIBLE SEMANTIC QUERIES
The present invention allows a user to issue very flexible semantic queries. The user is able to incoφorate local context into queries, e.g., by using filters such as "relates to this document on my hard drive." Neither Today's Web nor the Semantic Web allow for this. In addition, the present invention preferably incoφorates Smart Agents, which utilize references to a proprietary semantic query language (SQML) and includes local and remote resources, predicates, category references and objects. The present invention preferably incoφorates the easy to use user interface for creating and editing Smart Agents (representing semantic queries) using a simple wizard model. As discussed above, the system 10 allows semantic queries, to form the basis of new queries via the recursive drag and drop feature, e.g., a document , or an HTML link can be dragged to an existing or new Smart Agents, thereby creating successive new Smart Agents. Smart Agents are alternatively used" as lenses, can have objects pasted onto them to form new semantic queries, and can be added to Blenders, which in themselves are semantic query containers and which, in turn, can be filtered thereby creating sub-Blenders or containers of sub-Agents. READ/WRITE SUPPORT
The system 10 of the present invention offers support for read/write functionality by providing an XML Web Service that allows a user to publish information directly into the Semantic Network. This could be any document, an annotation, or a semantic link that conects a broken link or provides a new link. This is all subject to security restrictions at the XML Web Service and operating system layer. The system 10 employs authentication, access control, and other services from the operating system and application server that sit underneath the XML Web Service layer. These security services are preferably used to secure read and write access to the Semantic Network. ANNOTATIONS
The present invention includes built-in support for Annotations. There is a special predicate "Annotated By" that defines an Annotation semantic link between a person object and any other infoπnation object (e.g., a document, email posting, online course, etc.). The system 10 includes presentation-layer support for Annotations by allowing users to navigate to Annotations via intrinsic links, Smart Lenses, etc. The manner in which the present invention incoφorates Annotations provides advantages of existing techniques (such as in-place Annotation techniques that embed the Annotation as part of the infoπnation object it annotates). In the prefened embodiment of the present invention, Annotations are "first-class" infomiation objects. This means that they can be linked to and from, "lens" over (using Smart Lens), copied and pasted (using smart copy and paste), etc. The present invention exposes Annotations to all of the semantic tools of the present invention, thereby facilitating a user experience more powerful than capable with standard Annotation techniques. In addition, Annotations of the present invention are used with Context Templates. As a result, the Inference Engine is able to employ them to make the system smarter over time. In addition, the system 10 provides a unique and easy means of annotating objects by sending specially formatted email (with a qualified message body) to the email Agent of an Agency.
"WEB OF TRUST"
The present invention provides a "Web of Trast" via the XML Web Service. This service authenticates a user that wants to update the Semantic Network, make assertions, fix/update links, etc. This also allows rich content to be made available via the KIS Agency to registered subscribers for pay-per-view content. The value of the entire network increases when one can utilize the same platfomi tools to navigate seamlessly across many rich content sources. INFORMATION PACKAGES (BLENDERS)
The present invention provides for information packages or "Blenders." Blenders are semantic containers that include references to semantic queries from Smart Agents. This allows a user to deal with related semantic infonnation as a whole unit. The user is able to separately view the individual Agents within the Blenders or view the entire Blender as though the infonnation therein was from one aggregate Agent. This is preferably accomplished by driving each Agent via calls to the XML Web Service, hi the prefened embodiment, users drag and drop objects onto Blenders to create sub-Blenders. This is preferably accomplished recursively. Blenders can be created, deleted, and edited. The user is able to add and remove smart Agents to or from Blenders. Blenders can be thought of as a digital equivalent of a personal- newspaper that contains different sections. For example, the USA Today, New York Times, Wall Street Journal, etc. contain different sections such as News, Business, Sports, Life/Entertainment, etc. Each of these sections conesponds to a Smart Agent entry in a Blender and the entire newspaper conesponds to the Blender. The, flexible viewing and navigation provided by the present invention can be thought of as the digital equivalent of the user being able to browse each newspaper section completely and sequentially, one at a time, or browse the entire newspaper by starting as page one of each section, followed by page two of each section, etc.
CONTEXT TEMPLATES
As described in detail above, the present invention provides Context Templates, which are scenario-driven information query templates that map to specific semantic models for infoπnation access and retrieval. Essentially, Context Templates can be thought of as personal, digital semantic information retrieval "channels" that deliver infoπnation to a user by employing a predefined semantic template. In the prefened embodiment, the semantic browser 30 allows the user to create a new Blender or Special Agent using Context Templates to initialize the properties of the Agent. Context Templates preferably aggregate information across one or more Agencies. In addition, Context Templates are preferably used with Context Palettes to provide intelligent, dynamic, in-place context for any infoπnation object that is displayed or selected by the user. USER-ORIENTED INFORMATION AGGREGATION The present invention has intrinsic support for user-oriented infomiation aggregation.
Scenarios empower a user to' view context and time-sensitive infoπnation as though they came from one source even if they cut across infomiation repositories. This provides a significantly more productive user experience that with Today's Web and the conceptual Semantic Web by providing user-oriented computing wherein the user is presented with the right infoπnation in the right context and at the right time, regardless of the source of the infoπnation. The Infoπnation Agent aggregates information dynamically, across information sources, using client-side semantic queries via SQML and aggregating the XML results that come from different Agencies' response to SQML. E. SCENARIOS The following provides exemplar scenarios of the operation of prefened and alternative embodiments of the present invention as applied in different pragmatic situations. 1. EXAMPLES OF SEMANTIC QUERIES UTILIZING THE PRESENT INVENTION a. Find all context that relate to the specification on the file path c:\spec.doc
Drag and drop the icon representing a document to the icon representing the Infomiation Agent. The file is opened in the semantic browser and the Context Palettes are displayed. In the prefened embodiment, these include some or all of the following Context
Templates: Headlines, Discovery, Newsmakers, Upcoming Events, Timeline, Conversations,
Variety, Classics, Best Bets, Today, Breaking News, etc. These palettes include relevant context from Agencies in the "recent" and "favorite" lists in the namespace. • b. Find all experts on the Agency titled "R&D " that have expertise on wireless technology
Start the "New Smart Agent" wizard and select the "Use Context Template" option when creating the Agent. Select the "R&D" Agency from the "Select Agency" dialog and select the category called "wireless" from the category browser. Open the newly created Smart Agent. c. Find all information on Reuters that is relevant to a link on the
Currently viewed Web page
Drag and drop the link to the Agency icon representing "Reuters." A new Smart
Agent is created titled "Infomiation on Reuters relevant to [link title]" and opened in the Infonnation Agent.
• d. Find all information on Reuters that is relevant to a link on the current Web page and which is relevant to the specification on the file path c:\spec.doc
Drag and drop the icon representing the document to the Agent that was just created above ("All infomiation on Reuters relevant to [link title]"). This creates a new Smart Agent titled "Infomiation on Reuters relevant to [link title] and relevant to spec. doc." This illustrates user-controlled browsing and dynamic linking. e. Find all email on the internal Agency titled "Marketing" relevant to thefirst article on Reuters that was returned in the previous query
Highlight the Reuters article object and click on the button for "Verbs." This displays
, a popup menu. Select "Copy." Find the icon representing the Agency titled "Marketing" (on the Shell Extension Tree View). Right-click the icon. Hit "Paste." This creates and opens a new Smart Agent titled "Infomiation on 'Marketing' relevant to [Reuter's article title]."
Focus on the frame in the results window showing email objects. f. Navigate to the author of the email -
- .. Highlight the email object and click on the button for "Links." This displays a popup menu showing the intrinsic links. Navigate to the menu item titled "From:" This displays a popup menu showing the person object on the "from" line of the email, object. Select the desired object. This opens a new Smart Agent in the Infomiation Agent showing the metadata of the person that authored the email object. The context of the person is also displayed in the Context Palettes. Users are able to continue browsing using the person object or its context (on any of the Context Palettes). g. Navigate to the attachments in the email
Highlight the email object and click on the button for "Links." This displays a popup menu showing the intrinsic links of the email object. Navigate to the menu item titled
"Attachments." This displays a popup menu showing the titles of the attachments. Select the desired attacliment. This opens the attachment as a new Smart Agent in the Infomiation
Agent window. The context for the attachment is displayed in the Context Palettes. h. Find all events on the "Energy Industry Events" Agency that are relevant to the attachment
Highlight the attachment object and click on the button for "Verbs." This displays a popup menu. Select "Copy." Find the icon representing the Agency titled "Energy Industry
Events" (on the Shell Extension Tree View). Right-click the icon. Hit "Paste." This creates and opens a new Smart Agent titled "Information on Energy Industry Events relevant to [email attacliment title]." . Browse the "My Documents "folder using Reuters as a context
In the Infomiation Agent, select "Open Documents in Folder." Alternatively, drag and drop the "My Documents" folder to the icon representing the Information Agent. Indicate whether sub-folders are to be included. This creates and opens a new Dumb Agent titled "My Documents." When you click this Agent, the metadata for the documents in this folder are opened in the Information Agent. When one of the documents is selected, the Context Palettes for the document are displayed. To browse the documents using Reuters as a context, the user finds the icon representing the Reuters Agency, right-clicks on the icon and hits "Copy." The user hovers over any of the results showing the documents metadata in the Infonnation Agent and selects the icon indicating the Smart Lens. A Smart Lens window is displayed showing information on the results of the relational query. The number of items found on Reuters that are relevant to the document is displayed, in addition to infomiation such as the most recently posted item. In addition, a preview control is displayed to allow the user to preview the results in place. The user is able to choose to click on the results to open an Agent representing the new, relational query. If done, the context for the first object in the results is displayed using the Context Palettes. , j. Notify by email, voice or pager when there is Breaking News that relates to anything on XML technology and which relates to this document
Create a new Smart Agent using the "Breaking News" context and using the "XML"' category as a category filter. Drag and drop the icon representing this document to the Agent.
This creates a new Smart Agent with an appropriate title. Go to the "Options" menu in the Information Agent and enter the proper information in the notification section (your email address, pager number, telephone number, etc.). Right-click the Smart Agent and select
"Notify." 2. B us NESs PROBLEMS a. Information Access Today's Web. John Head-Master works at FastServe, a marketing consulting services company in San Diego. Everyday, he conies in to work and fires up his Web browser. On this day, he decides to browse the coφorate Web to see if he can discover new and interesting information. The browser home page is set (using an Enteφrise Infonnation Portal) to the coφorate home page. The coφorate home page has links for the home pages for different divisions within the company. John navigates to these links and from there, keeps clicking links. After a while, he gets frustrated because he knows that there.are more sources of information that he cannot navigate to, only because he does not know what paths to take. Eventually, he gives up.
Information Nervous System. John fires up his Infomiation Agent (semantic browser). This opens the home Agent. On the page, he sees a list of knowledge links conesponding to products, product groups, reports, coφorate events, online courses, and video presentations. He hovers over the "product groups" link. Automatically, a balloon popup appears indicating the number of product groups and other data about the link. He then opens the link. A list of product group objects is then displayed with a customizable look or "skin." He then hovers his mouse over the first one. A popup menu immediately appears over the link with the actions: "Show Members," "List Similar Product Groups," and "Subscribe to Group Events." He then clicks on "Subscribe to Group Events" and he will now be notified by email (via the Enteφrise Information Agent) about all events that relate to this product group. He then clicks "Show Members." This then opens a new "Knowledge Page" with icons conesponding to people. He then hovers over the icon for Susan Group-Leader. A balloon pop-up then appears showing infomiation on Susan. A right-click menu then appears with the actions, "Reports To," "List Direct Reports," "Member Of," "Authored Documents," and "Recently Attended Meetings." John then selects "Recently Attended Meetings." This opens up a new knowledge page with one meeting object.' John then hovers over this and continues browsing.
At some point, John decides to search for a co-worker he met the previous day. He then types in "Wilbur Jones." This then returns a person object conesponding to Wilbur. John then continues to browse using Wilbur as an Infomiation Knowledge Pivot.
Eventually, John realizes that Wilbur does not seem to have the information he (John) needs. John then types the following query into the search box on his Information Agent:
"List all online courses and documents that relate to the upcoming 2002 sales meeting." The
Infonnation Agent (via the Email Agent) then returns a list of actionable online courses and documents that conform to the knowledge query. b. Knowledge-Driven Customer Relationship Management Customer Touch-Points. AnySoft is a software manufacturer with 50 products in 100 different languages. They employ their web-site (anysoft.com) to provide up-to-date infomiation to their customers. However, customers have complained that their Web site is very hard to navigate and that they find it very hard to find infoπnation on products and to subscribe for notifications.
By deploying an Information Nervous System based on an embodiment of the present invention, AnySoft has deployed an Infoπnation Nervous System that co-exists with their existing Web site. The Information Agent is accessible from the home page and from the search bar. Customers now have a much more intuitive way of navigating the Web site for products, relevant white papers, announcements, press releases, coφorate events, etc. Customers can now issue natural language queries that return self-navigable and actionable knowledge objects. This feature alone gives customers access to knowledge at their fingertips. Customers can also now use natural language to navigate the AnySoft.com Web site from their handheld devices. Customer Feedback and Tracking. Comp-Mart is a reseller of computer peripherals with multiple distribution channels. The Company gets customer feedback from its Web site, its call center, its direct sales force, its telemarketing agents, etc. The feedback comes in as documents and email. The Company has identified a problem wherein customer feedback does not get properly routed around the Company to the people that need the infomiation. Employees in product development have complained to management that they find it hard to integrate customer feedback into the product development, process because they don't know where to find the information and because critical knowledge is not shared within the organization. ,. With an Infonnation Nervous System in place, email that contains customer feedback now gets semantically integrated into the Company's Semantic Enviromnent. The KIS of the present invention automatically adds semantic links between customer feedback email and" semantic objects like documents, projects, and employees that work on the germane products. Customer feedback intelligently bubbles up in the right places in the knowledge space. The Email Agent sends out periodic notifications to people that are likely be interested in reading customer feedback email.
Also, with the Information Nervous System, the customer becomes an Infomiation Knowledge Pivot. This makes it much quicker and easier to act on customer feedback and to track customer-related knowledge across the organization. The Infomiation Nervous System automatically annotates the customer object with relevant email messages, documents, similar customers, etc. This way, links to the customer can be forwarded via email and co-workers can navigate relevant information from there. The customer object can be searched for, can be browsed, etc. c. Knowledge-Driven Direct-Sales/Field-Senήce Marsha Mindset is a customer service agent for JustlnTime Support Services, a computer service firm in Kansas City, Missouri. Marsha visits customers around the Kansas City metro area, and" always takes her wireless PDA so she can send eniail to the support headquarters anytime she is in difficulty. JustlnTime recently deployed the KIS and the Email Agent. Now, whenever she has support questions, Marsha can now email the Email Agent and ask it questions in natural language. The Email Agent replies to her email with direct answers or with "knowledge links" that allows Marsha to instantly access relevant support email, documents, or people that she could then email or call up on the phone. The JustlnTime Direct Sales force also uses the teclinology of the present invention when in the field selling solutions to customers. The sales representatives also cany wireless PDAs and can issue requests to the Email Agent. d. Case Studies
Corporate Training,, Knowledge Transfer, and Sharing. WaveGen is a biotech company providing "managed care" solutions to doctors around the United States. The company recently deployed the Saba Learning Management System platform for training its employees (especially its sales reps). This reduces travel costs and enables the Company's sales-force to be better prepared to serve physicians in different healthcare regions in the country. It also assists the Company's researchers to be regularly informed of recent discoveries in the biotech research community.
The Company also has other software assets in place that hold valuable sources of knowledge. It has deployed content management solutions that host documents and media files, Microsoft Exchange for email, and collaboration software for online conferences. - However, the Company has noticed that knowledge transfer is not very effective because it is not integrated across all these solutions. Sales representatives have indicated that they do not have the tools to discover important sources of knowledge within and outside the organization to assist them in pitching the Company's products to doctors. Enteφrise Infoπnation Portals are cunently used to inform the sales force of upcoming online courses and of important events. However, the sales reps complain that a lot of knowledge (stored in email, documents, etc.) is not brought to their attention because no one knows who else might need them.
In addition, the sales representatives use Microsoft Outlook to add appointments to their calendars for upcoming doctor visits. However, they complain that they only get reminders for the appointments, and that a lot of infoπnation that could help them sell products more effectively is not made available to them automatically, ahead of their doctors' appointments.
WaveGen recently deployed an Information Agent based on teclinology from the present invention. The company deployed the KIS and the Email Agent to facilitate intelligent infomiation connections and routing to help their sales and research teams make better decisions to serve customers and improve the Company's products. Using the Information Agent, the sales force has instant access not just to documents but to "knowledge objects" that are more directly tied to their task at hand. For instance, the sales representatives now have an Agent with "Doctor Jones" as an XML object. This is not a document or a Web page. Rather, it is a semantic representation of the customer. A sales representative can then see semantic links like "Recent Email Messages", "Relevant Documents," "Properties," "Important Dates," "Relevant upcoming online courses," etc. This way, the customer becomes the pivot with which the sales agent is navigating the internal Web. These links might generate results from file-shares, Email stores, Microsoft Exchange, etc. But rather than searching or navigating for these knowledge sources as islands, the sales representative can discover new knowledge based on semantic relationships as they relate to the sales representative's task.
This way, the sales representative can have much more powerful knowledge at the sales representative's fingertips, thereby enabling much better customer service. And this knowledge emanates from co-workers, documents that were published by other sales agents, email sent on distribution lists that might not be known to exist, etc. The KIS does the smart thing by automatically making semantic connections from all these disparate sources. The sales representative can then email this "page" to a co-worker. This then becomes a very powerful form of knowledge sharing because the co-worker can then navigate the Infonnation Agent using the same "Dr. Jones" pivot. The Email Agent also allows the sales representative to issue knowledge queries via natural language. The query results are derived from the Inference Engine and could be based on knowledge that was deduced from existing knowledge. A powerful feature of the Infomiation Nervous System of the present is that knowledge transfer, sharing, discovery ail happen automatically based on the Semantic Network. 3. SITUATIONS a. Semantic Information Discovery, Retrieval, and Navigation
Joe Knowledge-Worker starts the Infomiation Agent (the XML-based semantic browser of the present invention). When he logs in, he is prompted with a dialog box indicating that there are new Agents available on the semantic intranet. He then sees a list of Agents from within and outside the organization that may include the following:
Documents.Technology.All
Documents.Marketing.All
People.Divisions.Sales.All
People.Division.Sales.Managers • OnlineCourses.Sales.101
OnlineCourses.Technology.XML.101
Meetings.ThisWeek.All
Meetings.LastWeek.All
Books. Computers.Programming. All • Newsgroups.Microsoft.Public.Soap
Email.Mine.All
Email.Mine.ProjecfX.All
Events.Technology.Wireless.All
Reports. Gartner.Software.All • Reports.IDC.All
• Videos.ExecutivePresentations.All
He then selects Meetings. ThisWeek.All. The Information Agent then displays a list of objects that represents meetings that he attended this week. This information comes from Microsoft Exchange but this is not exposed to him. Joe then hovers over a link for the first meeting object. A balloon pop-up is then displayed indicating that a new training course was just made available on the intranet. The balloon also indicates that there is a new report on IDC that might be relevant to Joe. In addition to the balloon, a pop-up menu is displayed to the right of the object. This menu has the following verbs: • List participants
• List possible replacement participants o Show Related Objects ->
• On News.Reuters.MarketForecasts.All
• On Documents.Technology.All • On Events. Coφorate.Today.All
• Subscribe for follow-up
Joe then selects "Subscribe for follow-up." This contacts the Meeting Follow-up Agent on the server. This Agent then sends periodic updates of relevant infoπnation to the participants of the meeting. This could be done either through the browser or through email. Joe then selects related objects on Events. Coφorate.Today.All. This then displays a list of event infoπnation objects. Joe then hovers over the first object and a pop-up menu gets displayed. Joe then selects "Add to calendar" and the event is added to his calendar. Joe then decides that he wants to find all industry events that relate to the coφorate event. He then drags the object to the Agent Events.Technology.All and releases his mouse. When the mouse is released, the browser then loads infomiation objects from Events.Technology.All (across web-sites and other islands) and which are related to the coφorate event the object of which he dragged. The next week, Joe gets email from the Email Agent. In the email, the Agent informs Joe that it has noticed that everyone that added the event to his or her calendar also watched a coφorate training video from the coφorate media server. The email contains an XML link, which takes Joe back into the Information Agent. The browser then displays the metadata for the video. One of the items on the pop-up is "Watch Video." Joe then selects it and watches the video.
The next time Joe logs in to his workstation, he notices that there are new Agents. He then subscribes to Books. Eb ay. Computers. All and adds it to his My Agent list. Automatically, an embodiment of the present invention adds this Agent into Joe's Semantic Enviromnent. The Information Agent perfonns implicit queries and provides recommendations (ranked by relevance and time-sensitivity) that include this Agent. He then clicks on this Agent and semantic infonnation objects (representing books) are displayed in the Results Pane. When he hovers over one of the objects, a pop-up balloon is immediately displayed, alerting him to the fact that there is a related industry conference being hosted by the author of the book. When he clicks the pop-up link, the event object is loaded in the browser, complete with verbs that allow him to add the event to his calendar (either Microsoft Outlook or an Internet-based calendar like the MSN Calendar (accessible via Microsoft's Hailstorm Web services), AOL Calendar, etc.)
•Explanation of the Scenario. This scenario shows how with the present invention, knowledge-workers are able to obtain access to "federated knowledge." In this example, Joe's company has "imported" knowledge Agents from Gartner, IDC, Reuters, Ebay, etc. into its knowledge space. As such, these Agents automatically add knowledge into the company's Semantic Network. The scenario also showed how Joe was able to get an "object model" view of the entire organization's knowledge-space via intuitively named Smart Agents. Joe was able to use these Agents to "enter" the Semantic Environment, and then navigate his way from there. All the infonnation objects were delivered in real-time and were actionable (with relevant verbs that were displayed in place). This way, Joe did not have to care about what infonnation islands the objects were coming from, or what applications generated them.
The scenario also shows how Joe was able to discover not just new infonnation but also new Agents. And the scenario shows knowledge collaboration in action — via collaborative filtering — wherein the Infonnation Agent gave recommendations to Joe based on what it noticed others in the enteφrise were doing.
Lastly, the scenario illustrates how time-sensitive infomiation is automatically brought to the user's attention at the point of context where it makes sense. The Email Agent automatically connected the book from Ebay with the upcoming industry event, infened and assigned a relevance and time-sensitivity ranking to the event, and decided that the event was critical enough to wanant displaying the infonnation immediately via an alert in the semantic browser. b. Peer-to-Peer Knowledge Sharing and Capture Nancy Hard-worker works at a Fortune 500 company with 40,000 employees, She subscribes to a variety of Web sites and has information forwarded to her by email from friends and co-workers. She just got a bunch of documents from someone at a partner company and she would like to share the information within the organization. She sends the documents to 'all the distribution lists of which she is a member. The Enteφrise Information Agent is a member of these lists also (the Agent adds itself to all public distribution lists when the server is installed). When the Agent receives the infomiation, it classifies it and adds it to the Semantic Network. The Inference Engine then picks up the information.
Several thousand co-workers are not members of any of the distribution lists to which Nancy forwarded the documents. However, they all use the Integrator and all of them have subscribed to the Email.Public.All Agent. While they browse other related parts of the knowledge-web, a balloon popup gets displayed indicating that there is new and relevant email on the Email.Public.All Agent. The co-workers then open up the Agent and the email object is displayed. One of the menu items on the email item is "Show distribution lists to (Which message was forwarded." The co-workers then select this and the distribution list infonnation objects are then displayed in the browser. The co-worker then hovers over the distribution list and a pop-up menu item gets displayed. The first item is "Show Members." The second is "Join." The co-workers then join the distribution list.
Explanation of the Scenario. This scenario illustrates how infoπnation was published, shared and captured via email and how, by use of the Semantic Network, other co-workers found out about this infoπnation (and about distribution lists the existence of which they were not aware) from different but related 1 "knowledge angles." The scenario shows peer-to-peer knowledge sharing in a way that is completely seamless and does not require users to public information to repositories, or to classify information themselves. With certain embodiments of the present invention, everything just happens automatically (in the background) and the knowledge gets bubbled up in relevant places. While the prefened embodiment as well as alternative embodiments of the invention have been illustrated and described, as noted above, many changes can be made without departing from the spirit and scope of the invention. Accordingly, the scope of the invention is not limited by the disclosure of the prefened or alternative embodiments.
The embodiments of the invention in which an exclusive property or privilege is claimed are defined as follows:
1. A system for knowledge retrieval, management, delivery and presentation, comprising: a first server programmable to add and maintain domain-specific semantic infonnation; a second server in communication with the first server, the second server programmable to host domain-specific infomiation that is used to classify and categorize semantic infomiation; a client providing a user interface for a user to communicate with the first and second servers; and wherein the processors of the first and second servers operate together to perform the steps of: securing infoπnation from information sources; semantically linking the infomiation from the infoπnation sources; maintaining the semantic attributes of the semantically linked information; delivering requested semantic information based upon user queries; and presenting semantic infomiation according to customizable user preferences.
2. The system of claim 1, wherein the first server further comprises structure or methodology directed to providing at least one of the following: a Semantic Network, a Semantic Data Gatherer, a Semantic Network Consistency Checker, an Inference Engine, a Semantic Query Processor, a Natural Language Parser, an Email Knowledge Agent, or a Knowledge Domain Manager. 3. The system of claim 1, wherein: the infonnation from the infoπnation sources consist of objects or events; and the objects or events are active agents semantically related to each other and representing queries that return data objects for presentation according to a predeteπnined theme.
4. The system of claim 3, wherein the predeteπnined these according to which the data objects are presented is customizable by a user.
5. The system of claim 1, wherein the client delivers and presents the semantic information resulting from the user query.
6. A method for knowledge retrieval, management, delivery and presentation for use with a server system programmed to add, maintain and host domain-specific information that is used to classify and categorize semantic information, comprising: securing information from infonnation sources; semantically linking the information from the information sources; maintaining the semantic attributes of the semantically linked infoπnation; delivering requested semantic information based upon user queries; and presenting semantic infomiation according to customizable user preferences.
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL, MANAGEMENT, DELIVERY AND PRESENTATION
ABSTRACT OF THE DISCLOSURE
The present invention is directed to an integrated implementation framework and resulting medium for knowledge retrieval, management, delivery and presentation. The system includes a first server component that is responsible for adding and maintaining domain-specific semantic infonnation and a second server component that hosts semantic and other knowledge for use by the first server component that work together to provide context and time-sensitive semantic infoπnation retrieval services to clients operating a presentation platform via a communication medium. Within the system, all objects or events in a given hierarchy are active Agents semantically related to each other and representing queries (comprised of underlying action code) that return data objects for presentation to the client according to a predetennined and customizable theme or "Skin." This system provides various means for the client to customize and "blend" Agents and the underlying related queries to optimize the presentation of the resulting information.
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL, MANAGEMENT, DELIVERY AND PRESENTATION
INVENTOR Nosa Omoigui APPENDIX
EXEMPLAR CODE SPECIFICATION
SAMPLE A — SRML DOCUMENT
<?xml version="1.0" encoding="utf-8" ?> <srml xmlns- 'http://schemas.nervana.net/snnl" xmlns:n="http://schemas.nervana.net/snnl" xmlns:rdf- 'http://rdfland.org" xmlns:dc="http://dcland.org" dummyarg="hello"> <results content="new"> <queryref guid="12345678-1234-1234-1234-123456789ABC" /> <document> ■ <objectref guid="I2345678-0000-1234-1234-123456789ABC" />
<rdf:Description>
<dc:titl,e>Nsync - A Constraint Based Toolkit for
Multimedia</dc:title> <dc:creator>Brian Bailey</dc:creator> <dc:creator>Joseph A. Konstan</dc:creator>
<dc:description>
<n:abstract>Nsync (pronounced 'in-sync') is a declarative synchronization toolkit, implemented entirely in Tel, designed to ease the complexity of designing innovative, interactive multimedia applications.</n:abstract>
</dc:description> <dc:date>2001-07-02</dc:date> <dc:type>text</dc:type> <dc:foπnat>application/pdf</dc:foπnat> <dc.ianguage>en</dc:language>
<dc:identifier>bailey97nsync.pdf</dc:identifier> </rdf:Description> <size>84957</size> </document> <document>
<objectref guid="12345678-0001-1234-1234-123456789ABC" /> <rdf:Description> <dc:title>A Comparison of Streams and Time Advance As
Paradigms for Multimedia Systems</dc:title> <dc:creator>Roger B. Dannenberg and Dean Rubine</dc:creator> <dc:description>
<n:abstract>A common model for multimedia systems is the stream, an abstraction representing the flow of continuous time-dependent data such as audio samples and video frames.</n:abstract> </dc:description>
<dc:date>2001 -07-0218:48</dc:date> <dc:identifier>dannenberg94comparison.pdf</dc:identifier>
<dc:type>text</dc:type> <dc:format>application/pdf</dc:format> <dc : language>en</dc :language> </rdf:Description> <size>55421</size>
</document> <document> <objectrefguid="12345678-0002-l234-1234-123456789ABC" /> <rdf:Description> <dc:title>Do You Have the Time? Composition and Linking in
Time-based Hypermedia</dc:title> <dc:creator>Lynda Hardman, et al.</dc:creator> <dc:description>
<n:abstract>In this paper we discuss the concept of presentation time - the timing of the individual parts of a presentation and the temporal relations among them.</n:abstract> </dc : descripti on>
<dc:date>2001 -06-2916:57</dc:date> <dc:identifier>hardman99do.pdf</dc:identifιer> <dc : typ e>text</dc : typ e> <dc:type>text</dc:type> <dc:fomιat>application pdf</dc:fonnat> <dc:language>en</dc:language> </rdf:Description>
<size>266873</size> </document> \<document>
<objectref guid="12345678-0003-1234-1234-123456789ABC" /> <rdf:Description>
<dc:title>Ilιterval-based conceptual models</dc:title> <dc:creator>Little and Ghafoor</dc:creator> <dc:date>2001-07-0218:52</dc:date> <dc:identifier>little93intervalbased.pdf</dc:identifier> <dc:type>text</dc:type>
<dc : fom at>application/pdf</dc: format> <dc:language>en</dc:language> </rdf:Description> <size>364367</size> </document>
<document>
<objectref guid=" 12345678-0004- 1234- 1234-123456789 ABC" /> <rdf:Description>
<dc:title>Efficient Generation of Motion Transitions using Spacetime Constraints</dc:title>
<dc:creator>Charles Rose</dc:creator> <dc:description> <n:abstract>This paper describes the application of space time constraints to creating transitions between segments of human body motion.</n:abstract>
</dc:description>
<dc:date>2001 -06-2917:22</dc:date> <dc:identifier>rose96effιcient.pdf</dc:identifιer> <dc:type>text</dc:type> <dc:format>app]ication/pdf</dc:format> <dc:language>en</dc:language> </rdf:Description> <size>1217898</size> </document> <document>
<objectref guid="12345678-0005-1234-1234-123456789ABC" /> <rdf:Description>
<dc:title>MODELING TECHNIQUES FOR HYTIME</dc:title> <dc:creator>Lloyd Rutledge</dc:creator> <dc:description>
<n:abstract>Hypermedia Time-based Stracturing Language (HyTime) defines constructs for representing general hypermedia document concepts.</n:abstract> </dc:description> <dc:date>2001-06-2917:07</dc:date>
<dc:identifier>rutledge95modeling.pdf</dc:identifier> <dc:type>text</dc:type> <dc:foπnat>application/pdf</dc:format> <dc : language>en</dc : language> </rdf.Description5
<size>129404</size> </document> <document>
<objectrefguid="12345678-0006- l234-1234-123456789ABC" /> <rdf:Description>
<dc:title>The Compositional Specifcation of Timed
Systems</dc:title> <dc:creator> Joseph Sifakis</dc:creator> <dc:description> <n:abstract>In this tutorial, we present an overview of existing executable timed formalisms with a global notion of time, by putting emphasis on problems of compositional description.</n:abstract> </dc:description> <dc:date>2001-07-0211 :37</dc:date> <dc:identifier>the-compositional-specification- of.pdf</dc: identified <dc : typ e>text</dc : typ e> <dc:fomιat>application/pdf</dc:foπnat> <dc:language>en</dc:language>
</rdf:Description> <size>132810</size> </document> <document> <objectref guid="12345678-0007-1234-1234-123456789ABC" />
<rdf:Description>
<dc:title>A Simple, Intuitive Hypermedia Synchronization Model and its Realization in the Browser/Java Environment</dc:title> <dc:creator>Jin Yu</dc:creator>
<dc:description>
<n:abstract>Thi's paper presents a simple and intuitive hypermedia synchronization model - the Media Relation Graph (MRG)</n:abstract> </dc:description>
<dc:date>2001-07-0211 :55</dc:date> <dc:identifier>>αι98simple.pdf</dc:identifier> <dc : typ e>text</dc : typ e> <dc:foπnat>application/pdf</dc:format> <dc:language>en</dc:language>
</rdf:Description> <size>288218</size> </document> <document> <objectref guid="12345678-0008-1234-1234-123456789ABC" />
<rdf:Description>
<dc:title>A Continuous Media Player</dc:title> <dc:creator>Lawrence A. Rowe and Brian C. Smith</dc:creator> <dc:description> <n:abstract>The design and implementation of a continuous media player for Unix workstations is described.</n:abstract> </dc:description> <dc:date>2001-06-2917:08</dc:date>
<dc:identifier>a-continuous-media-player.pdf</dc:identifier> <dc:type>text</dc:type> <dc:fonnat>application/pdf</dc:fomιat> <dc:language>en</dc:language> </rdf:Description>
<size>231817</size> ,</document> <document>
<objectrefguid='T2345678-0009-1234-1234-123456789ABC" /> <rdf:Description>
<dc:title>Multimedia Documents with Elastic Time</dc:title> <dc:creator>Michelle Y. Kim</dc:creator> <dc:description> <n:abstract>We present the elastic time model for multimedia documents. </n:abstract>
</dc : description>
<dc:date>2001 -07-0212:07</dc:date> <dc:identifier>pl43-kim.pdf</dc:identifιer> <dc : typ e>text</dc : typ e> <dc:fomιat>application/pdf</dc:format>
<dc : 1 anguage>en</dc : language> </rdf:Description> <size>272532</size> </document> <document>
<objectref guid='T2345678-000A-l 234-1234-123456789 ABC" /> <rdf:Description>
<dc:title>Clock Hierarchies: An Abstraction for Grouping and Controlling Media Streams</dc:title> <dc:creator>Kurt Rothermel</dc:creator> <dc:description>
<n:abstract>hι this paper, we propose a set of powerful abstractions for controlling and synchronizing continuous media streams in distributed . eιιvironments.</n:abstract>
</dc:description>
<dc:date>2001-07-0218:5K/dc:date> <dc:identifier>rothemιel96clock.pdf</dc:identifier> <dc:type>text</dc:type> <dc:foπnat>application/pdf</dc:foπnat>
<dc:language>en</dc:language> </rdf:Description> <size>93052</size> </document> <document>
<objectref guid="12345678-000B-1234-1234-123456789ABC" /> <rdf:Description>
<dc:title>Temporal Relations in Multimedia Objects: WWW Presentation from HyTime Specification</de:title> <dc:creator>Maria da Graca C. Pimentel</dc:creator>
<dc:description> :
<n:abstract>Initially, this paper discusses the use of HyTime for the specification of binary temporal relations. Next, the paper discusses an approach to the automatic. transformation of the HyTime synchronization specifications into elements to be presented in the context of the World Wide Web environment.</n:abstract> </dc:description> <dc:date>2001-06-2916:48</dc:date>
■■• <dc:identifier>temporal-relations-in- multimedia.pdf</dc:identifier> <dc:type>text</dc:type> <dc:format>application/pdf</dc:format> <dc:language>en</dc:language> </rdf:Description> <size>102862</size> </document> <document> <objectref guid="12345678-000C-1234-1234-123456789ABC" />
<rdf:Description>
<dc:title>Event and Action Representation and Composition for multimedia application scenario modelling.</dc:title> <dc:creator>M. Vazirgiannis</dc:creator> <dc : description>
<n:abstract>In this paper we present a model for the representation of multimedia applications based on the scenario concept.</n:abstract> </dc:description> <dc:date>2001-07-0211 :43</dc:date>
<dc:identifier>vazirgiannis96event.pdf</dc:identifier> <dc:type>text</dc:type> <dc:fomιat>application/pdf</dc:format> <dc : language>en</dc : language> </rdf:Description>
<size>387575</size> </document> <document>
<objectref guid="12345678-000D-1234-1234-123456789ABC" /> <rdf:Description>
<dc:title>Representing Time in Multimedia Systems</dc:title> <dc:creator>Thomas Wahl, Kurt Rothermel</dc:creator> <dc:description>
<n:abstract>This paper evaluates and classifies a selection of the most common existing models applying fundamental statements of the time theory and temporal logic.</n:abstract> " </dc:description> <dc:date>2001-06-2916 :49</dc:date> <dc:identifier>wahl93representing.pdf</dc:identifier> <dc:type>text</dc:type> <dc:fonnat>application/pdf</dc:fomιat> <dc:language>en</dc:language> </rdf:Description> <size>67658</size>
</document> <document>
<objectref guid="12345678-000E-1234-1234-123456789ABC" /> <rdf:Description> <dc:title>Nsync - A Toolkit for Building Interactive Multimedia
Presentations</dc:title> <dc:creator>Brian Bailey</dc:creator> <dc:description>
<n:abstract>We have developed a multimedia synchronization toolkit, called Nsync (pronounced 'in- sync'), to address the complicated issues inherent in designing flexible, interactive multimedia presentations. </n:abstract> </dc:description> <dc:date>2001-06-2917:09</dc:date>
<dc:identifier>bailey98nsync.pdf</dc:identifier> <dc:type>text</dc:type> <dc:fomiat>application/pdf</dc:format> <dc:language>en</dc:language> </rdf:Description>
<size>115592</size> </document> <document>
<objectref guid="12345678-000F-1234-1234-123456789ABC" /> <rdf:Description>
<dc:title>Issues in Temporal Representation of Multimedia
Documents</dc:title> <dc:creator>Nabil Layaida</dc:creator> <dc:description> <n:abstract>We present in this position paper a set of relevant issues that may help in the design of multimedia document representation. These are based on our experience in designing and implementing an authoring and browsing tool called MADEUS.</n:abstract>
</dc:description>
<dc:date>2001 -07-021 l:54</dc:date> <dc:ideιιtifier>issues-in-temporal- representation.pdf</dc:identifier> <dc:type>text</dc:type>
<dc:fomιat>application/pdf</dc:fomιat> <dc:language>en</dc:language> </rdf:Description> <size>17348</size> </document>
<document>
<objectref guid="12345678-0010-1234-1234-123456789ABC" /> <rdf:Description>
<dc:title>Spatio - temporal composition in multimedia applications</dc:title>
<dc:creator>Dr. Michael Vazirgiannis</dc:creator> <dc:description> <n:abstract>The motivation for this research work is the lack of a complete declarative way for representative spatio- temporal composition of objects in the cunent multimedia document standards and authoring tools.</n:abstract> </dc:description>
<dc:date>2001-07-0211:43</dc:date> <dc:identifier>spatio-temporal-composition- in.pdf</dc:identifier> <dc : typ e>text</dc : typ e> <dc:fomιat>application pdf</dc:format> <dc.ianguage>en</dc:language> </rdf:Description> <size>89176</size> </document> <document>
<objectrefguid="12345678-0011-1234-1234-123456789ABC" /> <rdf:Description>
<dc:titIe>An Object-Oriented Framework for the Integration of
Interactive Animation Techniques</dc:title> <dc:creator>Robert C. Zeleznik, et al.</dc:creator> <dc:description> <n:abstract>We present an interactive modeling and animation system that facilitates the integration of a variety of simulation and animation . paradigms.</n:abstract> </dc:description> <dc:date>2001-06-2916:30</dc:date>
<dc:identifier>zeleznik91objectoriented.pdf</dc:identifier> <dc:type>text</dc:type> <dc:format>application/pdf</dc:format> <dc:language>en</dc:language> </rdf:Description>
<size>133440</size> </document> <document>
<objectrefguid="12345678-0012-1234-1234-123456789ABC" /> <rdf:Description>
<dc:title>Dealing with Synchronization and Timing Variability in the Playback of Interactive Session Recordings</dc:title> <dc:creator>Nelson R. Manohar</dc:creator> <dc:description> <n:abstract>In this paper, we describe scheduling and synchronization support for a novel multimedia document, refened to as a session object.</n:abstract> </dc:description>
<dc:date>2001-07-0216:30</dc:date> <dc:identifier>p45-manohar.htm</dc:identifιer> <dc:type>text</dc:type> <dc : fomιat>text/htn l</dc: foπnat> <dc:language>en</dc:language> </rdf:Description> <size>99000</size>
</document> <document>
<objectref guid=" 12345678-0013- 1234- 1234- 123456789 ABC" /> <rdf:Description> <dc:title>Multimedia Made Easy Through
DirectAnimation</dc:title> <dc:description> <n:abstract>This paper describes Microsoft
DirectAnimation, the component of the DirectX API family that provides animation and integrated media support for Web pages, CD-ROM titles, and multimedia applications.</n:abstract> </dc:description>
<dc:date>1998-10-0216:30</dc:date> <dc:identifier>DirectAnimation.doc</dc:identifier>
<dc:type>text</dc:type> <dc:format>application/msword</dc:format> <dc:language>en</dc:language> </rdf:Description> <size>152000</size>
</document> <document> <objectrefguid='T2345678-0014-1234-1234-123456789ABC" /> <rdf:Description> <dc:title>Windows Media Technologies</dc:title>
<dc:description>
<n:abstract>This paper gives a technical overview of the streaming media components of the Windows Media Technologies: Windows NT Server NetShow Services, NetShow Theater Server and the Microsoft Windows Media Player.</n:abstract> </dc:description>
<dc:date>1998-07-0216:30</dc:date> <dc:identifier>NetShow3.doc</dc:identifier>
<dc:type>text</dc:type> <dc:format>application/msword</dc:format> <dc : language>en</dc: language> </rdf:Description> <size>631000</size>
</document> <document>
<objectref guid="12345678-0015-1234-1234-123456789ABC" /> <rdf:Description> <dc:title>Integrating Timing into XML Documents</dc:title>
<dc:creator>Patrick Schmitz</dc:creator> <dc:date>2000-07-0216:30</dc:date>
<dc:identifier>IntegratingTimingWithXML.ppt</dc:identifιer> <dc : typ e>text</dc : typ e> <dc:foπnat>powe oint</dc:foπnat>
<dc:language>en</dc:language> </rdf:Description> <size>195000</size> </document> <document>
<objectref guid="12345678-0016- 1234-1234-123456789ABC" /> <rdf:Description> <dc:title>The SMIL 2.0 Timing and Synchronization model</dc:title> <dc:creator>Patrick Schmitz</dc:creator>
<dc:description>
<n:abstract>A powerful, flexible model is needed to unify scheduling, interaction, advanced control for animation, and mntime synchronization management. SMIL 2.0 defines a language and semantic model that addresses these needs.</n:abstract> </dc:description>
<dc:date>2001-07-0216:30</dc:date> <dc:identifier>SMILTimingForTheWeb.doc</dc:identifier>
<dc:type>text</dc:type> <dc:fomiat>application/msword</dc:format> <dc:language>en</dc:language> </rdf:Description> <size>175000</size>
</document> <document>
<objectrefguid="12345678-0017-1234-1234-123456789ABC" /> <rdf:Description> <dc:title>A Unified Model for Representing Timing in XML
Documents</dc:title> <dc:creator>Patrick Schmitz</dc:creator> <dc:description>
<n:abstract>We need to provide a common model for timing so that authors are not forced to lea and remember different models for different document types or different authoring scenaιrios.</n:abstract> </dc:description>
<dc:date>2001-07-0216:30</dc:date>
<dc:identifier>TimingIntegrationPositionPaper.htm</dc:ident ifier> <dc : typ e>text</dc : type> <dc:fomιat>text/html</dc:format> ■ <dc:language>en</dc:language>
</rdf:Description> <size>10000</size> </document> <email> <objectref guid="12345678-0018-1234-1234-123456789ABC" /> <rdf:Description>
<dc:title>Some ideas on the Nervana client</dc:title> <dc:creator>Patrick Schmitz</dc:creator> <dc:date>2002-04-0410:30</dc:date> <dc:type>text</dc:type>
<dc:format>text</dc:fomιat> <dc:language>en</dc:language>
<dc:identifιer>outlook:fooBarWlιoKnowsl23xyz</dc:identifιer> </rdf:Description> <subject>Some ideas on the Nervana client</subject>
<from>Patrick Sclimitz [cogit@ludicrum.org]</from> <to>Nosa Omoigui [nosa@nervana.net]</to> <to>Steven Judkins [stevenj007@hotmail.conι]</to> <cc>Omoigui, Eghosa D [eghosa.d.omoigui@intel.com]</cc> <cc>Jerome Beard [jbeard@picsmart.net]</cc>
<size>434</size> </email> <email>
<objectref guid="12345678-0019-1234-1234-123456789ABC" /> <rdf:Description>
<dc:title>FW: The Next Web (Business Week)</dc:title> <dc:creator>Nosa Omoigui</dc:creator> <dc:description>
<n:abstract>On the other hand, partly because the industry is acutely aware of EDI's problems and limitations, executives are optimistic. "It'll be a chicken-or-egg situation until a killer app comes along— but I'm very confident that that will happen," says W. Daniel Hillis, a supercomputer pioneer who now heads startup Applied Minds Inc.</n:abstract>
</dc:description>
<dc:date>2002-02-2513 :44</dc:date> <dc : typ e>text</dc : typ e> <dc:format>text</dc:format> <dc:language>en</dc:language> <dc:identifier>outlook:fooBarWhoKnowsl23xyz</dc:identifier> </rdf:Description>
<subject>FW: The Next Web (Business Week)</subject> <from>Nosa Omoigui [nosa@nervana.net]</from> <to>Patrick Schmitz [cogit@ludic m.org]</to>
<to>Steven Judkins [steveιιj007@hotιuail.com]</to> <to>Omoigui, Eghosa D [eghosa.d.omoigui@intel.com]</to> <to>EghosaO@aol.com</to> <to>ChereceO@aol.com</to> <to> Jerome Beard [jbeard@picsmart.net]</to>
<cc>Nosa Omoigui [nosa@nervana.net]</cc> <size>434</size> </email> <email> <objectref guid="12345678-001A-1234-1234-123456789ABC" />
<rdf:Description>
<dc:title>Software's Giants Gird for Upheaval
[Fortune]</dc:title> <dc:creator>EghosaO@aol.com</dc:creator> <dc:description>
<n:abstract>A vague new thing called web services is stining up new battles among Microsoft, Oracle, IBM, and all of software's biggest names.</n:abstract> </dc:description> <dc:date>2002-03-0213:4K/dc:date>
<dc : typ e>text</dc : typ e> <dc: format>text</dc: format> <dc: language>en</dc : language>
<dc:identifier>outlook:fooBarWhoKnowsl23xyz</dc:identifιer> </rdf:Description>
<subject>Software's Giants Gird for Upheaval [Fortune]</subject> <from>EghosaO@aol.com</from> <to>Nosa Omoigui [nosa@nervana.net]</to> <to>cogit@ludic m.org</to> <to>lisah@stanfordalumni.org</to> <to>stevenj007@hotmail.com</to> <size units="KB">13</size> </email> </results> </sπnl>
SAMPLE B — SEMANTIC QUERY DOCUMENT
<?xml version="1.0" encoding="utf-8"?> <sqml>
<head title="foo"> </head> <filters>
<include>
<type class="nervana:objecttype"> documents </type> </include>
</filters>
<attributes></attributes> <skins>
<documents color="http://nervana.net/dc.xslt" design="http://nervana.net/dd.xslt" animation="http://nervana.net/da.xslt" > </documents> <email color="http://nervana.net/ec.xslt" design- 'http://nervana.net/ed.xslt" animation="http://nervana.net/ea.xslt" > </email> </skins> <query>
<resource type- 'nervana:filepath"> c:\foo.doc </resource>
<resource type="nervana:url"> file://c:\bar.doc
</resource> <resource type="nervana:url"> file://c:?includesubfolders=tme </resource> <resource type="nervana:url"> http ://www.bar.com/doc.htm </resource>
<resource type— 'nervana:url"> 5 ftp://gate.com/doc.txt
</resource>
<resource type="nervana:url"> Wservers\server\file.pdf </resource> 10 <resource type- nervana:text" arg- 'contains=fox">
The quick brown fox </resource> ■^resource type="nervana:cacheentry"> ef9c90ea-282d-46d6-b355-ac8a4fc2Oe5 15 <linlc predicate="nervana:relatedto" type- 'nervana:url"> c:\foo.doc </link> </resource> 20 <resource type="nervana:url"> agent://email. all@ibm.com </resource> <resource type- 'nervana:url"> obj ects ://rad. com/agency . asp 25 <link predicate- 'nervana ontainstext" type="xml : string"> 80211 </link> </resource> 30 <resource
Figure imgf000408_0001
obj ects ://rad. com/agency . asp <link predicate="nervana:postedon" type- 'nervana: d atetimeref '> today ' 35 . </link> /075466
</resource>
<resource type="nervana:url"> obj ects ://rad . com/agency, asp <link predicate="nervana:postedon" type- 'xml:datetime">
01-10-2002 </link>' <link operator="or" predicate="nervana:postedbefore" type="xml:datetime">
01-11-2002 </link> </resource>
<resource type- 'nervana:uri"> agent://docurnents.all@abccoφ.com
<link predicate="nervana:relatedto" , type- nervana:url"> objects://98@in.com/m.asp </link> <link operator="and" , predicate- nervana.isofpriόrity" type- 'nervana:priority"> criticalpriority </link> </resource>
</query> </sqml> SAMPLE C — SEMANTIC ENVIRONMENT HKERARCHY
Infonnation Agent (the root of the Semantic Environment) My Agents
Frequently Used Agents Agencies
Agency Al
Documents
Documents.All
Documents. CriticalPriority.All Email
Email.Technology.Wireless.All Annotations
Annotations.RecentlyPosted.PastOneDa y.All People
People.Research.All Events
Events.All
Events.Upcoming.NextOneDay.All Agency A2
Documents
Documents. Technology.XML.XPath.All Email
Email.Favorites.All E-Leaming Courses
ELearning.All Agency A3
News Articles
News.Technology.Semiconductors.All
Agency AN... Recently Used Agents
[Similar hierarchy as above but for recently used Agents] Recently Created Agents [Similar hierarchy as above but for recently created Agents] Favorite Agents
[Similar hierarchy as above but for Agents marked by the user as favorites] All Agents
[Similar hierarchy as above but for all Agents in the My
Agents list] Deleted Agents
[Similar hierarchy as above but for Agents marked for deletion]
...Custom View
[Similar hierarchy as above but for Agents consistent with the custom view]
SAMPLE D — SQML OUTPUT FROM A HEADLINES CONTEXT TEMPLATE
<?xml version="1.0" encoding- 'utf-8"?> <sqml>
<head title="foo" type="all infoπnation"> </head> <filters>
<include>
<type class- 'nervana: obj ecttype"> all information </type> ■ </include>
</filters> <query>
<resource type="nervana:Agency"> wsAgency://marketing.com/ Agency, wsdl <link predicate— 'nervana:postedinthelast" type- 'nervana:time:minutesref>
30 :
</link> <link predicate="nervana:relevantto" type="nervana:sqml">
[object sqml] </link> </resource> <resource type="nervana:Agency"> wsAgency://research. com/Agency, wsdl
<link predicate="nervana:postedinthelast" type="nervana:time:minutesref>
30 </link> <link predicate="nervana:relevantto" type="nervana:sqml">
[object sqml] </link> </resource> <resource type="nervana:Agency"> wsAgency://sales. com/Agency .wsdl <link predicate- iιervana:postedinthelast" type="nervana:time:minutesref'>
30 </link> <link predicate="nervana:re!evantto" type="nervana:sqml"> [object sqml]
</link> </resource> <resource type="nervana:Agency"> wsAgency://humanresources.com/Agency.wsdl <link predicate="neιvana:postedinthelast" type— 'nervana:time:minutesref>
30 </link> <link predicate- iιervana:relevantto" type="nervana:sqml">
[object sqml] </link> </resource> </query> </sqml>
IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
Applicant: Nosa Omoigui Attorney Docket No. FORE-1-1001
Serial No.: — Group Art Unit: . ' —
Filing Date: June 24, 2002 Examiner: —
Title: SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
TRANSMITTAL LETTER FOR UTILITY PATENT APPLICATION
Enclosed with this trans'mittal letter are the following papers:
1. Application comprising 245 pages of written description and 73 drawing sheets.
2. Executed original Combined Declaration and Power of Attorney in Patent Application.
Check No. 6352 in the amount of $370 (filing fee). Applicant claims small entity status.
Return receipt postcard.
Respectfully submitted,
Figure imgf000414_0001
Registration No. 39,281 Direct Dial: 206.381.3303
EXPRESS MAIL CERTIFICATE
I hereby certify that this communication is being deposited with the United States Postal Service via Express Mail No. EL518605615US under 37 C.F.R. § 1.10 on the date indicated below addressed to: BOX PATENT APPLICATION,
Figure imgf000414_0002
Attorney Docket No. FORE-t-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT. DELIVERY AND PRESENTATION
Filed: June 24. 2002
Express Mail No. ELS 18605615US
Sheet 1 of 73
INFORMATION TOOLS LAYER
Hypertext (Dumb Links)
PRESENTATION LAYER
Client-static, server-generated HTML, Flash, SVG, etc.
PROTOCOL LAYER
HTTP
FIGURE 1
INFORMATION TOOLS LAYER
Hypermedia (Dumb Semantic Links) + Metadata + Semantics/Meaning
PRESENTATION LAYER
Client-static, server-generated HTML, Flash, SVG, etc. (generated from RDF.and XML)
PROTOCOL LAYER HTTP
SERVER-SIDE SEMANTIC QUERY LAYER SQL, XQuery, XPath, etc.
SEMANTIC WEB DATA LAYER Metadata with RDF, RDFS, etc.
FIGURE 2
Figure imgf000415_0001
FIGURE 3 NOSA OMOIGUI
Attorney Docket No. FORE-1-100I
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL.
MANAGEMENT. DELIVERY AND PRESENTATION
Filed: June 24, 2002
ExpTess Mail No. EL518605615US
Sheet 2 of 73
Figure imgf000416_0001
FIGURE 4 SYSTEM AND METHOD FOR KNOWLEDGE RETRlhVAL. MANAGEMENT, DELIVERY AND PRESENTATION Filed June 24, 2002 ExpresbMailNo EL518605615US Sheet 3 of 73 -:m i.i'^'^i
! J- u
Figure imgf000417_0001
FIGURE 5
Attorney Docket No. FORE- 1-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL.
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24. 2002
Express Mail No. EL518605615US
Sheet 4 of 73
Figure imgf000418_0001
FIGURE 6 SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL.
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL518605615US
Sheet 5 of 73
Figure imgf000419_0001
FIGURE 7
Attorney Docket No. FORE-1-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL
MANAGEMENT. DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL518605615US
Sheet 6 of 73
Figure imgf000420_0001
FIGURE 8 i'β. Dαcfet No. FORE-1-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT. DELIVERY AND PRESENTATION
Filed: June 24. 2002
Express Mail No. EL518605615US
Sheet 7 of 73
Figure imgf000421_0001
FIGURE 9 UbA UMUlϋUl
Attorney Docket No. FORE-l-IOOl
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24. 2002
Express Mail No. EL51S605615US
Sheet 8 of 73
Enterprise Knowledge Portals (EKPs)
Knowledge-Integrated E-Business Applications
Knowledge" Web Application. Services (Agents, Navigation, Connections, Discovery, Sharing)
Knowledge Logic Knowledge Query Services
Knowledge Ontology/Inference Services
Knowledge Representation
Figure imgf000422_0001
Knowledge Indexing and Classification
Structured Unstructured Information Sources Information Sources
Today's Web Information Nervous System
FIGURE 10 Attorney Docket No. FORE-1-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mai! No. EL518605615US
Sheet 9 of 73
Figure imgf000423_0001
Data Repositories
FIGURE 11 S) STEM AND METHOD FOR ΛNOH LEDGE RETRIEVAL,
SI NΛGEMENT DELIVERY AND PRESENTATION
Filed June 242002
Express Mail No EL51gό05fi_5US
Sheet 10 of 73 - s«t« 5t, '_iL,w . Ii -^-^5 -IfflJSl
__
_tl i vubli tictinnϊx*
Vie recently rεgl-lere- J_OdA_l]\Jlii3ti ) our domain name uith raffs' ,__rκ NIC S _3τm»
P SI niiim
Figure imgf000424_0001
i i Sb Vis Week
* =_.«
&*«
.-I) -t-«| a L U " A__≥__!_i___!__r S""" *"»» |B]FCBEI imwal I EJaAfcnifa n aM |fe]MlP;fMW_t (o,Q, |83l_s u5AHκi-r'.aι lcjlnbo- -.ltaaK-B-eM| )np»_n,lnI-,,n,»
FIGURE 12
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL.
MANAGEMENT, DELI ERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL5186G561SUS
Sheet II of 73
Figure imgf000425_0001
Alicπicy Docket No. FORE-1-lOOI
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24.2002
Express Mail No. EL51-605615US
Sheet 12 of 73
Figure imgf000426_0001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. E U6Q5615US
Sheet U or73
Breaking News Agent: Documents on Reuters Related to [My Current User Interface Specification]
Figure imgf000427_0001
Object Title: [Email: Yuying's Thoughts on the Current User Interface]
Link: All Information Related to this Item
Last Item Posted On: May 17, 12:13 PM [Preview]
Next Event Start on: June 28, 09:00 [Preview]
Show only items in the following type:
Show only items posted in the last:
Found a Total of 50 Items [Preview]
Figure imgf000427_0002
FIGURE 15
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT. DELIVER AND PRESENTATION
Filed June 24, 2002
Express Mail No EL5I860561-US
Sheet 14 of73
Figure imgf000428_0001
FIGURE 16
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL.
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL51S605615US
Sheet IS of 73
Figure imgf000429_0001
FIGURE 17
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL.
MANAGEMENT. DELIVER AND PRESENTATION
Filed June 24. 2002
Express Mail No EL51S6056I5US
SIιect l of73
Figure imgf000430_0001
FIGURE 18
Altoniey Docket No. FORE-I-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL.
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL518605615US
Slιccl l7 or73
Figure imgf000431_0001
FIGURE 19
Figure imgf000432_0001
Figure imgf000432_0002
FIGURE 20
Figure imgf000432_0003
FIGURE 21
Figure imgf000432_0004
FIGURE 22
Figure imgf000432_0005
FIGURE 23
Figure imgf000433_0001
Λ'U-iΛ U UIUUI
Attorney Docket No. FORE-1-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
E.xpress Mail No. EL51 S605615US
Sheet 20 of 73
Figure imgf000434_0001
FIGURE 25 NOSA OMOIGUI
Attorney Docket No. FORE-1-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL51 S605615US
Sheet 21 of 73
Figure imgf000435_0001
FIGURE 27 Attomev Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNO WLEDGE RETRIEVAL.
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. ELS 18605615US
S eet 22 of 73
Figure imgf000436_0001
FIGURE 29 Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL518605615US
Sheet 23 of 73
Figure imgf000437_0001
FIGURE 29 (continued)
NOSA OMOIGUI
Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL518605615US
Sheet 24 of 73
Figure imgf000438_0001
FIGURE 31
DOCUMENT OBJECT SCHEMA
Figure imgf000438_0002
FIGURE 32 Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL518605615US
Sheet 25 of 73
Figure imgf000439_0001
FIGURE 34
NUbA UMOIGUI
Attorney Docket No. FORE-I- 1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AHD PRESENTATION
Filed: June 24. 2002
Express Mail No. EL51S6056I 5US
Sheet 26 of 73
Figure imgf000440_0001
FIGURE 35
Figure imgf000440_0002
FIGURE 36
Email Public Folder Object Schema
Field Name Type
PublicFolderName Text
PublicFolderTypelD Number
Owner Text
PublicAccess Bool
SourceUrl Text
GURE 37 NOSA OMOIGUI
Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL.
MANAGEMENT. DELIVERY AND PRESENTATION
Filed: June 24, 2002
E.xpress Mail No. EL518605615US
Sheet 27 of 73
Figure imgf000441_0001
FIGURE 38
Figure imgf000441_0002
FIGURE 39
Event Type ID Value
EVENTTYPEIDJVlEETING 1
EVENTTYPEID CORPORATEEVENT
EVENTTYPEID INDUSTRYEVENT
EVENTTYPEID TVPROGRAM
EVENTTYPEID RADIOPROGRAM
EVENTTYPEID PRINTMEDIAPRQGRAM
EVENTTYPEID QNLINEMEETING 7
EVENTTYPEID ARTSANDENTERTAINMENT 8
FIGURE 40 NOSA OMOIGUI
Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL.
MANAGEMENT. DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL51 S605615US
Sheet 23 of 73
Figure imgf000442_0001
FIGURE 41
Figure imgf000442_0002
FIGURE 42
NOSA OMOIGUI
Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL.
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
ENpress Mail No. EL518605615US
Sheet 29 of 73
Figure imgf000443_0001
FIGURE 43
Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT. DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. E 518605615US
Sheet 30 of 73
Figure imgf000444_0001
FIGURE 44
Figure imgf000445_0001
Figure imgf000445_0002
FIGURE 45 Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT. DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL518605615US
Sheet 32 of 73
Figure imgf000446_0001
FIGURE 45 (continued) llUOf
Figure imgf000447_0001
Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. E 518605615US
Sheet 33 of 73
Email Object:
<email from="john(5).foo,com" to="nosa(S>nerva na.net" cc="steve@nervaπa,net" bcc="patrick@nervaπa.net" subject- ' eeting this Friday" body="Let us meet on Friday at 2pm" attachments="c:\foo.doc; c:\bar.htm" >
</email>
User Object:
Predicate: From
<user emailaddress- 'iohn(S)foo.com" > </user>
User Object:
Predicate: To
<user emailaddress="nosa(5)nervana,nβt" > </user>
User Object:
Predicate: Cc
<user emailaddress="steve@nervana.net" > </user>
User Object:
Predicate: Bcc
<user emailaddress="patrick@neryana.net" > </user>
Document Object:
Predicate: Attached To
<document path="c:\foo.doc" > </document>
Document Object:
Predicate: Attached To
<docυment pat -'cΛbar.htm" > </document>
FIGURE 46
Figure imgf000448_0001
ύ l i t.M AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed; June 24. 2002
Express Mail No. EL5I86056I5US
Sheet 35 of 73
Figure imgf000449_0001
. ,.vv run ANUHLEDOE ETRIEVAL,
MANAGEMENT. DEUVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL5I86Q5615US
Slιect36of73
Figure imgf000450_0001
FIGURE 49
ZIΛJIM ΛNU MtJllUD l-UK KNOWLEDGE RETRIEVAL,
MANAGEMENT. DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL518605615US
Sheet 37 of 73
Figure imgf000451_0001
FIGURE 50
Figure imgf000452_0001
Express Mail No. EL5186056I5US
Sheet 38 of 73
Figure imgf000452_0002
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL518605615US
Sheet 39 of73
Figure imgf000453_0001
SiSTEUAND METHOD FOR KNOWLEDGE RETKILV IL
MANAGEMENT DELIVERY AND PRESENTATION
Filed June 24 2002
Express Mail No CL5IB605615US
Sheet 40 of 73
Figure imgf000454_0001
FIGURE 53
NOSA O UlϋUI
Attorney Doc el No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL.
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
E.\press Mail No. EL5186056I5US
Sheet 41 of 73
Details Metadata (Balloon Popup)
Intrinsic
Alerts,
Indicator έreaking
Figure imgf000455_0001
News Alert
Indicator,
Annotation s Indicator,
Recomme ndation
Indicator
Recommenda Breaking Deep Lens Intrinsic Verb tions News Info Info Links s
FIGURE 54
NUiA UMU1UU1
Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL.
MANAGEMENT, DELIVERY AND PRESENTA TION
Filed: June 24, 2002
Express Mail No. E 5I8605615US
Sheet 42 of 73
Figure imgf000456_0001
FIGURE 55
NU-.A UMUKJUI
Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNO WLEDGE RETRIEV. IL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL518605615US
Sheet 43 of 73
Figure imgf000457_0001
FIGURE 56
NUbA UMUIUU1
Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mai! No. EL518605615US
Sheet 44 of 73
Info Nugget Template: [Template]
Steve Judkins is most likely to have authored this document [Preview]
This document is of category: [category]
This document has 3 likely experts: [Preview]
Steve Judkins has 6 direct reports [Preview]
Steve Judkins has posted a total of 500 items on this agency [Preview]
Steve Judkins has posted a total of 45 related items on this agency [Preview]
Steve Judkins is an expert on 78 documents on this agency [Preview]
Steve Judkins is hosting 12 upcoming events [Preview]
FIGURE 57
Figure imgf000459_0001
FIGURE 58
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. 0L5186056I5US
Sheet 46 of 73
Figure imgf000460_0001
SYSTEM AND METHOD FOR KNO WLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL518605615US
Sheet 47 of 73
Figure imgf000461_0001
FIGURE 60
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL5I8605S15US
Sheet 48 of 73
Figure imgf000462_0001
FIGURE 61
Allomey Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT. DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL518605615 US
Sheet 49 of 73
Figure imgf000463_0001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT. DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. E 5I 8605615US
Sheet 50 of 73
Figure imgf000464_0001
FIGURE 63
Attorney UOCKCLI O. 1-uκh-ι-ιtκιl
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL..
MANAGEMENT. DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL518605615 US
Slice.51 of 73
Figure imgf000465_0001
Alioπiey Ducket No. TORE-l-lCOl
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT. DELIVER Y AND PRESENTATION
Fileit: June 24, 2002
Express Mall No. EL51S605615US
Sheet 52 of 73
Figure imgf000466_0001
FIGURE 65
Altoniey Docket No FORC-I-IOOi
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT DELIVER YAND PRESENTATION riled June 24, 2002
Express Mail No CL5I8605615US
Slice! 53 of 73
Figure imgf000467_0001
FIGURE 66
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT. DELIVERY AND PRESENTATION
Filed; June 24, 2002
Express Mail No. EL51860S61SUS
Sheet 54 of73
Figure imgf000468_0001
SYSTEM AND METHOD FOR KNOWLEDGE RETRlhVAL,
MANAGEMENT, DELIVERY AND PRESENIATION
Filed. June 24, 2002
Express Mail No CL518605615US
Sheet 55 of73
Figure imgf000469_0001
FIGURE 68
NOSA OMOIGUI
Attorney Docket No. FORE-1-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL51S605615US
Sheet 56 of 3
Agent Lens: Documents on Reuters Related to [My Nervana Ul Specification]
Object Title: [Email: Yuying's Thoughts on the Nervana Ul]
Link: All Information Related to this Item
Last Item Posted On: May 17, 12:14PM [Previewl
Next Event Starting On: June 28, 09:00AM [Previewl
Show only items of the following type:
Show only items posted in the last:
Found a Total of 50 Items [Previewl
FIGURE 69 NOSA OMOIGUI
Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL.
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. E 518605615US
Sheet 57 of 73
Object Lens Title: , [My Nervana Ul Specification]
Agent Title: Documents bn Reuters Related to [My Nervana Ul Specification]
Link: All Information Related to this Item
Last Item Posted On: May, 17, 12:14PM [Previewl
Next Event Starting On: June 28, 09:00AM [Previewl
Show only items of the following type:
V
Show only items posted in the last:
Found a Total of 50 Items [Previewl
FIGURE 70 Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24. 2002
Express Mail No. E 5IS605615US
Sheet 58 of 73
Object Lens Title: [My Nervana UI Specification]
Agent Title: Documents on Reuters Related to [My Nervana Ul
Link: All Information Related to this Item
Last Item Posted On: May 17, 12:14PM [Preview]
Next Event Starting On: June 28, 09:00AM [Preview]
Show only items of the following type:
Show only items posted in the last:
Found a Total of 50 Items [Preview!
FIGURE 71
NUSA UMUlCiUI
Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL5I8605615US
Sheet 59 of 73
•Object Lens Title: [My Nervana Ul Specification]
Object Title: [Email: Yuying's Thoughts on the Nervana Ul]
Link: Relatedness Measure according to Agency [Agency Name] 1
Relatedness Measure: 93%
0% 100%
FIGURE 72
INU5A VIU1UU1
Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. E 518605615US
Sheet 60 of 73
Figure imgf000474_0001
FIGURE 73 Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT. DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL51S605615US
Sheet 61 of 73
Figure imgf000475_0001
FIGURE 74
Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL5I8605615US
Sheet 62 of 73
Figure imgf000476_0001
FIGURE 75
nuoΛ uiviuiυυi
Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNO WLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. E 518605615US
Sheet 63 of 73
Figure imgf000477_0001
FIGURE 76
NOSA OMOIGUI
Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL.
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. E 51 S605615US
Sheet 64 of 73
Figure imgf000478_0001
FIGURE 77
Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL
MANAGEMENT. DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. E 518605615US
Sheet 65 of 73
Tool Regular Web Browser Information Agent
Back Navigates to the previous Navigates to the previous agent web page
Forward Navigates to the next web Navigates to the next agent page
Home Navigates to the "home Navigates to the "home agent" page"
Stop Stops the current operation Stops the current operation (or semantic query request)
Refresh Refreshes the current Web Refreshes the current semantic query page and also refreshes the semantic environment (and Ul and the tree view reflecting this)
Print Prints the current Web page Prints the currently displayed ., information objects (semantic query results)
FIGURE 78
Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT. DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL5I86056I5US
Sheet 66 of 73
Tool File System Explorer or
Document Viewer Semantic Browser
New Creates a new file (via its Creates a new smart agent (via the application) or folder "Create Smart Agent" wizard or a new blender (via the "Create Blender" wizard)
Open Opens the selected file (via Opens an agent (via the "Open Agent" its application) or folder dialog), file-system documents (to import them into the semantic environment and make them smart and context/time- aware), searches for local file-system documents (ditto), opens a URL (like an Internet HTTP or FTP address) (ditto), or opens what object that is currently on the clipboard (ditto)
Open ^ Opens a dialog box that Opens a dialog box that allows the user to
Document allows the user to browse browse his/her semantic environment in Dialog his/her file-system order to open a desired agent or just to environment in order to browse the environment open a desired document or just to browse the environment
Save Saves the current document Saves the current agent into the "favorites" list (if not already a favorite) or copies the current agent into a blender
FIGURE 79
INUώΛ IVI IUUI
Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. E 518605615US
Sheet 67 of 73
Tool File System Explorer or Document Viewer Semantic Browser
Copy Copies the currently selected file, From the toolbar of the semantic browser, if a folder, or textual/graphical/link Smart Agent is currently selected or opened, the information to the clipboard so it SQML for the Agent is copied to the clipboard can be pasted to a different folder using the proprietary SQML clipboard format. This (in the case of files and/or folders) enables the smart lens functionality - the user can or pasted to the same or a new then select any information object in the Presenter document (in the case of window and select the "paste as smart lens" icon. textual/graphical/link information) This then opens an in-place smart lens pane (see the sample lens pane in the section showing sample Uls) that displays the results of the relational guery that is based on merging the SQML of the underlying object with the SQML on the clipboard (by adding the SQML of the underlying object as an argument to the SQML on the clipboard with an appropriate predicate (likely the default predicate) and a link to the object's resource identifier or URL. Essentially, this creates a new semantic SQML by relationally merging the SQML for the "lens" and the "object" to create an "image."
Preferably only Smart Agents can be copied, pasted, and used as Smart Lenses. In addition, only objects returned by queries on Smart Agents can contain deep info. However, all objects can be dragged, dropped, copied, pasted, and can automatically generate context palettes (including dumb objects - e.g., from dumb agents that refer to the local file-system). This is how "dumb" objects can be made "smart" once they are in the semantic sandbox or environment.
If the "Copy" operation is invoked on an object in the Presenter results window (e.g., via the "Copy" verb on a popup menu), this also copies the SQML of the object to the clipboard. The Presenter will derive the SQML of the object from the object's XML (or SRML) - primarily by adding a reference to its identifier or URL/path.
FIGURE 79 (continued) Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT. DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL518605615US
Sheet 63 of 73
Tool File System Explorer oir
Document Viewer Semantic Browser
Paste Pastes the contents of the From the toolbar of the semantic browser, if a clipboard to the selected smart agent is currently selected or opened, the destination. If the destination is a SQML for what is on the clipboard is inserted into folder, this pastes the contents of the SQML for the smart agent to create a new the selected file or folder (if SQML buffer or file that includes the relational appropriate). If the destination is semantic query with the clipboard object as the a document, this pastes the filter argument. The tool on the toolbar will only be textual, graphical or link content enabled if there is something on the clipboard and of what is on the clipboard. if what is on the clipboard can be pasted onto the agent. The semantic browser will check this by examining the format of what is on the clipboard. If it is the SQML format or is a format that can be converted to SQML, the tool is enabled. If necessary, implicit conversion to SQML is performed when the Paste operation is invoked. Note that if the SQML on the clipboard is that of a smart agent, the semantic browser can support it if the agents belong to the same agency - in this case, the query will result in a complex sub-query that filters the results of the destination agent with the results of the source agent. If the SQML on the clipboard belongs to the currently selected agent (e.g., if the user wants to hit "Copy" and then immediately hit "Paste" on the same smart agent), the tool should be disabled since this will be a redundant operation (akin to the user copying a piece of text in a document and pasting over the same piece of text).
If the "Paste" operation is invoked on an object in the Presenter results window (via, say, the "Paste as Smart Lens" icon on an object), this opens an in-place smart lens pane (see the sample lens pane in the section showing sample Uls) that displays the results of the relational query that is based on merging the SQML of the underlying object with the SQML on the clipboard (by adding the SQML of the underlying object as an argument to the SQML on the clipboard with an appropriate predicate (likely the default predicate) and a link to the object's resource identifier or URL. Essentially, this creates a new semantic SQML by relationally merging the SQML for the "lens" and the "object" to create an "image."
FIGURE 79 (continued) Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNO WLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. E 518605615US
Sheet 69 of 73
Tool File System Explorer or Document Viewer Semantic Browser
Properties Displays a dialog box showing the If an agent (smart or dumb) is currently selected properties of the selected file or or opened, this displays a dialog box showing the folder, or the current document. properties of the agent (e.g., the name, description, the creation date/time, the last used date/time, etc.
Delete Deletes the currently selected file, Deletes the currently selected or opened agent folder, graphic object, link, or text and refreshes the environment to reflect the new state
Drag and Drop Copies or moves the selected file, Links the file(s), folder(s), URL, text, or other local folder, graphics or link from its contextual link that are dragged to the smart agent source to the destination. If the that they are dragged to, in order to create a new object is a file or folder, the smart agent that contains a semantic query that contents are copied to the reflects the relationship between the dragged destination folder. If the object is object (the context) and the smart agent (the a graphic or text, the contents are destination). The operation results in a new smart copied to the "paste" section in agent with a new SQML buffer that has the the same document or a new original SQML of the smart agent (destination) document. that the source (files, folders, URLs, etc.) were dragged on, filtered with the references of the source. The new SQML is then "compiled" by the client, individual SQMLs are then sent to the server via its XML Web service (the reference of which is copied from the SQML of the destination smart agent), qualified with the XML metadata of the source references. For instance, if the source is a file document, the' document metadata is extracted and the metadata sent to the XML Web Services as an SQML argument. If the source has multiple files or is a folder, the metadata.for the files in the folder are sent (qualified by whether the source arguments are to be used as a UNION or INTERSECTION filter). If the source is an URL (HTTP, FTP, etc.), the URL is first downloaded and then the metadata for the document the URL refers to is sent as an SQML argument to the XML Web Services in the new SQML buffer or file. The browser will first validate that the source refers to an object type that the server can understand (e.g., a document, an email object, a person, etc.). If it is not a valid object, the browser should ignore the request and/or notify the user via a dialog alert message. •
Hierarchical Displays a hierarchical view of the Displays a hierarchical view of the semantic
Object View folders in the file-system environment, including agent views, agency environment views, agents viewed by "all," "context," "category," "time," "agency," "information type," "blender", etc., and agencies viewed by "ati," "category," "time," etc. (see screenshots)
FIGURE 79 (continued) tυjn υiriυi ui
Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL,
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL5I 8605615US
Sheet 70 of 73
Palette Vertical Results Horizontal Results
Results Navigation Scroll Navigation Scroll
Object Bar Bar
Palette Results Object
Palette Window
Figure imgf000484_0001
Main results for Headlines History Context Guide Context the current Context Palette Palette for Palette for agent for Object 1 Object 1 Object 1
FIGURE 80
rrυjn um iuul
Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL.
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL5I8605615US
Sheet 71 of 73
Last Item Posted On: May 17, 12:14PM [Preview]
Next Event Startinq On: June 28, 09:00AM [Previewl
Show only items of the followinq type: V
Show only items posted in the last: 1
Found a Total of 50 Items [Preview!
FIGURE 81
Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL.
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL518605615US
Sheet 72 of 73
Information Tools Layer
Proprietary — Hypermedia (Smart Semantic Links) +
Metadata + Semantics/Meaning + Drag and Drop + Smart
Copy and Paste + Intrinsic Alerts + Smart Lens + + Deep Info
+ Intrinsic Verbs + Intrinsic Links + Smart Agents + Context
Templates (Special Agents) + Blenders + Smart
Recommendations + Breaking News Agents + smart, context-sensitive links to breaking news Agents + semantic hints + Preview Windows + Presentation Skins (subtle, moderate, exciting, animations, etc) + Smart Screensavers, etc.
Presentation Layer
Dynamic, client-side, configurable, context-sensitive, time-sensitive "skins" - client-side conversion of XML to XHTML+TIME, SVG, etc. using XSLT.
Client-Side Semantic Query Layer
SQML (Proprietary)
Protocol Layer XML Web Services
Server-Side Semantic Query Layer
SQL, XQuery, XPath, etc.
Semantic Network Data Model
(Proprietary — data stored in database)
Semantic Data Source Adapters (DSAs) for Semantic and Non-Semantic Data Sources
Existing stores for documents, email, contacts, events, customer data, e-learning data, etc., metadata stores (RDF, XML, databases, etc.); proprietary mapping from data sources to semantic network
FIGURE 82 Attorney Docket No. FORE-l-1001
SYSTEM AND METHOD FOR KNOWLEDGE RETRIEVAL.
MANAGEMENT, DELIVERY AND PRESENTATION
Filed: June 24, 2002
Express Mail No. EL518605615US
Sheet 73 of 73
Figure imgf000487_0001
1. Metadata
2. Semantics (Meaning, Context, Deep Info, etc.)
3. Context-Sensitivity (Drag and Drop, Smart Copy and Paste, Smart Skins, Intrinsic Links, Smart Lens Initiation and Response, Verbs, Deep Info, Smart Recommendations, etc.)
4. Time-Sensitivity (Intrinsic Alerts, links to Breaking News Agents)
5. Automatic and Intelligent Discoverability (Links to new agencies in the namespace)
6. Dynamic Linking (drag and drop, smart copy and paste)
7. User-Controlled Navigation and Browsing (drag and drop, smart copy and paste, intrinsic alerts, breaking news, etc.)
8. Flexible Presentation (Smart Skins, Smart Screensavers, Animations, Semantic Ul Hints, Preview Windows, etc.)
9. Non-HTML and Non-Local Document Participation in the Network (Links to Dumb Agents that get information from sources without agencies (e.g., the local file system), responses to Smart Lens from local files/content, Drag and Drop from local files/content, etc.)
10. Flexible User-Driven Information Analysis (Smart Lens, Deep Info, Preview Windows, Pivot Views, etc.)
11. Flexible semantic queries (Ability to create, edit, and link to Smart Agents, Drag and Drop, Smart Lens, and Smart Copy and Paste as the basis pf semantic queries, etc.)
12. Information Packages/Blenders (Ability to drag and drop or copy/paste object to blender to create a new blender with new sub-queries, smart agent results can form blender, object can respond to blender when blender is used as a smart lens, etc.)
13. Pivots for Context Templates (ability for user to create a Headlines Smart Agent, Newsmakers Smart Agent, etc. using this object as a pivot)
FIGURE 83 Appendix B
Additional Inventions
Additional Inventions for the Information Nervous System
By: Nosa Omoigui
Figure imgf000489_0001
EffEff -fcjggSfflffHBH
Overview
The Smart Selection Lens is similar to the Smart Lens feature of the Information Nervous System information medium. In this case, the user can select text within the object and the lens will be applied using the selected text as the object (dynamically generating new "images" as the selection changes). This way, the user can lens over a configurable subset of the object metadata, as opposed to being constrained to lens over the entire object or none at all. It is akin to a selection cursor/verb overloaded with context. For example, the user can select a piece of text in the Presenter and hit the "Paste as Lens" icon over the object in which the text appears. The Presenter will then pass the text to the client runtime component (e.g., an ActiveX object) with a method call like: bstrSRML = GetSRMLForText( bstrText );
This returns a temporary SRML buffer that encapsulates the argument text. The Presenter will then call a method like: bstrSQML = GetQueryForSmartLensOnObject( bstrSRMLObject );
This method get the SQML from the clipboard, takes the argument SRML for the object, and dynamically creates new SQML that includes the resource in the SRML as a link in the SQML (with the default predicate "relevant to"). The method then returns the new SQML. The Presenter then calls the method:
ProcessSemanticQuery( bstrSQML);
It passes the generated lens SQML and then picks up the number of items in the results and the SRML results asynchronously. For details on this call, see the spec "Information Nervous System Semantic Runtime OCX." The Presenter then displays a preview window (or the equivalent, based on the current skin) with something like:
[Lens Agent Title]
Found 23 items f PREVIEW OBJECT 1 1
[PREVIEW WINDOW CONTROLS 1 where the "Lens Agent Title" is the title of the agent on the clipboard. For details of the preview window (and the preview window controls), see the main patent application.
The preview window should: . • .'
• Disappear after a timer expires (maybe 500ms) - on mouse move, the timer should be reset (this will avoid flashing the window when the user moves the mouse around the same area)
• Fade out slowly (eventually)
Figure imgf000490_0001
In addition, the preferred embodiment should have the following features:
1. One selection range per object but multiple selections per results-set is the best option. Else, it will result in a confusing user experience and complex Ul to show lens icons per selection per object (as opposed to per object).
2. Outstanding lens query requests (which are regular SQML queries, albeit with SQML dynamically generated consistent with the agent lens) should be cancelled when the Presenter no longer needs them (e.g. if the Presenter is navigating to a new page, or if we are requesting new lens info for an object). In any case, this is not critical from a performance (and bandwidth) standpoint since lens queries will likely only ask for a few objects at a time. Even if the queries are not cancelled, the Presenter can ignore the results. In any case, the Presenter has to deal with stale results, dropping them on the floor - the Presenter will have to do this "anyway (whether or not lens queries are also cancelled). There will be a window between when the Presenter issues a cancel request and when the cancellation actually gets done. Some results can trickle in during this time and they need to be dropped on the floor. This is typical with asynchronous cancellation implementations - the software component always needs to be prepared to ignore bad or stale results.
3. The Presenter should have both icons (indicating the current lens request state) and tool-tips: When the user hovers over or clicks on an object, the Presenter can put up a tool-tip with the words, "Requesting Lens Info" (or words to that effect). When the info comes back, hovering will show the "Found 23 Objects" tip and clicking will show the results. This interstitial tool tip can then be transitioned to the preview window if it is still up when the results arrive.
In addition, note that the smart selection lens, like the smart lens, can be applied to objects other than textual metadata. For instance, it can be applied to images, video, a section of an audio stream, or other metadata. In these cases, the Presenter would return the appropriate SRML consistent with the data type and the "selection region." This region could be an area of an image, or video, a time span in an audio stream, etc. The rest of the smart lens functionality would apply as described above, with the appropriate SQML being generated based on the SRML (which in turn is based on the schema for the data type under the lens).
Pasting Person Objects
Overview
The Information Nervous System also supports the drag and drop or copy and paste of 'Person' objects (People, Users, Customers, etc.). There are two scenarios in this case:
1. Pasting a Person object on a smart request representing an Information Community (or Agency) from whence the Person came. In this case, the server's semantic query processor merely resolves the SQML from the client using the Person as the argument. For instance, if the user pastes (or drags and drops) a person 'Joe' on top of a smart request 'Headlines on Reuters,' the client will create a new smart request using the additional argument. The Reuters Information Nervous System Web service will then resolve this request by returning all Headlines published or annotated by 'Joe.' In this case, the server will essentially apply the proper default predicate ('published or annotated by') - that makes sense for the scenario.
2. Pasting a Person object on a smart request representing an Information Community (or Agency) from whence the Person did not come. In this case, because the Person object is not in the semantic network of the destination Information Community (on its SMS), the server's semantic query processor would not be able to make sense of the Person argument. As such, the server must resolve the Person argument using the categories on which the person is an expert (in the preferred embodiment) or a newsmaker. For instance, taking the above example, if the user pastes (or drags and drops) a person 'Joe' on top of a smart request 'Headlines on Reuters' and Joe is not a person on the Reuters Information Community, the Reuters Web service (in the preferred embodiment) must return Headlines that are "relevant to Joe's expertise." This requires that the client must take a two-pass approach before sending the SQML to the destination Web service. First, it must ask the Information Community that the person belongs to for "representative data (SRML)" that represents the person's expertise. The Web service resolves this request by: a. Query the Information Community on which the person object is pasted or dropped for its semantic domain information (e.g., Reuters) - this represents the taxonomy and ontology. Note that there could be several semantic domains. b. Query- the Information Community from whence the person object came for its semantic domain infomiation. c. If the semantic domains are identical or if there is at least one common semantic domain, the client queries the Information Community from whence the person came for the person's categories of expertise. The client then constructs SQML with these categories as arguments and passes this SQML to the Information Community on which the person was pasted or dropped. d. If the semantic domains are not identical or there is not least one common semantic domain, the client queries the Information Community from whence the person came for several objects that belong to categories on which the person is an expert. In the preferred embodiment, the implementation should pick a high enough number of objects that accurately represent the categories of expertise (this number should be picked based on experimentation). The reason for picking objects in this case is that the destination Web service will not understand the categories of the Information Community from whence the person came and as such will not be able to map them to its own categories. Alternatively, a category mapper can be employed (via a centralized Web service on the Internet) that maps categories between different Information Communities. In this case, the destination Infonnation Community will always be passed categories as part of the SQML, even though it does not understand those categories - the Information Community will then map these categories to internal categories using the category mapper Web service. The category mapper Web service will have methods for resolving categories as well as methods for publishing category mappings. Saving and Sharing Smart
Overview
Users of the Information Nervous System semantic browser (the Information Agent or Librarian) will also be able to save smart requests to disk, email them as an attachment, or share them via Instant Messenger (also as an attachment) or other means. The client will expose methods to save a smart request as a sharable document. The client will also expose methods to share a smart request document as an attachment in email or Instant Messenger.
A sharable smart request document is a binary document that encapsulates SQML (via a secure stream in the binary format). It provides a safe serialized representation of a semantic query that protects the intellectual property of the specification. The protection can be accomplished by strongly encrypting the XML version of the semantic query (the SQML) or via a strong one-way hash. The sharable document has an extension (.REQ) that represents the request. An extension handler on the client operating system is installed to represent this extension. When a document with the extension is opened, the extension handler is invoked to open the document. The extension handler opens the document by extracting the SQML from the secure stream, and then creating a smart request in the semantic namespace with the SQML. The handler then opens the smart request in the semantic namespace.
When a smart request in the semantic namespace is saved or if the user wants to send it as an email attachment, the client serializes the SQML representing the smart request in the binary .REQ format and saves it at the requested directory path or opens the email client with the .REQ document as an attachment.
Figure 1 below shows the binary document format that encapsulates the SQML buffer with the smart request and also illustrates how the extension handler opens the document. A similar model can also be employed for sharing results (via SRML). In this case, a binary document encapsulates the SRML, rather than the SQML as in the case above.
Figure imgf000493_0001
Figure 1: Sharable Smart Request System Interaction
Figure 2 below shows an illustration of two .REQ documents (titled 'Headlines on Reuters Related to My Research Report (Live)' and 'Headlines on Reuters (as of January 21 2003, 08 17AM)' on the far right) with a registered association in the Windows shell. The first request document is 'live' and the second one is a snapshot at a particular time (they are both time-sensitive requests). Notice that the operating system has associated the semantic browser application (Nervana Librarian) with the document. When the document is opened, the semantic query gets opened in the application.
aMS-SHrSSMasaw irV&r i,' - ,• . _ja._ϋ ,„ t
- — ... . ... - ... .. __ -Jfl" .<- •
D~ u~~ t 1_J »-— u — t *~ 4\r ~
^ -• &F - ■
Figure 2: Screenshot showing two saved semantic request documents (one live and one snapshot)
Saving and sharing entities - the same process applies as above except with a .ENT extension to represent an entity. When an entity document is invoked, the Nervana Librarian opens the entity SQML in the browser.
Extension Property Sheet - this will create a temporary smart request or entity (depending on the kind of document) in the semantic environment and display the property sheet for a smart request or entity
Extension Tool tips - this will display a helpful tool tip when the user hovers over a librarian document (a request, .REQ or an entity, .ENT)
Saving and Sharing Smart Snapshots
Overview
The Information Nervous System also supports the sharing of what the inventor calls "Smart Snapshots." A smart snapshot is a smart request frozen in time. This will enable a scenario where the user wants to share a smart request but not have it be "live." For instance, by default, if the user shares the smart request "Breaking News on Reuters related to this document" with a colleague, the colleague will see the live results of the smart request (based on the "current time"). However, if the user wants to share "[Current] Breaking News on Reuters related to this document," a smart snapshot will be employed.
A smart snapshot is the same as a smart request (it is also represented by an SQML query document) except that the "attributes" section of the SQML document contains attributes marking it as a snapshot (the flag QUERYATTRIBUTES_SNAPSHOT). The creation date/time of the SQML document is also stored in the SQML (as before - the SQML schema contains a field for the creation date/time). When the user indicates that he/she wants to share the smart request, the user interface (the semantic browser, Information Agent, or Librarian) prompts him/her whether he/she wants to share the smart request (live) or a smart snapshot. If the user indicates s smart request, the process described above (in Part 3) is employed. If the user indicates a smart snapshot, the binary document is populated with the edited SQML (containing the snapshot attribute) and the remainder the process is followed as above.
When the recipient of the binary document receives it (by email, instant messaging, etc.), and opens it, the extension handler opens the document and adds an entry into the semantic namespace as a smart request (as described above). When the recipient opens the smart request, the client's semantic query processor will send the processed SQML to the server's XML web service (as previously described). The server's semantic query processor then processes the SQML and honors the snapshot attribute by invoking the semantic query relative to the SQML creation date/time. As such, results will be relative to the original date/time, thereby honoring the intent of the sender.
Figure imgf000496_0001
Virtual Information Communities (agencies) refer to a feature of the Information Nervous System that allows the publisher of an information community to publish a group of servers to appear as though they were one server. For instance, Reuters could have per-industry Reuters information communities (for pharmaceuticals, oil and gas, manufacturing, financial services, etc.) but might also choose to expose one 'Reuters' information community. To do this, Reuters will publish and announce the SQML for the virtual information community (rather than the URL to the WSDL of the XML Web Service). The SQML will contain a blender (or collection) of the WSDLs of the actual information communities. The semantic browser will then pick up the SQML and display an icon for the information community (as though it were a single server). Any action on the information community will be propagated to each server in the SQML. If the user does not have access for the action, the Web service call will fail accordingly, else the action will be performed (no different from if the user had manually created a blender containing the information communities).
plementing Time-Sensitive Semantic Queries
Semantic queries that are time-sensitive must be implemented in an intelligent fashion to account for the rate of knowledge generation at the information community (agency) in question. For instance, 'Breaking News' on a server that receives 10 documents per second is not the same as 'Breaking News' on a server that receives 10 documents per month. As such, the server-side semantic query processor must adjust its time-sensitive semantic query handling according to the rate at which information accumulates at the server. To implement this, general rules of thumb could be used, for instance:
• The most recent N objects where N is adjusted based on the number of new objects per minute
• All objects received in the last N minutes with a cap on the number of objects (i.e., min (cap, all objects received in the last N minutes)).
N can also be adjusted based on whether the query is a Headline or Breaking News. In the preferred embodiment, newsmaker queries should be implemented with the same time-sensitivity parameters as Headlines. Text-To-Speech Skins
Overview
Text-to-speech is implemented at the object level and at the request level. At the object level, the object skin runs a script to take the SRML.of the object, interprets the SRML, and then passes select pieces of text (in the SRML fields) to a text-to-speech engine (e.g., using the Microsoft Windows Speech SDK) that generates voice output.
Figure 3 below shows an illustration of an email message being rendered via a text-to-speech object skin. When executed, the pipeline shown in Figure 3 results in the following voice output:
1 Reading Email Message
2. Appropriate Delay
3. Message From Nosa Omoigui
4. Appropriate Delay
5. Message Sent to John Smith
6. Appropriate Delay
7. Message Copied To Joe Somebody
8. Appropriate Delay
9. Message Subject Is Web services are software building blocks used for distributed computing
10. Appropriate Delay
11. Message Summary is Web services
12. Appropriate Delay
13. [Optional] Message Body is Web services are software building blocks used for distributed computing
This example assumes a voice skin template as follows:
1. Reading Email Message
2. Appropriate Delay
3. Message From <message author name>
4. Appropriate Delay
5. Message Sent to <message to: recipient name>
6. Appropriate Delay
7. Message Copied To <message cc: recipient name>
8. Appropriate Delay
9. Message Subject Is <message subject text>
10. Appropriate Delay
11. Message Summary is <message body summary>
12. Appropriate Delay
13. [Optional] Message Body is <message body>
Other templates can also be used to render voice that is easily understandable and which conveys the semantics of the object type being rendered. Like the example shown above (which is for email), the implementation should use appropriate text-to-speech templates for all information object types, in order to capture the semantics of the object type. At the request level, the semantic browser's presentation engine (the Presenter) loads a skin that takes the SRML for all the current objects being rendered (based on the user-selected cursor position) and then invokes the text-to-speech object skin for each object. This essentially repeats the text-to-speech action for each XML object being rendered, one after another.
Figure imgf000498_0001
Figure 3: Diagram Illustrating Text-to-Speech Object Skin
Figure 4 below shows an illustration several email objects being presented in the semantic browser via a request skin. .
Figure imgf000499_0002
From: Nosa Omoigui To: John Smith Cc: Joe Somebody Subject: Web services Summary: Web services are software building blocks used for distributed computing Body: Web services...
Object Skin (Object 1)
Figure 4: Diagram Illustrating Text-to-Speech Request Skin
Figure imgf000499_0001
Language translation skins are implemented similar to text-to-speech skins except that the transform is on the language axis. The XSLT skin (smart style) can invoke a software engine to automatically perform language translation in real-time and then generate XML that is encoded in Unicode (16 bits per character) in order to account for the universe of languages. The XSLT transform that generates the final presentation output then will render the output using the proper character set given the contents of the translated XML.
Categories; as First-Class
Objects in the User
Experience
This refers to a feature by which categories on an infonmation community are exposed to the end user. The end user will be able to issue a query for a category as an information type - e.g., 'Web services.' The metadata will then be displayed in the semantic browser, as would be the case for any first-class information object type. Visualizations, dynamic links, context palettes, etc. will also be available using the category object as a pivot. This feature is useful in cases where the user wants to start with the category and then use that as a pivot for dynamic navigation, as opposed to starting off with a smart request (smart agent) that has the category as a parameter.
Categorized Annotations
Categorized annotations follow from categories being first-class objects. Users will be able to annotate a category directly- thereby simulating an email list that is mapped to a category. However, for cases where there are many categories (for instance, in pharmaceuticals), this is not recommended because information can belong to many categories and the user should not have to think about which category to annotate - the user should publish the annotation directly to the information community (agency) where it will be automatically categorized or annotate an object like a document or email message that is more contextual than a category.
Figure imgf000500_0001
1. Experts - this was indicated as a special agent in the original patent application but was erroneously left out of the context templates section. Experts are a context template and as the name implies indicate people that have expertise on one or more subject matters or contexts (indicated by the PREDlCATETYPEID_EXPERTON predicate).
2. Annotations of My Items - this is a context template that is a variant of Annotations but is further filtered with items that were published by the calling user. This will allow the user to monitor feedback specifically on items that he/she posted or annotated.
HTJJ Importing and Exporting User State — -'
The semantic browser will support the importation and exportation of user state. The user will be able to save his/her personal state to a document and export it to another machine or vice-versa. This state will include information (and metadata) on:
• Default user state (e.g., computer sophistication level, default areas of interest, default job role, default smart styles, etc.) Profiles
Entities (per profile) Smart requests (per profile) Local Requests (per profile) Subscribed information communities (per profile) The semantic browser will show Ul (likely a wizard) that will allow the user to select which of the user state types to import or export. The Ul will also ask the user whether to include identity/logon information. When the Ul is invoked, the semantic browser will serialize the user state into an XML document that has fields corresponding to the metadata of all the user state types. When the XML document is imported, the semantic browser will navigate the XML document nodes and add or set the user state types in the client environment corresponding to the nodes in the XML document.
Local Smart Requests
Local smart requests would allow the user to browse local information using categories from an information community (agency). In the case of categorized local requests, the semantic client crawls the local hard drives, email stores, etc. extracts the metadata (including summaries) and stores the metadata in a local version of the semantic metadata store (SMS). The client sends the XML metadata (per object) to an information community for categorization (via its XML Web Service). The information community then responds with the category assignment metadata. The client then updates the local semantic network (via the local SMS) and responds to semantic queries just like the server would. Essentially, this is equivalent to having a local server.
Figure imgf000501_0001
Integrated Navigation allows the user to dynamically navigate from within the Presenter (in the main results pane on the right) and have the navigation be integrated with the shell extension navigation on the left. Essentially, this merges both stacks. In the preferred embodiment, this is accomplished via event signaling. When the" Presenter wants to dynamically navigate to a new request, it sets some state off the GUID that identifies the current browser view. The GUID maps to a key in the registry that also has a field called 'Navigation Event,' 'Next Namespace Object ID' and 'Next Path.' The "Navigation Event' field holds a DWORD value that points to an event handle that gets created by the current browser view when it is loaded. When the Presenter wants to navigate to a new request, it creates the request in the semantic environment and caches the returned ID of the request. It then dynamically gets the appropriate namespace path of the request (depending on the information/context type of the request) and caches that too. It then sets the two fields ('Next Namespace Object ID' and 'Next Path' with these two values). Next, it sets the 'Navigation Event' (in Windows, this is done by calling a Win32 API named 'SetEvent').
To catch the navigation event, the browser view starts a worker thread when it first starts. This thread waits on the navigation event (and also simultaneously waits on a shutdown event that gets signaled when the browser view is being terminated - in Windows, it does this via a Win32 API named WaitForMultipleObjects'). If the navigation event is signaled, the 'Wait' API returns indicating that the navigation event was signaled. The worker thread then looks up the registry to retrieve the navigation state (the object id and the path). It then calls the shell browser to navigate to this object id and path (in Windows, this is done by retrieving a 'PIDU and then calling IShellBrowser::BrowseTo off the shell view instance that implements IShellView). Hints for Visited Results
The Nervana semantic browser empowers the user to dynamically navigate a knowledge space at the speed of thought. The user could navigate along context, information or time axes. However, as the user navigates, he/she might be presented with redundant information. For instance, the user can navigate from a local document to 'Breaking News' and then from one of the 'Breaking News' result objects to 'Headlines.' However, semantically, some of the Headlines might overlap with the breaking news (especially if not enough time has elapsed). This is equivalent to browsing the Web and hitting the same pages over and over again from different 'angles.'
The Nervana semantic browser will handle this by having a local cache of recently presented results. The Presenter will then indicate redundant results to the user by showing the results in a different color or some other Ul mechanism. The local cache is aged (preferably after several hours or the measured time of a typical 'browsing experience'). Old entries are purged and the cache is eventually reset after enough time might have elapsed.
The semantic browser will also handle duplicate results by removing duplicates before rendering them in the Presenter - for instance if objects with the same metadata appear on different information communities (agencies). The semantic browser will detect this by performing metadata comparisons. For unstructured data like documents, email, etc., the semantic browser will compare the summaries - if the summaries are identical the documents are very likely to be identical (albeit this is not absolutely guaranteed, especially for very long documents).
Figure imgf000502_0001
Knowledge Federation is technology that allows external knowledge to be federated within the confines of a knowledge community. For instance, many companies rely on external content providers like Reuters to provide them with information. However, in the Information Nervous System, security and privacy issues arise - relating to annotations, personal publications, etc. Many enterprise customers will not want sensitive annotations to be stored on remote servers hosted and managed by external content providers.
To address this, external content providers will provide their content on a KIS metadata cache, which will be hosted and managed by the company. For instance, Reuters will provide their content to a customer like Intel but Intel will host and manage the KIS. The Intel KIS would crawl the Reuters KIS (thereby chaining KIS servers) or the Reuters DSA. This way, sensitive Intel annotations can be published as 'Post-Its' using Reuters content as context while Intel will still maintain control over its sensitive data.
Category Naming for Federated Information Communities - e.g.. a Reuters information community (agency) deployed at Intel will be named Reυters(5 lntel with categories named like ' Reuters@lntel/lnformation Technology Wireless/80211'
Figure imgf000503_0001
The semantic browser will also allow users to anonymously annotate and publish to an information community (agency). In this mode, the metadata is completely stored (with the user identity) but is flagged indicating that the publisher wishes to remain anonymous. This way, the Inference Engine can infer using the complete metadata but requests for the publisher will not reveal his/her identity. The admin will also be able to configure the information community (agency) such that the inference engine cannot infer using anonymous annotations or publications.
Offline Support in th
The semantic browser will also have offline support. The browser will have a cache for every remote call. The cache will contain entries to XML data. This could be SRML or could be any other data that gets returned from a call to the XML Web Service. Each call is given a unique signature by the semantic browser and this signature is used to hash into the XML data. For instance, a semantic query is hashed by its SQML. Other remote calls are hashed using a combination of the method name, the argument names and types, and the argument data.
For every call to the XML Web Service, the semantic runtime client will extract the signature of the call and then map this to an entry in the local cache. If the browser (or the system) is currently offline, the client will return the XML data in the cache (if it exists). If it does not exist, the client will return an error to the caller (likely the Presenter). If the browser is online, the client will retrieve the XML data from the XML Web Service and update the cache by overwriting the previous contents of the file entry with a file path indicated by the signature hash. This assumes that the remote call actually goes through - it might not even if the system/browser is online, due to network traffic and other conditions. In such a case, the cache does not get overwritten (it only gets overwritten when there is new data; it does not" get cleared first).
^■.•S&fMi
<4afa Guarantee Cross-Platfor Support in the Semantic
Browser
Overview
As discussed in the main application, the Information Nervous System should be implemented in a cross-platform manner. Standard protocols should be employed where possible and the Web service layer should use interoperable Web service standards and avoid proprietary implementations. Essentially, the test is that the semantic browser does not have to "know" whether the Information Community (or agency) Web service it is talking to is running on a particular platform over another. For example, the semantic browser need not know whether the Web service it is talking to is running on Microsoft's .NET™ platform or Sun's J2EE platform (to take 2 examples of proprietary application servers). The Information Community Web service and the client-server protocol should employ Web service standards that are commonly supported by different Web service implementations like .NET™ and J2EE™.
In an ideal world, there will be a common set of standards that would be endorsed and properly implemented across Web service vendor implementations. However, this might not be the case in the real world. To handle a case where the semantic browser must handle unique functionality in different Web service implementations, the Information Community schema should be extended to include a field that indicates the Web service platform implementation. For instance, a .NET™ implementation of the Information Community should be published with a field that indicates that the platform is .NET™. The same applies to J2EE™. The semantic browser will then have access to this field when it retrieves the metadata for the Information Community (either directly via the WSDL URL to the Infonnation Community, or by receiving announcements via multicast, the enterprise directory (e.g., LDAP), the Global Infoπnation Community Directory, etc.).
The semantic browser can then issue platform-specific calls depending on the platform that the Information Community is running on1. This is not a recommended approach but if it is absolutely necessary to make platform-specific calls, this model should be employed in the preferred embodiment.
1 For example, there are some standards that have been endorsed by Microsoft, IBM, and other vendors but that have not been fully endorsed by Sun (e.g., WS-Security, WS-Routing, WS-Attachments, etc.) Knowledge Modeling
Knowledge Modeling refers to the recommended way enterprises will deploy an Information Nervous System. This involves deploying several KIS servers (per high-level knowledge domain) and one (or at most few) KDS (formerly KBS) servers that host the ontology and taxonomy. KIS servers should be deployed per domain to strike a balance between being too narrow such that there is not enough knowledge sharing possibility of navigation and inference in the network and being too high that scalability (in storage and CPU horsepower needed by the database and/or the inference engine) becomes a problem. In addition, KIS servers should be deployed where access control becomes necessary at the server level (for higher-level security) as opposed to imposing access control at the group level with multiple groups sharing the same KIS. For instance, a large pharmaceutical company could have an information community KIS for oncology for the entire company and another KIS for researchers working on cutting-edge R&D and applying for strategic patents. These two KIS server might crawl the same sources of information but the latter KIS would be more secure because it would have only users from the R&D group. Also, these researchers' publications and annotations will not be viewable on the corporate KIS.
Figure 5 below illustrates an example of a knowledge architecture for a pharmaceuticals company.
Knowledge Integration Knowledge Integration Server 1 (Oncology) Server 2 (Pharmacology)
Knowledge Domain Server (Pharmaceuticals)
Knowledge Integration Knowledge Integration Server 4 (Cardiology) Server 3 (Biotechnology)
Figure 5: Diagram Illustrating Knowledge Modeling for a Pharmaceuticals Company Example KIS Housekeeping Rules
The Knowledge Integration Server (KIS) will allow the admin to set up 'housekeeping' rules to purge old or stale metadata. This will prevent the SMS on the KIS from growing infinitely large. These rules could be as simple as purging any metadata older than a certain age (between 2-5 years depending on the company's policies for keeping old data) and which does not have any annotations and that is not marked as a favorite (or rated).
Figure imgf000506_0001
1 ) Shell: User implicitly creates a SQML query (i.e. an agent) via Ul navigation or a wizard.
2) Shell: User opens an agent (via tree or folder view)
3) The query buffer is saved as a file, and a registry entry created is created for the agent. ;, a) Registry entry contains: Agent Name, Creation date, Agent (Request)-GUID, SQML path, Comments, Namespace object type (agency, agent, blender, etc), and attributes
3) Shell: The request is handed off to the presenter: a) A registry request GUID entry is created containing (namespace path that generated the request, and SQML file URL). b) Browser is initialized and opened with command line http://PresenterPaqe.html#ReqυestGUID http://presenterpaqe.html/. The Presenter loads default Chrome contained in the page. c) Presenter page loads presenter binary behavior and Semantic Runtime OCX. 5) Presenter: Loads SQML and issues requests via the query manager. a) Resolves request GUID to get SQML file path. b) Loads SQML file into buffer, creates resource handler requests, passes them to resource handlers, waits for and gathers results. Summarization of local resources happens here. All summarization follows one of two paths: Summarize the doc indicated by this file path, or summarize this text (extracted from clipboard, Outlook, Exchange, etc.). Both paths produce a summary in the same form, suitable for inclusion in a request to the semantic server XML Web service. c) Compiles SQML file into individual server request buffers, including any resource summary from above. d) Initiates Server Requests by calling semantic runtime client Query Manager. 7) Query Manager: Monitors server requests and makes callback on data. It also signals an event on request completion or timeout. The callback is into the Presenter, which mean inter-process messaging to pass the XML.
8) Presenter: receives data and loads appropriate skin a) Receives SRML data in buffer; this will happen incrementally b) Determines if there is a preferred skin (smart style) associated with this agent, otherwise chooses default skin. c) Transforms SRML into preferred skin format via XSLT. This is multistage, for the tree of results (root is list, then objects, then Deep/Lens/BN info) as results come in. d) Display results in target DIV in page. The target is an argument to the behavior itself and is defined by the root page.
9) Presenter: Calls Semantic Runtime to fill context panels (per context template), deep info, smart copy and paste, and other semantic commands. The Presenter also loads the smart style, which then loads semantic images, motion, etc. consistent with the semantics of the request.
Figure 6 below illustrates the client component integration and interaction workflow.
Figure imgf000508_0001
Figure 6: Diagram Illustrating Client Component Integration and Interaction Workflow Peep IrH SpecHϊcatscri
Deep Information Specification for the Information Nervous System
By: Nosa Omoigui
Deep Infomiation
Deep Info Overview
Introduction
The Nervana 'Deep Info' tool is aimed at providing context-sensitive story-like information for a Nervana information object. Deep Info essentially provides Nervana users with information that otherwise would be lost, given a particular context. Deep Info is like the contextual information that gets displayed on music videos on MTV (showing information on the current artiste, the current song, and in some case, the current musical instrument in the song).
The 'deep' in 'deep info' refers to the fact that the contextual information will span multiple hops in the semantic network on the agency from whence the object came. 'Deep Info' consists of 'deep info nuggets' which can either be plan textual metadata or metadata with semantic query links (via SQML).
There are five kinds of Deep Info nuggets:
Basic Semantic Link Nuggets Context Template Nuggets Trivia Nuggets Matchmaker Nuggets Recursive Nuggets
Basic Semantic Link Nuggets
With basic semantic link truths, deep info nuggets merely convey a semantic link of the current object. These nuggets involve a semantic link distance of 1. In this case, there is overlap with what will be displayed in the 'Links' context/task pane.
Examples are:
Patrick Schmitz reports to Nosa Omoigui Patrick Schmitz has 5 Direct Reports Patrick Schmitz annotated 47 objects Patrick Schmitz authored 13 objects Patrick Schmitz was copied on 56 email objects Context Template Nuggets
Context template nuggets display contextual information for each relevant context template, based on the information at hand. These nuggets are identical to those that will be displayed in the context bar or context panel for each type of context template. For example:
Patrick Schmitz posted 3 breaking news items Patrick Schmitz posted 14 classics Patrick Schmitz authored 7 headlines Patrick Schmitz is involved in 13 discussions Patrick Schmitz is a newsmaker on 356 objects
Trivia Nuggets
For all email objects on an agency: o Steve Judkins appears on the to list of all of them o Steve Judkins replied to 23% of them
• Patrick Schmitz annotated 50% of them
• Only 3 of these have a thread depth greater than 2
For all people objects on an agency:
• Patrick Schmitz has sent email to 47% of them o 14% of them report to Nosa Omoigui o Lisa Heilbrαn has had discussions with 85% of them o 12% of them are newsmakers on at least one topic
• All of them have been involved in at least one discussion this week
• 33% of them are experts on at least one topic
• 8% of them are experts on more than three topics
For a given distribution list on an agency:
• Steven Judkins has posted the most email to this list
• Lisa Heibron has replied to the most email on this list
• Nosa Omoigui has never posted to this list
• Patrick Schmitz has posted 87 messages to this list this month
• Richard Novotny has posted 345 messages to this list this year
For all distribution lists on an agency:
• Steven Judkins has posted the most email to all lists
• Lisa Heibron has replied to email on only 2% of the lists
• Nosa Omoigui has never posted to any list
• Patrick Schmitz has posted at least once every week to all the lists
• Richard Novotny has posted messages on 3 lists
For all information objects on an agency:
• Steven Judkins has been the most prolific publisher (he published 5% of them)
• Lisa Heilbron has been the most prolific annotator (she annotated 2% of them)
• Nosa Omoigui has been the most active newsmaker
• Patrick Schmitz has the most aggregate expertise Steve Judkins has the most expertise for infoπnation published this year Gavin Schmitz has been involved in the most discussions (12% of them) Richard Novotny has been involved in the most discussions this month (18% of them)
Matchmaker Nuggets
Person to Person
Semantic Link Based
Patrick Schmitz has sent mail to 13 people
47 people have appeared on same To list as Patrick Schmitz
47 people have appeared on same CC list as Patrick Schmitz
89 people in total have been referenced on email sent by Patrick Schmitz
24 people have annotated the same information as Patrick Schmitz
3 people are on all the same distribution lists as Patrick Schmitz
29 people are on at least one of Patrick Schmitz's distribution lists
Context-Template Based
• 12 people have expertise on the same information categories as Patrick Schmitz
• 14 people and Patrick Schmitz are newsmakers on the same information items
• 27 people are in discussions with Patrick Schmitz
Information to Person
Semantic Link Based o Patrick Schmitz posted this information item
• Steve Judkins authored this information item
• This information item was copied to 2 people
• 3 people annotated this information item
Context Template Based (similar to context template nuggets)
• There are 4 experts on this information item
• There are 27 newsmakers on this information item
Information to Infoπnation
Context Template Based (similar to context template nuggets)
• There are 578 relevant 'all bets'
• There are 235 relevant 'best bets'
• There are 4 relevant breaking news items
• There are 46 relevant headlines
Semantic Link Based (via people)
There are 21 information items that have the same experts with this one There are 23 information items that have the same newsmakers with this one There are 34 information items posted by the same person that posted this one There are 34 information items authored by the same person that authored this one' There are 44 information items annotated by people that annotated this one Recursive Nuggets
With recursive nuggets, displaying deep info on the subject of the current information nugget forms a contextual hierarchy. The system then recursively displays the nuggets based on the object type of the subject. With recursive nuggets, the system essentially probes the semantic network starting from the source object and continues to display nuggets along the path of the network. It is TBD at what depth to stop the probing. The following is an example:
Patrick Schmitz authored this email
Patrick Schmitz reports to Nosa Omoigui
Nosa Omoigui has 6 Direct Reports Steve Judkins ... Steve Judkins posted ... Steve Judkins is an expert on ... Steve Judkins is a newsmaker on ... Steve Judkins has been involved in 6 discussions [etc.]
Richard Novotny... [The remaining 6 direct reports] Nosa Omoigui annotated 13 objects...
[More context template nuggets on the 13 objects] Nosa Omoigui has authored 278 objects
Nosa Omoigui has annotated 23 items [• ..]
Patrick Schmitz has 5 Direct Reports John Doe ...
More Native Nuggets based on the direct reports Patrick Schmitz annotated 47 objects
Figure imgf000514_0001
Security Specification for the Infonmation Nervous System
By: Nosa Omoigui
ESEHS BTI
Authorization Overview
Introduction
The 'People' DSA will be initialized with an LDAP Directory URL and Group Name. The 'Users' DSA will also be initialized with an LDAP Directory URL and Group Name. Typically, the 'Users' will be a subset of 'People.' For instance, a pharmaceuticals corporation might install a KIS for different large pharmaceutical categories (e.g., Biotechnology, Life Sciences, Pharmacology, etc). Each of these will have a group of users that are knowledgeable or interested in that category. However, the KIS will also have the 'People' group populated with all employees of the corporation. This will enable users of the KIS to navigate to members of the entire employee population even though those members are not users of the KIS. In addition, the inference engine will be able to infer expertise with semantic links off people that are in the corporation, not necessarily just users of the KIS.
This is also important for access control at the KIS level. The Users group will contain people that have access to the KIS knowledge. However, the People group will contain people that are relevant to the KIS knowledge, even though those people don't have access to the KIS.
Both People and Users DSA populate the People table in the Semantic Metadata Store (SMS) and indicate the object type id appropriately. Note that the passwords are NOT stored in the People table in the SMS.
The Users DSA also populates the User Authentication Table (UAT). This is an in-memory hash table that maps the user names to passwords. The server's Web service will implement the IPasswordProvider interface or an equivalent.1 The implementation of the PasswordProvider object will return the password that maps to a particular user name. The C# example below illustrates this: namespace SDK_Security
{ public class PasswordProvider :
Microsof .WSDK. Security. IpasswordProvider
{ public string GetPass ord( string username )
{ return "opensezme" ;
}
}
The following C# code shows how the Web service can retrieve the user infoπnation after the user has been authenticated:
1 See http://ms n.microsoft com/iibrary/default asp?uri=/libr3iΥ/en-us dnwsseour/hrjτιlfwssecwithwsdk. usάng~ System; using System. Collections ; using System. ComponentModel; using System. Data; using System. Diagnostics ; using System.Web; using System.Web. Services; using Microsoft. WSDK. Security; using Microsoft .WSDK; namespace SDK_Security
{ public class Servicel : System. eb. Services .WebService
{
[WebMethod] public string PersonalHello ( )
{ string response = " " ;
SoapContext requestContext = HttpSoapContext . RequestContext ; if (requestContext == null )
{ throw new ApplicationException("Non-SOAP request.");
} foreach (SecurityToken tok in requestContext. Security. Tokens)
{ if (tok is UsernameToken)
{ response += "Hello " + ( (UsernameToken) tok) .Username;
} } return response;
} }
The Nervana Web service can then go ahead and call the Server Semantic Runtime with the calling user name. The runtime then maps this to SQL and uses the appropriate filters to issue the semantic query.
For the Nervana ASP.NET application, the following entry is added as a child of the parent configuration element in the Web.config file:
<microsof . wsdk> :εecurity>
<passwordProvider type="WSDK_Security .PasswordProvider, WSDK-Security" /> ./security> </microsoft . sdk> Client-Side Authorization Request
In order to create a UsernameToken for the request, the Nervana client has to pass the usemame and password as part of the SOAP request. The Nervana client can pass multiple tokens as part of the request - this is critical for cases where the user's identity is federated across multiple authentication providers. The Nervana client will gather all the user account information the user has supplied (including user name and password information), convert these to WS-Security tokens, and then issue the SOAP request. The client code will look like the following (reference: http ://www. msdn .m icrosoft.com ): localhost . Servicel proxy = new lσcalhost . Servicel ( ) ; UsernameToken clearTextToken
= new UsernameToken ( "Joe" ,
"opensezme " ,
PasswordOption. SendHashed) ; proxy . Reques SoapCon text . Security . Tokens .Add (clearTextToken) ; labell . Text = proxy . PersonalHello 0 ;
Validating the UsernameToken on the Server
(http://msdn.microsoft.com librarv/default.asp?url=/librarv/en-us/dnwssecur/html/wssecwithwsdk.asp)
Although the WSDK verifies the Security header syntax and checks the password hash against the password from the Password Provider, there is some extra verification that must be performed on the request. For instance, the WSDK will not call the Password Provider if a UsernameToken is received that does not include a password element. If there is no password to check, there is no reason to call the password provider. This means we need to verify the format of the UsernameToken ourselves.
Another possibility is that there is more than one UsernameToken element included with the request. WS-Security provides support for including any number of tokens with a request that may be used for different purposes.
The code above should be modified for the Nervana Web method to verify that the UsernameToken includes a hashed password and to only accept incoming requests with a single UsernameToken. The modified code is listed below.
[WebMethod] public string ProcessSemanticQuery ( string Query )
{
SoapContext requestContext = HttpSoapContext . RequestContext ; if (requestContext == null)
{ throw new ApplicationException ( "Non-SOAP request . " ) ; } if (requestContext . Security . Tokens . Count == 1 )
{ foreach (SecurityToken tok in requestContext . Security.Tokens)
{ if (tok is UsernameToken)
{
UsernameToken UserToken = (UsernameToken) tok; if (UserToken. PasswordOption
== PasswordOption. SendHashed)
{ ' return ProcessSemanticQuerylnternal ( Query, UserToken . Usemame ) ;
} else
• { throw new SoapException(
"Invalid UsernameToken password type.", SoapException.ClientFaultCode) ;
}
} else
{ throw new SoapException(
"UsernameToken security token required.", SoapException.ClientFaultCode) ;
}
}
} else
{ throw new SoapException(
"Request must have exactly one security token.",
SoapException.ClientFaultCode) ;
} return null;
People Groups
The KIS will include metadata for people groups. These are not unlike user groups in modern operating systems. The People Group will be a Nervana first-class object (i.e., it will inherit from the Object class). In addition, the People Group schema will be as follows:
Figure imgf000520_0001
In most cases, people groups will map to user groups in directory systems (like LDAP). For instance, the KIS server admin will have the KIS crawl a configurable set of user groups. There will be a People DSA that will crawl the user groups and populate the People Groups and Users tables in the SMS. The People DSA will perform the following actions:
• Create the group (if it doesn't exist in the SMS) or update the metadata of the Group (if it exists)
• Enumerate all the users in the group (at the source - an LDAP directory in the preferred embodiment)
• For all the users in the group, create People objects (or update the metadata if the objects already exist in the SMS)
• Update the semantic network (via the 'SemanticLinks' table in the SMS) by mapping the people objects to the group objects (using the BELONGS_TO_GROUP semantic link type). This ensures that the SMS has semantic links that capture group membership information (in addition to the groups and users themselves). Identity Metadata Federation
Identity Metadata Federation (IMF) refers to a feature wherein an Information Community (agency) is deployed over the Internet but is used to service corporate or personal customers. For instance, Reuters could set up an information community for all its corporate customers that depend on its proprietary content. In such a case where multiple customers share an information community (likely in the same industry), Reuters will have a group on the SMS for each customer. However, each of these customers would have to have its corporate directory mirrored on Reuters in order for people metadata to be available. This would cause problems, particularly from a security and privacy standpoint. Corporation will not be comfortable with having external content providers obtaining access to the metadata of their employees. IMF addresses this problem by having the Internet-hosted information community (agency) host only enough metadata for authentication of the user. For instance, Reuters will store only the logon information for the users of its corporate customers in its SMS. When the semantic browser receives SRML containing such incomplete metadata, the client will then issue another query to the enterprise directory (via LDAP access or via UDDI if the enterprise directory metadata is made available through a Web services directory) to fetch the complete metadata of the user. This is possible because the externally stored metadata will have the identity information with which the remaining metadata can be fetched. Since the client fetches the remaining metadata within the firewall of the enterprise, the sensitive corporate metadata is not shared with the outside world.
Figure imgf000521_0001
Access Control Policy
The KIS will include and enforce access control semantics. The KIS employs a policy of "default access." By this, I mean that the KIS will grant access to the calling user to any metadata in the SMS, except in cases where access is denied. As such, the system can be extended to provide new forms of denial, as opposed to new forms of access. In addition, this implies that if there is no basis for denial, the user is granted access (this leads to a simpler and cleaner access control model).
The KIS will have an Access Control Manager (ACM). The ACM is primary responsible for generating a Denial Semantic Query (DSQ) which the SQP will append to its query for a given semantic request from the client. The ACM will expose the following method (C# sample):
String GetDenialSemanticQuery( String CallingUserName ) The method takes in the calling user name and returns a SQL query (or equivalent) that encapsulates exception objects. These are objects that must not be returned to the calling user by the SQP (i.e., objects for which the user does not have access).
The SQP then builds a final raw query that includes the denial query as follows:
Aggregate Raw Query AND NOT IN (Denial Query)
For example, if the aggregate raw query is:
SELECT OBJECTID FROM OBJECTS WHERE OBJECTTYPEID = 5, and the denial query is:
SELECT OBJECTID FROM OBJECTS WHERE OWNERUSERNAME <> 'JOHNDOE',2
The final raw query (which is that the SQP will finally execute and serialize to SRML to return to the calling user) will be:
SELECT OBJECTID FROM OBJECTS WHERE OBJECTTYPEID = 5 AND NOT IN
(SELECT OBJECTID FROM OBJECTS WHERE OWNERUSERNAME <> 'JOHNDOE')
Semantically, this is equivalent to:
"Select all objects that have an obiect type id of 5 but that are not in an obiect list not owned by John Doe."
This in turn is semantically equivalent to:
"Select all objects that have an object type id of 5 that are owned by John Doe. "
General Access Control Rules
Each semantic query processed by the semantic query processor (SQP) will contain an access control check. This will guarantee that the calling user only receives metadata that he/she has access to. The SQP will employ the following access control rules when processing a semantic query:
1. • If . the query is for 'People' objects (people, users, customers, experts,' newsmakers, etc.), the returned 'People' objects must either: ■;
• • Include the calling user, or
• Include people that share at least one people group with the calling user, and be owned by the calling user or the system
The corresponding denial query maps to the following rule: The returned objects must satisfy the following:
2 The <> operator corresponds to "Not Equal to" in SQL • Is not the calling user +
• Is not owned by the calling user or the system +
• Has people that do not share any people group with the calling user
Sample Denial Query SQL
The SQL below illustrates the access control denial query that will be generated by the ACM and appended by the SQP to enforce the access control policy. In this example, the name of the calling user is 'JOHNDOE.'
SELECT OBJECTID FROM OBJECTS WHERE
OWNERUSERNAME <> 'JOHNDOE' OR
OWNERUSERNAME <> 'SYSTEM'OR
WHERE OBJECTID NOT IN (SELECT OBJECTID FROM PEOPLE WHERE NAME='JOHNDOE) OR
WHERE OBJECTID NOT IN
( SELECT OBJECTID FROM SEMANTICLINKS WHERE
OBJECTTYPEID = "PERSON AND PREDICATETYPEID='BELONGS_TO_GROUP,AND SUBJECTID IN (SELECT SUBJECTID FROM SEMANTICLINKS WHERE OBJECTID IN (SELECT OBJECTID FROM PEOPLE WHERE NAME='JOHNDOE' ) )
2. e query is for non-People objects (documents, email, events, etc.), the returned objects must:
• Be owned by the calling user or the system user, and
• Be the subject of a semantic link with the calling user as the object, or
• Be the object of a semantic link with the calling user as the subject, or
• Be the subject of a semantic link with the object being a person that shares at least one people group with the calling user, or
• Be the object of a semantic link with the subject being a person that shares at least one people group with the calling user
I
The corresponding denial query maps to the following rule: The returned objects must satisfy the following:
• Is not owned by the calling user +
• Is not owned by the system user + .
• Is not the subject of a semantic link with the calling user as the object +
• Is not the object of a semantic link with the calling user as the subject + • Is not the subject of a semantic link with the object being a person that shares at least one people group with the calling user +
• Is not the object of a semantic link with the subject being a person that shares at least one people group with the calling user
Sample Denial Query SQL
The SQL below illustrates the access control denial query that will be generated by the ACM and appended by the SQP to enforce the access control policy. In this example, the name of the calling user is 'JOHNDOE.'
SELECT OBJECTID FROM OBJECTS WHERE OWNERUSERNAME <> 'JOHNDOE' OR
OWNERUSERNAME <> 'SYSTEM' OR
OBJECTID NOT IN (SELECT OBJECTID FROM SEMANTICLINKS WHERE
OBJECTTYPEID = "PERSON'AND OBJECTID IN (SELECT OBJECTID FROM PEOPLE WHERE NAME='JOHNDOE') OR
WHERE OBJECTID NOT IN (SELECT OBJECTID FROM SEMANTICLINKS INNER JOIN PEOPLE WHERE SEMANTICLINKS.SUBJECTTYPEID='PERSON'AND SEMANTICLINKS.SUBJECTID = PEOPLE.OBJECTID) OR
OBJECTID NOT IN (SELECT OBJECTID FROM SEMANTICLINKS WHERE OBJECTTYPEID='PERSON'AND PREDICATETYPEID='BELONGS_TO_GROUP'AND SUBJECTID IN (SELECT SUBJECTID FROM SEMANTICLINKS WHERE OBJECTID IN (SELECT OBJECTID FROM PEOPLE WHERE NAME='JOHNDOE)) OR
OBJECTID NOT IN (SELECT OBJECTID FROM SEMANTICLINKS WHERE OBJECTTYPEID='PERSON'AND PREDICATETYPEID='BELONGS_TO_GROUP' AND OBJECTID IN (SELECT OBJECTID FROM PEOPLE WHERE NAME='JOHNDOE'))
Sample Merged Denial Query SQL
By merging these two rules, the ACM returns the following merged query to the SQP for access denial:
SELECT OBJECTID FROM OBJECTS WHERE
OWNERUSERNAME <> 'JOHNDOE' OR
OWNERUSERNAME <> 'SYSTEM' OR
OBJECTID NOT IN (SELECT OBJECTID FROM PEOPLE WHERE NAME='JOHNDOE') OR
OBJECTID NOT IN (SELECT OBJECTID FROM SEMANTICLINKS WHERE OBJECTTYPEID = "PERSON AND PREDICATETYPEID='BELONGS_TO_GROUP'AND SUBJECTID IN (SELECT SUBJECTID FROM SEMANTICLINKS WHERE OBJECTID IN (SELECT OBJECTID FROM PEOPLE WHERE NAME='JOHNDOE)) OR
OBJECTID NOT IN (SELECT OBJECTID FROM SEMANTICLINKS WHERE OBJECTTYPEID = "PERSON' AND OBJECTID IN (SELECT OBJECTID FROM PEOPLE WHERE NAME='JOHNDOE') OR
OBJECTID NOT IN (SELECT OBJECTID FROM SEMANTICLINKS INNER JOIN PEOPLE ON SEMANTICUNKS.SUBJECTTYPEID=PERSON'AND SEMANTICLINKS.SUBJECTID ■ PEOPLE.OBJECTID) OR
OBJECTID NOT IN (SELECT OBJECTID FROM SEMANTICLINKS WHERE OBJECTTYPEID='PERSON' AND PREDICATETYPEID='BELONGS_TO_GROUP'AND SUBJECTID IN (SELECT SUBJECTID FROM SEMANTICLINKS WHERE OBJECTID IN (SELECT OBJECTID FROM PEOPLE WHERE NAME='JOHNDOE)) OR
OBJECTID NOT IN (SELECT OBJECTID FROM SEMANTICLINKS WHERE OBJECTTYPEID='PERSON' AND PREDICATETYPEID='BELONGS_TO_GROUP'AND OBJECTID IN (SELECT OBJECTID FROM PEOPLE WHERE NAME='JOHNDOE))
Example Scenario
For instance, A Reuters agency (KIS) might have people groups for each enterprise customer that Reuters serves. The agency will have a common information base (Reuters content) but will have people groups per enterprise customer. These groups might include competitors. As such, it is critical to ensure that the knowledge flow, generation, and inference do not cross competitor boundaries. For instance, an employee of Firm A must not derive knowledge directly from an employee of Firm B that competes with Firm A, not must he or she derive knowledge indirectly (via inference). An employee of Firm A must not be able to get recommendations for items annotated .by employees of Firm B. Or an employee of Firm A must not be able to find experts that work for Firm B. Of course, this assumes that Firm A and Firm B are not partners in some fashion (in which case, they might want to share knowledge). In the case of knowledge partners, Reuters would create a people group (likely via LDAP) that includes the people groups of Firm A and Firm B. The Reuters KIS will then have the following people groups: Firm A, Firm B, and Firms A&B. The SMS will also include metadata that indicates that the people in Firms A and Firms B belong to these groups (via the "belongs to group" semantic link type). With this process in place, the aforementioned rules will guarantee that knowledge gets shared between Firms A and B.
Access Control Rules for Annotations
In the case of annotations, the calling user will be editing the semantic network, as opposed to querying it. In this case, the following rules would apply:
1. If the object being annotated is a Person object, the object must either be:
• The calling user, or • A person that shares at least one people group with the calling user, and be owned by the calling user or the system
2. if the object being annotated is a non-Person object (e.g., a document, email, event, etc.), the object must either be:
• Owned by the calling user
• Owned by the system
Sample Denial Query SQL
The SQL below illustrates the access control denial query that will be generated by the ACM (for checking access control for annotations) and appended by the SQP to enforce the access ■ control policy. In this example, the name of the calling user is 'JOHNDOE.'
SELECT OBJECTID FROM OBJECTS WHERE
OWNERUSERNAME <> 'JOHNDOE' OR
OWNERUSERNAME <> 'SYSTEM' OR
OBJECTID NOT IN (SELECT OBJECTID FROM PEOPLE WHERE NAME='JOHNDOE') OR
OBJECTID NOT IN (SELECT OBJECTID FROM SEMANTICLINKS WHERE OBJECTTYPEID^ PERSON' AND PREDICATETYPEID='BELONGS_TO_GROUP' AND OBJECTID IN (SELECT OBJECTID FROM SEMANTICLINKS WHERE OBJECTID IN (SELECT OBJECTID FROM PEOPLE WHERE NAME='JOHNDOE'))
Access Control Enforcement
The ACM enforces access control for annotations and other write operations on the KIS. The KIS XML Web Service exposes an annotation method as follows (C# sample):
AnnotateObject( String CallingUserName, String ObjectlD);
This method calls the ACM to get the denial query. It then creates a final query as follows:
Annotation Object Query AND NOT IN (Denial Query)
The annotation object query is always of the form:
SELECT OBJECTID FROM OBJECTS WHERE OBJECTID=ObjectlD, where ObjectlD is the argument to the AnnotateObject method.
The ACM then builds a final access control query SQL and uses this SQL to check for access control. Because the ACM does not have to return the SQL, it merely invokes it directly in order to check for access control. In addition, because it is a binary check (access or no access), the ACM merely checks whether the denial query returns at least one row. For instance, a final query might look like:
SELECT OBJECTID FROM OBJECTS WHERE OBJECTID = ObjectlD AND NOT IN (SELECT OBJECTID FROM OBJECTS WHERE OWNERUSERNAME <> 'JOHNDOE')
The ACM then runs this query (via the SQL query processor) and asks for the count of the number of rows in the result set. If there is one row, access is granted, else access is denied. This model is implemented this way in order to have consistency with the denial query model (the ACM always builds a denial query and uses this as a basis for all access control checks).
Semantic Definitions and Visualizations Specification
Specification for Semantic Query Definitions and Visualizations for the Information Nervous
System
By: Nosa Omoigui
Semantic Images and
Overview
Semantic images and motion are a critical part of the Nervana semantic user experience. By this, I mean having image/motion metadata stored on a Nervana agency (information community) and accessed via the Nervana XML Web Service. Via Nervana, end users will have context and time-sensitive semantic access to their images. Imagine using a Getty Images (or Corbis) agent as a smart lens over an email message - when invoked, this will open images that are semantically related to the message. Or dragging and dropping a document from your hard drive to a Getty agent to view semantically related images. This will involve having image metadata (consistent with an image schema). The Nervana toolbox will remain the same - we will merely add a new infonmation object type for images. Also, there will be semantic skins for semantic images - different views, thumbnails, slide shows, filtering, aggregation, etc. For examples of semantic images, visit: http://creative.qettyimaqes.com/source/search/resultsmain. asp?source=advSearch&hdnSync=Medicine%7E0%2C12%2C 449%2C3%2C15%2C1%2C0%2C0%2C0%2C12287%2CQ%2C7%2C14%2C6%2C3%2C3%2C0%2C12%2C449%2Cen %2Dus&UQR=tfxfwz
Figure imgf000529_0001
Industry-specific semantic images/motion are images/motion that will be used as part of the presentation atmosphere for semantic results for one or more categories (that map to industries). For instance, visit http://www.corbis.com and http://www.qettvimaqes.com and enter a search for the following keywords (which, in aggregate, map to target industries, based on the Reuters taxonomy) - see below. These images/motion will be used as backgrounds, filter effects, transforms, and animations for context and category skins (that map to context templates and categories). In addition, these images/motion will be used for visuals for motion paths extracted from some of these images for super exciting screensavers. For example, imagine a skin displaying metadata and visualizations along a motion path extracted from one of these semantic images (e.g., metadata rotating inside a light bulb - for the "electric utilities" industry), along with chrome with other surrounding images and animations...
Pharmaceuticals Medicine Healthcare Life Sciences Biotechnology Oil and gas Chemical Energy Electric Utilities Gas Utilities Water Utilities Entertainment Environmental Services
Publishing
Real Estate
Financial
Brokerages
Financial Services
Banking
Consumer
Consumer Products
Consumer Services
Communications
Telecommunications
Telecom Equipment
Telecom Services
Telecom Technology
Telecom Regulations
Tobacco
Automotive
Automobiles
Insurance
Consulting
Information Technology
Technology
Computer Equipment
Computer Manufacturers
Computing
Semiconductors
Nanotechnology
Public Sector
Government
Homeland Security
Travel
Tourism
Transportation
Airlines
Retail
Fashion
Advertising
Aerospace
Defense
Agribusiness
Agriculture
Beverages
Business services
E-commerce
Food
Forest products
Health Care Providers
Hospitality
Internet
Law
Legal
Manufacturing
Marketing
Media • Networking
The same applies to information types and context templates. Skins should do the smart thing based on the context/information type and the category/ontology and mix and match semantic images/motion across these properties in an intelligent manner. For instance, an agent titled "Headlines on Wireless Technology" can have chrome (and/or a smart hourglass - see below) that shows an image/motion-based animation toggling between a "Headlines" image/motion and a "Wireless" image/motion. A blender titled "Headlines on Wireless and Breaking News on Semiconductors and Email by anyone in my group related to the product specification" can have chrome (and/or a smart hourglass) that "toggles" between images/motion for "Headlines," "News," "Wireless," "Semiconductors," and "Email."
The Presenter's query processor can enumerate all context template and information types and all categories (from the agent blender SQML) and set up the chrome animation accordingly.
For information types, enter searches (on Corbis and Getty) for:
• Documents
• Email
• Books
• Magazines
• Multimedia
• Online Learning
• People » Users o Customers
Also, for context templates, enter searches for:
• Headlines
• News
• Discovery o Conversations o Experts o Favorites o Places
• Time (for "timeline" and "upcoming events")
• Schedule
• Appointment
Also, note that semantic images/motion should not be completely random. However, they should not be from a bounded set either. They should be carefully picked and then skins can randomly select from the chosen set. But they should not be random from the entire set on, say, Corbis or Getty Images. Else there will be silly images, cartoons, and some potentially offensive images. Also, some of these guidelines should vary depending on whether the skin theme is in subtle, moderate, exciting, or super-exciting mode. In subtle mode, the skin might decide to choose one image/motion per visualization pivot. In other modes, this would likely lead to a boring user experience.
In low-flashiness mode, the skin can use a semantic image/motion as part of the chrome - not unlike a PowerPoint slide- deck background (e.g., alpha blended). Semantic images/motion can also be used in the smart hourglass (see below) as well as in part of the visualization (on the context bar, panel, or palette). For visualizing context and information types, semantic images/motion should be carefully picked to clearly indicate the information type or context. In addition, the • selection mode can also be a skin property.
Also, the number of possible semantic images/motion used per skin would likely need to be capped - depending on where the images/motion are being displayed. However, in some scenarios, this might not be necessary. For instance, a blender skin might cycle between chrome backgrounds as the user navigates the blender results (from page to page or agent to agent) - to be consistent with what is currently being displayed from the blender. This can also be a skin property. The Client-Side Semantic Image and Motion Cache
The Presenter will have a smart expandable client-side cache with semantic images and motions that are downloaded and stored on the client (on installation). Skins can then select from these pre-cached images and motions. The images/motions can be pre-cached based on the user's favorite categories and areas of interest (which he or she selects) - which map to target industries. Skins can then complement the pre-cached semantic images/motions with on-demand image queries to an image server (an XML Web Service that exposes server-side images/motions - hosted by Nervana or a third party like Corbis or Getty Images).
The Presenter will also do the smart thing and have a bias function such that recently downloaded images/motions are selected before older ones (as a tiebreaker). A "usage count" is also be cached along with each image/motion - the Presenter uses this count in filtering which images/motions to display and when. Such "load balancing" will yield a fresher and non-repetitive user experience.
The cache should be populated on demand (based on the user's semantic queries) - for instance, there is no point in pre- caching pharmaceutical images/motions for a user's machine at Boeing. The cache size also should be capped and the image cache manager should purge "old" and "unused" images using an LRU algorithm or the equivalent. This way, the cache can be in "semantic sync" with the user's agent usage pattern and favorite agent's list.
The Smart Hourglass
Majority of the calls that the Nervana Presenter will make to provide the "semantic user experience" will be remote calls to the XML Web Service. As such, there will be unpredictable, potentially unbounded delays in the Ul. One can expect a fair amount of bandwidth and server horsepower within the enterprise but the Nervana user interface must still "plan" for unknown latency in method invocations.
Operating systems today have this problem with unbounded I/O calls to disk or to the network. Some CPU-bound operations also have substantial delays. In the Windows and Mac Ul, the user is made to perceive delay via a "wait" cursor - usually in the shape of an "hourglass."
In Nervana, the Presenter will have semantic hints (via direct access to the SQML "method call") with which it can display the equivalent of a "smart or semantic hourglass." This could be in the form of an intermediate page that displays "Loading" or some other effect. More importantly, the Presenter can convey the semantics of the query by reading the SQML to get hints on the categories that the query represents and the information type or context template. The Presenter can then use these hints to display semantic images and text consistent with the query, even though it has not received the results. The more hints the query has, the smarter the hourglass can get. The "Loading" page can then convey the atmosphere of "what is to come" - even before the actual results arrive from the Web service and are merged (if necessary) by the Presenter to yield the final results.
This "smart hourglass" can be displayed not just on the main results pane, but perhaps also on smart lens balloon popup windows and inline preview windows (essentially at every call site to the Web service and where there is "focus"). The Presenter can do the smart thing by timing out on the query (perhaps after several hundred milliseconds - the implementation should use usability tests to arrive at a figure for this) before displaying the "hourglass." Visualizations - Context Templates
Overview
Introduction
Context templates are scenario-driven information query templates that map to specific semantic models for information access and retrieval. Essentially, context templates can be thought of as personal, digital semantic information retrieval "channels" that deliver information to a user by employing a predefined semantic template. Context templates preferably aggregate information across one or more Agencies.
The context templates described below have been defined. Additional context templates, directed towards the integration and dissemination of varied types of semantic information, are contemplated (examples include context templates related to emotion, e.g., "Angry," "Sad," etc.; context templates for location, mobility, ambient conditions, users tasks, etc.).
Breaking News
The Breaking News context template can be analogized to a personal, digital version of CNN's "Breaking News" program insert in how it conveys semantic information. The context template allows a user to access information that is extremely time-critical from one or more Agencies, sorted according ' to the information creation or publishing time and a configurable amount of time that defines information criticality.
Figure imgf000534_0001
Sample Object and Context Bar Visualizations
1. Ticking clock showing publication or scheduled time of most recent or pending breaking news item over a background of the total number of upcoming breaking news items
2. Ticking clock showing publication or scheduled time of most recent or pending breaking news item over semantic image(s)
3. Ticking clock showing publication or scheduled time of most recent or pending breaking news item over semantic image(s) and the total number of breaking news items
4. Ticking clock showing publication or scheduled time of most recent or pending breaking news item over a plain background
5. Non-ticking clocks showing publication or scheduled time of all breaking news items (sequentially) over various backgrounds
6. Calendar view showing publication or scheduled time of most recent or pending breaking news item over various backgrounds
7. Calendar view showing publication or scheduled time of all breaking news items (sequentially) over various backgrounds
8. Scaled font size - depending on the publication or scheduled time of the most recent or pending breaking news item
9. Scaled font size - depending on the number of breaking news items
10. Animated font (e.g., flashing text, rotating text, text on motion path, etc.) with animation rate depending on the publication or scheduled time of the most recent or pending breaking news item
11. Animated font (e.g., flashing text, rotating text, text on motion path, etc.) with animation rate depending on the number of breaking news items
12. Varying font color - depending on the publication or scheduled time of the most recent or pending breaking news item
13. Varying font color - depending on the number of breaking news items
14. Animated graphic of breaking news semantic image(s) or an equivalent
15. Number of breaking news items
16. Titles of breaking news items animated in a sequence (list view)
17. Titles and details of breaking news items animated in a sequence (tiled view)
18. Semantic image/motion moving on an orbital motion path around the object
19. Balloon popup showing number of items on semantic image/motion background
20. Balloon popup showing number of items with plain background but animated with semantic image/motion
Headlines
The Headlines context template can be analogized to a personal, digital version of CNN's "Headline News" program in how it conveys semantic information. The context template allows a user to access information headlines from one or more Agencies, sorted according to the information creation or publishing time and a configurable amount of time or number of items that defines information "freshness." For example, CNN's "Headline News" displays headlines every 30 minutes (around the clock). In a preferred embodiment, the Headlines context template will be implemented as a SQL query on the server with the following sub queries chained in sequence: Recommendations Published Today, Favorites Published Today, Best Bets Published Today, Upcoming Events Occurring Today and Tomorrow, Annotated Items Published Today.
All sub queries will be sorted by the publishing date/time and then be chained together. Additional filters will be applied to the query based on the predicate list in the SQML.
Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
Figure imgf000536_0001
Conversations
The Conversations context template can be analogized to a personal, digital version of CNN's "Crossfire" program in how it conveys semantic information. Like "Crossfire," which uses Conversations and debates as the context for information dissemination, in the preferred embodiment, the Conversations context template tracks email postings, annotations, and threads for relevant information.
The Conversations context template consists of the following infonnation object types:
1. Email of a thread depth of at least one (An email reply to an email message)
2. Annotations of a thread depth of at least one (The annotation of an annotation of an object)
3. Internet News Postings (A news posting reply to a news posting)
The query will be sorted by thread depth. Additional filters will be applied to the query based on the predicate list in the SQML. In addition, the context skin should display the information items by thread.
Visualization- Sample Hrnage for smart hourglass, interstitial page, transition effects, rbacl gromnd chrome, etc.
Figure imgf000537_0001
Sample Object and Context Bar Visualizations
1. Animated graphic of semantic image/motion(s) (icon and context guide view)
2. Maximum thread depth over plain background (icon and context guide view)
3. Maximum thread depth over semantic image/motion (icon and context guide view)
4. Titles of conversations animated in a sequence (list view)
5. Titles and details of conversations animated in a sequence (tiled view)
6. The number of conversations over a plain background (icon and context guide view)
7. The number of conversations over semantic image/motion(s) (icon and context guide view)
Newsmakers
The Newsmakers context template can be analogized to a personal, digital version of NBC's "Meet the Press" program in how it conveys semantic information In this case, the emphasis is on "people in the news," as opposed to the news itself or Conversations. Users navigate the network using the returned people as Information Object Pivots. The Newsmakers context template can be thought of as the Headlines context template, preferably with the "People" or "Users" object type filters, and the "aujhored by," "possibly authored by," "hosted by," "annotated by," "expert on," etc predicates (predicates that relate people to information) The "relevant to" default predicate preferably is used to cover all the germane specific predicates The sort order of the relevant information, e g , the newsmakers, is sorted based on the order of the "news they make," e g , headlines.
The query will be sorted by number of headlines. Additional filters will be applied to the query based on the predicate list in the SQML.
Visualisation - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
Figure imgf000539_0001
Sample Object and Context Bar Visualizations
1. Animated graphic of 2 talking heads in conversation (icon and context guide view)
2. Animated graphic of semantic image/motion(s) (icon and context guide view)
3. Total number of newsmakers (icon and context guide view)
4. Total number of newsmakers over semantic image/motion (icon and context guide view)
5. Names of newsmakers animated in a sequence (list view)
6. Names and details of newsmakers animated in a sequence (tiled view)
Upcoming Events
The Upcoming Events context template (and its resulting Special Agent) can be analogized to a personal digital version of special programs that convey information about upcoming events. Examples include specials for events such as "The World Series," "The NBA Finals," "The Soccer World Cup Finals," etc. The equivalent in a knowledge-worker scenario is a user that wants to monitor all upcoming industry events that relate to one or more categories, documents or other Information Object Pivots.' The Upcoming Events context template is preferably identical to the Headlines context template except that only upcoming events are filtered and displayed (preferably using a semantically appropriate "context Skin" that connotes events and time-criticality). Returned objects are preferably sorted based on time-criticality with the most impending events listed first.
Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
Figure imgf000541_0001
Sample Object and Context Bar Visualizations
I . Ticking clock showing time till next event over a background of the total number of upcoming events (icon and context guide view)
Ticking clock showing time till next event over semantic image/motion(s) (icon and context guide view)
Ticking clock showing time till next event over semantic image/motion(s) and the total number of upcoming events (icon and context guide view)
4. Ticking clock showing time till next event over a plain background (icon and context guide view)
5. Non-ticking clocks showing time till all upcoming events (sequentially) over various backgrounds (icon and context guide view)
Calendar view showing scheduled time of next upcoming event over various backgrounds (icon and context guide view)
Calendar view showing scheduled time of all upcoming events (sequentially) over various backgrounds (icon and context guide view)
8. Animated graphic showing calendar motion (icon and context guide view)
9. Animated graphic of semantic image/motion(s) (e.g., schedule book) (icon and context guide view)
10. The total number of upcoming events over semantic image/motion(s) (icon and context guide view)
I I . The total number of upcoming events over a plain background (icon and context guide view)
12. Titles of upcoming events animated in a sequence (list view)
13. Titles and details of upcoming events animated in a sequence (tiled view)
Discovery
The Discovery context template can be analogized to a personal, digital version of the "Discovery Channel." In this case, the emphasis is on "documentaries" about particular topics. The Discovery context template simulates intelligent aggregation of information by randomly selecting infoπnation objects that relate to a given set of categories and which are posted within an optionally predetermined, configurable time period. The semantic weight as opposed to the time is the preferred consideration for determining how the information is to be ordered or presented. The context template can be implemented by filtering all information types by the semantic link strength for the categorization predicate. In this case, the filter should be less than the 'Best Bets' filter- the context template lies somewhere between 'Best Bets' and "All Items' in terms of filtering.
Visualization - Sampie image for smart hourglass, interstitial page, transition effects, background chrome, etc.
Figure imgf000543_0001
Sample Object and Context Bar Visualizations
1. Animated graphic of semantic imageΛnotion(s) (e.g., a telescope, a voyager spacecraft, an old ship at sea) (icon and context guide view)
2. Titles of the first N information items in a sequential animation (list view)
3. Titles and details of the first N information items in a sequential animation (tiled view)
4. The total number of items over semantic image/motion(s) (icon and context guide view)
5. The total number of items (icon and context guide view)
History
The History context template can be analogized to a personal, digital version of the "History Channel." In this case, the emphasis is on disseminating information not just about particular topics, but also with a historical context. For this template, the preferred axes are category and time. The History context template is similar to the Discovery context template, further in concert with "a minimum age limit." The parameters are preferably the same as that of the Discovery context template, except that the "maximum age limit" parameter is replaced with a "minimum age limit" parameter (or an optional "history time span" parameter). In addition, returned objects are preferably sorted in reverse or random order based on their age in the system or their age since creation.
Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
Figure imgf000545_0001
Visualizations - Sample Object and Context Bar Animations
1. Animated graphic of semantic image/motion(s) or an equivalent
2. Titles of the oldest (or random) N infomiation items in a sequential animation (list view)
3. Titles and details of the oldest (or random) N information items in a sequential animation (tiled view)
4. Total number of items over semantic image/motion(s) (icon and context guide view)
5. Total number of items over plain background (icon and context guide view)
All Items
The All Items context template represents context that returns any information that is relevant based on either semantics or based on a keyword or text-based search In this case, the emphasis is on disseminating information that may be even remotely relevant to the context. The primary axis for the All Items context template is preferably the mere possibility of relevance. In the preferred embodiment, the All Items context template employs both a semantic and text-based query in order to return the broadest possible set of results that may be relevant.
Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
Figure imgf000547_0001
Visualizations - Sample Object and Context Bar Animations
1. Animated graphic of semantic image/motion(s) or an equivalent
2. Titles of the most recent N information items in a sequential animation (list view)
3. Titles and details of the most recent N information items in a sequential animation (tiled view) 4. ' Total number of items over semantic image/motion(s) (icon and context guide view)
5. Total number of items over plain background (icon and context guide view)
Best Bets
The Best Bets context template (and its resulting Special Agent) represents context that returns'only highly relevant information. In a preferred embodiment, the emphasis is on disseminating information that is deemed to be highly relevant and semantically significant. For this context template, the primary axis is relevance. In essence, the Best Bets context template employs a semantic query and will not use text-based queries since it cannot guarantee the relevance of text-based query results. The Best Bets context template is preferably initialized with a category filter or keywords. If keywords are specified, the server performs categorization dynamically. Results are preferably sorted based on the relevance score, or the strength of the "belongs to category" semantic link from the object to the category filter.
Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
Figure imgf000549_0001
Visualizations - Sample Object and Context Bar Animations
1. Animated graphic of semantic image/motion(s) or an equivalent
2. Titles of the most recent N information items in a sequential animation (list view)
3. Titles and details of the most recent N information items in a sequential animation (tiled view)
4. Total number of items over semantic image/motion(s) (icon and context guide view)
5. Total number of items over plain background (icon and context guide view)
Favorites
The Favorites context template (and its resulting Special Agent) represents context that returns "favorite" or "popular" information. In this case, the emphasis is on disseminating information that has been endorsed by others and has been favorably accepted In the preferred embodiment, the axes for the Favorites context template include the level of readership interest, the "reviews" the object received, and the depth of the annotation thread on the object. In one embodiment, the Favorites context template returns only information that has the "favorites" semantic link, and is sorted by counting the number of "votes" for the object (based on this semantic link).
Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
Figure imgf000551_0001
Visualizations - Sample Object and Context Bar Animations
1. Animated graphic of semantic image/motion(s) or an equivalent
2. Titles of the most recent N information items in a sequential animation (list view)
3. Titles and details of the most recent N information items in a sequential animation (tiled view)
4. Total number of items over semantic image/motion(s) (icon and context guide view)
5. Total number of items over plain background (icon and context guide view)
Classics
The Classics context template (and its resulting Special Agent) represents context that returns "classical" information, or information that is of recognized value. Like the Favorites context template, the emphasis is on disseminating infomiation that has been endorsed by others and has been favorably accepted. For this context template, the preferred axes include a historical context, the level of readership interest, the "reviews" the object received and the depth of the annotation thread on the object. The Classics context template is preferably implemented based on the Favorites context template but with an additional minimum age limit filter and voting score, essentially functioning as an "Old Favorites" context template.
Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
Figure imgf000553_0001
Visualizations - Sample Object and Context Bar Animations
1. Animated graphic of semantic image/motion(s) or an equivalent
2. Titles of the most recent N information items in a sequential animation (list view)
3. Titles and details of the most recent N information items in a sequential animation (tiled view)
4. Total number of items over semantic image/motion(s) (icon and context guide view)
5. Total number of items over plain background (icon and context guide view)
Recommendations
The Recommendations context template represents context that returns "recommended" information, or infoπnation that the Agencies have inferred would be of interest to a user. Recommendations will be inserted by adding "recommendation" semantic links to the "SemanticLinks" table and by mining the favorite semantic links that users indicate. Recommendations are preferably made using techniques such as machine learning and collaborative filtering The emphasis of this context template is on disseminating information that would likely be of interest to the user but which the user might not have already seen. For this context template, the primary axes preferably include the likelihood of interest and freshness.
Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
Figure imgf000555_0001
Visualizations - Sample Object and Context Bar Animations
1. Animated graphic of semantic image/motion(s) or an equivalent
2. Titles of the most recent N information items in a sequential animation (list view)
3. Titles and details of the most recent N information items in a sequential animation (tiled view)
4. Total number of items over semantic image/motion(s) (icon and context guide view)
5. Total number of items over plain background (icon and context guide view)
Today
The Today context template represents context that returns information posted or holding (in the case of events) "today." The emphasis with this context template is preferably on disseminating information that is deemed to be current based on "today" being the filter to determine freshness.
Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
Figure imgf000557_0001
Visualizations - Sample Object and Context Bar Animations
1. Animated graphic of semantic image/motion(s) or an equivalent
2. Titles of the most recent N information items in a sequential animation (list view)
3. Titles and details of the most recent N information items in a sequential animation (tiled view)
4. Total number of items over semantic image/motion(s) (icon and context guide view)
5. Total number of items over plain background (icon and context guide view) Annotated Items
The Annotated Items context template represents context that returns annotated information. The emphasis with this context template is on disseminating infomiation that is likely to be important based on the fact that one or more users have annotated the items.
Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
Figure imgf000558_0001
Visualizations - Sample Object and Context Bar Animations
1. Animated graphic of semantic imageΛnotion(s) or an equivalent
2. Titles of the most recent N information items in a sequential animation (list view)
3. Titles and details of the most recent N information items in a sequential animation (tiled view)
4. Total number of items over semantic image/motion(s) (icon and context guide view)
5. Total number of items over plain background (i<~on and context guide view) Annotations
The Annotations context template represents context that returns annotated information. The emphasis with this context template is on disseminating information that are annotations.
Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
Figure imgf000559_0001
Visualizations - Sample Object and Context Bar Animations
1. Animated graphic of semantic image/motion(s) or an equivalent
2. Titles of the most recent N information items in a sequential animation (list view)
3. Titles and details of the most recent N information items in a sequential animation (tiled view)
4. Total number of items over semantic image/motion(s) (icon and context guide view)
5. Total number of items over plain background (icon and context guide view) Experts
Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
Figure imgf000560_0001
Visualizations - Sample Object and Context Bar Animations
1. Animated graphic of semantic image/motion(s) or an equivalent
2. Names of the most recent N experts in a sequential animation (list view)
3. Names and details of the most recent N experts in a sequential animation (tiled view)
4. Total number of experts over semantic ιmage/motιon(s) (icon and context guide view)
5. Total number of experts over plain background (icon and context guide view) Places
Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
Figure imgf000561_0001
Visualizations - Sample Object and Context Bar Animations
1. Animated graphic of semantic image/motιon(s) or an equivalent
2. Names of the most recent N places in a sequential animation (list view)
3. Names and details of the most recent N places in a sequential animation (tiled view)
4. Total number of places over semantic image/motιon(s) (icon and context guide view) 6. Total number of places over plain background (icon and context guide view)
Figure imgf000562_0001
Blenders
Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
Figure imgf000562_0002
Visualizations - Sample Iconic Animations
1. Animated graphic of semantic image/motion(s) or an equivalent
2. Animated graphic of blender or mixer in action
3. Titles of the blender items in a sequential animation (list view)
4. Titles and details of the blender items in a sequential animation (tiled view)
5. Total number of items over semantic image/motion(s) (icon and context guide view)
6. Total number of items over plain background (icon and context guide view)
Figure imgf000564_0001
Documents
Figure imgf000564_0002
łooks
Figure imgf000565_0001
ōagazines
Figure imgf000566_0001
Őresentations
Figure imgf000567_0001
Œesumes
Figure imgf000568_0001
œpreadsheets
Figure imgf000569_0001
Text
Figure imgf000570_0001
Web Pages
Figure imgf000571_0001
White Papers
Figure imgf000572_0001
Ņmail
Figure imgf000573_0001
Email Annotations
Figure imgf000574_0001
Email Distribution Lists
Figure imgf000575_0001
Ņvents
Figure imgf000576_0001
ōeetings
Figure imgf000577_0001
ōultimedia
Figure imgf000578_0001
Online Courses
Figure imgf000579_0001
Őeople
Figure imgf000580_0001
Ńustomers
Figure imgf000581_0001
ŕsers
Figure imgf000582_0001
Presentation Skin Types
Timeline
Visualization - Sample Image for smart hourglass, interstitial page, transition effects, background chrome, etc.
Figure imgf000583_0001
Visualizations - Sample Object and Context Bar Animations
1. Calendar view showing effective time (publication time, scheduled time, etc.) of information item over various backgrounds (icon and context guide view)
2. Calendar view showing effective time of all information items (sequentially) over various backgrounds (icon and context guide view)
3. Animated graphic showing calendar motion (icon and context guide view)
4. Animated graphic of semantic image/motiαn(s) (e.g., time warp image/motion) (icon and context guide view)
5. The total number of information items over semantic image/motion(s) (icon and context guide view)
6. The total number of information items over a plain background (icon and context guide view)
7. Titles of information items animated in a sequence (list view)
7. Titles and details of infoπnation items animated in a sequence (tiled view)
8. Scrolling, linear timeline control with items populated based on effective date/time
9. Animated timeline ticker control sorted by effective date/time
Semantic R ntirne Client Control API Specification
Client-side Runtime Control API for the Information Neivous System
By: Nosa Omoigui
Introducing the Nervana Semantic Runtime Control
Semantic Runtime Control Overview
Introduction
The Nervana Semantic Runtime Control is an ActiveX control that exposes properties and methods for use in displaying semantic data using the Nervana semantic user experience. The control will be primarily called from XSLT skins that take XML data (using the SRML schema) and spit out DHTML+TIME or SVG output, consistent with the requirements of the Nervana semantic user experience. Essentially, the Nervana control encapsulates the "SDK" on top of which the XSLT skins sit in order to produce a semantic content-driven user experience.
Figure imgf000586_0001
EnumObjectslnNamespacePath
Introduction
The EnumObjectslnNamespacePath method returns the objects in a namespace path.
Usage Scenario
A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to open a namespace path in order for the user to navigate the namespace from within the semantic browser. Prototype
SCODE
EnumObjectslnNamespacePath ( [in] BSTR Path, [in] LONG QueryMask, [out] BSTR *pQueryRequestGuid ) ;
CompileSemanticQueryFromBuffer
Introduction
The CompileSemanticQueryFromBuffer method opens an SQML buffer and compiles it into one or more execution-ready SQML buffers. For instance, an SQML file containing a blender will be compiled into SQML buffers representing each blender entry. If the blender contains blenders, the blenders will be unwrapped and an SQML buffer will be returned for each contained blender. A compiled or "execution-ready" SQML buffer is one that can be semantically processed by an agency. The implication is that a blender that has agents from multiple agencies will have its SQML compiled to buffers with the appropriate SQML from each agency.
Note: If the buffer is already compiled, the method returns S_FALSE and the return arguments are ignored.
Usage Scenario
A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to compile an SQML buffer and retrieve generated "compiled code" that is ready for execution. In typical scenarios, the application or skin will compile an SQML buffer and then prepare frame windows where it wants each individual SQML query to sit. It can then issue individual SQML semantic calls by calling OpenSemanticQueryFromBuffer and then have the results displayed in the individual ' frames.
Prototype
SCODE CompileSemanticQueryFromBuffer (
[in] BSTR SQMLBuffer,
[in] DWORD Flags, [out] DWORD *pd NumCompiledBuf fers , [out] BSTR *pbεtrCoπιpiledBuf ers ) ;
OpenSemanticQueryFromBuffer
Introduction
The OpenSemanticQueryFromBuffer method opens an SQML buffer and asynchronously fires the XML results (in SRML) onto the DOM, from whence a Nervana skin can sink the event. Note that the SQML has to be "compiled" and ready for execution. If the SQML is not ready for execution, the rail will fail. To compile an SQML buffer, call CompileSemanticQueryFromBuffer.
ϊenano
A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to open a compiled SQML buffer.
SCODE
OpenSemanticQueryFromBuffer ( [in] BSTR SQMLBuffer, [in] DWORD Flags, [out] GUID *pQueryID ) ;
GetSemanticQueryBufferFromFile
Introduction
The GetSemanticQueryBufferFromFile method opens an SQML file, and returns the buffer contents. The buffer can then be compiled and/or opened. Usage Scenario
A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to convert an SQML file into a buffer before processing it.
Prototype
SCODE
GetSemanticQueryBufferFromFile ( [in] BSTR SQMLFilePath, [in] DWORD FileOpenFlags , [out] BSTR *pbstrSQ IjBuf f er ) ;
Figure imgf000589_0001
Introduction
The GetSemanticQueryBufferFromMamespace method opens a namespace object, and retrieves its SQML buffer.
Usage Scenario
A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to open an SQML buffer when it already has access to the id and path of the namespace object.
Prototype
SCODE
GetSemanticQueryBufferFromNamespace (
[in] GUID ObjectlD,
[in] .BSTR Path,
[out] BSTR *pbstrSQMLBuffer ) ; GetSemanticQueryBiLifferFromURL
Introduction
The GetSemanticQueryBufferFromURL method wraps the URL in an SQML buffer, and returns the buffer. ,
Usage Scenario
A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to convert an URL of any type to SQML. This can include file paths, HTTP URLs, FTP URLs, Nervana agency object URLs (prefixed by "wsobject:/") or Nervana i agency URLs (prefixed by "wsagency:/ ").
Prototype
SCODE GetSemanticQueryBufferFromURL (
[in] BSTR URL,
[out] BSTR *pBuf fer ) ;
GetSemanticQueiyBufferFromClipboard
Introduction
The GetSemanticQueryBufferFromClipboard method converts the clipboard contents to SQML, and returns the buffer.
Usage Scenario
A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to get a semantic query from the clipboard. The application can then load the query buffer.
Prototype
SCODE GetSemanticQueryBufferFromClipboard ( [out] BSTR *pBuffer ) ; Stop
Introduction
The Stop method stops current open request.
Usage Scenario
A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to stop a load request is just issued.
Prototype
SCODE Stop ( [in] GUID Query ID ) ;
Refresh
SnirødHciEoji
The Refresh method refreshes the current open request.
Usage Scenario
A Nervana client application (for instance, the semantic browser) or a Nervana skin will call this method to refresh the currently loaded request.
Prototype
SCODE Refresh ( [in] GUID Query ID ) ;
CreateNamespaceObject
Introduction
The CreateNamespaceObject method creates a namespace object and returns its GUID. Usage Scenario
A Nervana client application (for instance, the semantic browser) or a Nervana skin will typically call this method to create a temporary namespace object when a new query document has been opened.
Prototype
SCODE
CreateNamespaceOb j ect (
[in] BSTR Name ,
[in] BSTR Description,
[in] BSTR QueryBuffer,
[in] LONG AgentObjectType,
[in] LONG Atributes ,
[in] LONG NamespaceObj ectType ,
[out] GUID *pObj ectID ) ;
[D sG©te!M]si[rϊrι]©s a(£e(Q) |e Ϊ
Introduction
The DeleteNamespaceObject method deletes a namespace object.
Usage Scenario
A Neivana client application (for instance, the semantic browser) or a Nervana skin will typically call this method to delete a temporary namespace object.
Prototype
SCODE DeleteNamespaceObj ect ( [in] GUID ObjectlD ) ; CopyObject
Introduction
The CopyObject method copies the semantic object to the clipboard as an SQML buffer using a proprietary SQML clipboard format. The object can then be "pasted" onto agents for relational semantic queries, or used as a lens over other objects or agents.
Usage Scenario
A Nervana skin will typically call the CopyObject method when the user clicks on the "Copy" menu option - off a popup menu on the object.
SCODE CopyObj ect ( [in] BSTR Obj ectSRML )
Figure imgf000593_0001
Introduction
The CanObjectBeAnnotated method checks whether the given object can be annotated. Usage Scenario
A Nervana skin will typically call the CanObjectBeAnnotated method to determine whether to show Ul indicating the "Annotate" command.
Prototype
SCODE CanObjectBeAnnotated ( [in] BSTR bstrOb ectSRML ); AnnotateObject
Introduction
The AnnotateObject method invokes the currently installed email client and initializes it to send an email annotation of the object to the email agent of the agency from whence the object came.
Usage Scenario
A Nervana skin will typically call the AnnotateObject method when the user clicks on the "Annotate" menu option - off a popup menu on the object.
Prototype
SCODE AnnotateObj ect ( [in] BSTR bstrObj ectSRML ) ;
CanObjectBePublished
Inirøduciion
The CanObjectBePublished method checks whether the given object can be published.
Usage Scenario
A Nervana skin will typically call the CanObjectBePublished method to determine whether to show Ul indicating the "Publish" command.
Prototype
SCODE CanObjectBePublished ( [in] BSTR bstrObjectSRML ) ;
PublishObject
Introduction
The PublishObject method invokes the currently installed email client and initializes it to send an email publication of the object to the email agent of the agency from whence the object came. Usage Scenario
A Nervana skin will typically call the PublishObject method when the user clicks on the "Publish" menu option - off a popup menu on the object.
Prototype
SCODE AnnotateObject ( [in] BSTR bstrObjectSRML ) ;
OpenObjectContents
Introduction
The OpenObjectContents method opens the object using an appropriate viewer. For instance, an email object will be opened in the email client, a document will be opened in the browser, etc..
Usage Scenario
A Nervana skin will typically call the OpenObjectContents method when the user clicks on the "Open" menu option - off a popup menu on the object.
SCODE OpenObjectContents ( [in] BSTR Obj ectSRML ) ;
SendEmailToPersonObject
Introduction
The SendEmailToObject method is called to send email to a person or customer object. The method opens the email client and initializes it with the email address of the person or customer object.
Usage Scenario
A Nervana skin will typically call the SendEmailToObject method when the user clicks on the "Send Email" menu option - off a popup menu on a person or customer object. Prototype
SCODE SendEmailToObject ( [in] BSTR ObjectSRML );
GetObjectAnnotations
Introduction
The GetObjectAnnotations method is called to get the annotations an object has on the agency from whence it came.
Usage Scenario
A Nervana skin will typically call the GetObjectAnnotations method when it wants to display the titles of the annotations an object has - for instance, in a popup menu or when it wants to display the annotations metadata in a window.
Prototype
SCODE
GetObjectAnnotations (
[in] BSTR Obj ectSRML,
[in] LONG QueryMask,
[out] BSTR *pQueryRequestGuid )
IsObjectMarkedAsFavorite
Introduction
The IsObjectMarkedAsFavorite method is called to check whether an object is marked as a favorite on the agency from whence it came.
Usage Scenario
A Nervana skin will typically call the IsObjectMarkedAsFavorite method to determine what Ul to show - either the "Mark as Favorite" or the "Unmark as Favorite" command. If the object cannot be marked as a favorite (for instance, if it did not originate on an agency), the error code EJN VALIDARG is returned.
Prototype
SCODE IsObjectMarkedAsFavorite ( in] BSTR ObjectSRML } ;
MarkObjectAsFavorite
Introduction
The MarkObjectAsFavorite method is called to mark the object as a favorite on the agency from whence it came.
Usage Scenario
A Nervana skin will typically call the MarkObjectAsFavorite method when the user clicks on the "Mark as Favorite" command.
Prototype
SCODE
MarkAsFavorite ( in] BSTR Obj ectSRML ) ;
UnmarkObjecfAsFaεvΘBΪte
Introduction
The UnmarkObjectAsFavorite method is called to unmark the object as a favorite on the agency from whence it came.
Usage Scenario
A Nervana skin will typically call the UnmarkObjectAsFavorite method when the user clicks on the "Unmark as Favorite" command.
Prototype
SCODE
UnmarkAsFavorite { in] BSTR ObjectSRML ) ; IsSmartAgentOnClipboard
Introduction
The IsSmartAgentOnClipboard method is called to check whether a smart agent has been copied to the clipboard.
Usage Scenario
A Nervana skin will typically call the IsSmartAgentOnClipboard method when it wants to toggle the user interface to display the "Paste" icon or when the "Paste" command is invoked.
Prototype
SCODE
IsSmartAgentOnClipboard ( )
GetSmartLensQueryBuffer
Introduction
The GetSmartLensQueryBuffer method is called to get the query buffer of the smart lens. This returns the SQML of the query that represents the objects on the smart agent that is on the clipboard, and which are semantically relevant to a given object.
Usage Scenario
A Nervana skin will typically call the GetSmartLensQueryBuffer method when the user hits "Paste as Smart Lens" to invoke the smart lens off the smart agent that is on the clipboard.
Prototype
SCODE
GetSmartLensQueryBuffer (
[in] BSTR Obj ectSRML,
[in] LONG QueryMask,
[out] BSTR *pQueryRequestGuid ) ; OpenObjectContents
Introduction
The OpenObjectContents method opens the object using an appropriate viewer. For instance, an email object will be opened in the email client, a document will be opened in the browser, etc.
Usage Scenario
A Nervana skin will typically call the OpenObjectContents method when the user clicks on the "Open" menu option - off a popup menu on the object.
Prototype
SCODE OpenObjectContents ( [in] BSTR Obj ectSRML ) ,-
ifϊrareirn fl
Email_GetFromLinkObjects
Introduction
The Email_GetFromLinkObjects method is called to get the metadata for the "From" links on an email object from the agency from whence it came.
Usage Scenario
A Nervana skin will typically call the Email_GetFromLinkObjects method when it wants to navigate to the "From" list from an email object, or to display a popup menu with the name of the person in the "From" list.
Protc ρs
SCODE
Email_GetFromLinkObjects (
[in] BSTR EmailObj ectSRML,
[in] LONG QueryMask,
[out] BSTR *pQueryRequestGuid )
Email GetToLϊnkObjects
Introduction
The Email_GetFromLinkObjects method is called to get the metadata for the 'To" links on an email object from the agency from whence it came.
Usage Scenario
A Nervana skin will typically call the Email_GetToLinkObjects method when it wants to navigate to the "To" list from an email object, or to display a popup menu with the name of the person in the "To" list. Prototype
SCODE
Email_GetToLinkObjects (
[in] BSTR EmailObj ectSRML,
[in] LONG QueryMask,
[out] BSTR *pQueryRequestGuid ) ;
EmaiLGetCcLϊnkObjects
Introduction
The Email_GetCcLinkObjects method is called to get the metadata for the "CC" links on an email object from the agency from whence it came.
Usage Scetrøri©
A Nervana skin will typically call the Email_GetCcLinkObjects method when it wants to navigate to the "CC" list from an email object, or to display a popup menu with the name of the person in the "CC" list.
Prototype
SCODE
Email_GetCcLinkObjects (
[in] BSTR EmailObj ectSRML,
[in] LONG QueryMask,
[out] BSTR *pQueryReguestGuid ) ;
Emaal GetBccLϊrrkObjects
Introduction
The Email_GetBccLinkObjects method is called to get the metadata for the "BCC" links on an email object from the agency from whence it came.
Usage Scenario
A Nervana skin will typically call the Email_GetBccLinkObjects method when it wants to navigate to the "BCC" list from an email object, or to display a popup menu with the name of the person in the "BCC" list. Prototype
SCODE Email_GetBccLinkObj ects (
[in] BSTR EmailObjectSRML,
[in] LONG QueryMask,
[out] BSTR *pQueryRequestGuid )
Email GetAttachmentLinkObjects
Introduction
The Email_GetAttachmentLinkObjects method is called to get the metadata for the "Attachment" links on an email object from the agency from whence it came.
Usage Scenario
A Nervana skin will typically call the Email_GetAttachmentLinkObjects method when it wants to navigate to the "Attachments" link from an email object, or to display a popup menu with the titles of the attachments in the "Attachments" list.
Prototype
SCODE
Email_GetAttachmentLinkObjects (
[in] BSTR EmailObj ectSRML,
[in] LONG QueryMask,
[out] BSTR *pQueryRequestGuid ) ;
Person Control APIs
Person_GetDϊrectReports
Introduction
The Person_GetDirectReports method is called to get the metadata for the "Direct Reports" links on a person object from the agency from whence it came.
Usage Scenario
A Nervana skin will typically call the Person_GetDirectReports method when it wants to navigate to the "Direct Reports" link from a person object, or to display a popup menu with the names of the direct reports in the "Direct Reports" list.
Prototype
SCODE
Person_GetDirectReports (
[in] BSTR EmailObj ectSRML,
[in] LONG QueryMask,
[out] BSTR *pQueryRequestGuid
Person GetDistributionLists
Introduction
The Person__GetDistributionLists method is called to get the metadata for the "Member of Distribution Lists" links on a person object from the agency from whence it came.
Usage Scenario
A Nervana skin will typically call the Person_GetDistributionLists method when it wants to navigate to the "Member of Distribution Lists" link from a person object, or to display a popup menu with the names of the distribution lists of which the person is a member. Prototype
SCODE Person_GetDistributionLists (
[in] BSTR PersonObjectSRML,
[in] LONG QueryMask,
[out] BSTR *pQueryRequestGuid )
Person GetlnfoAuthored
Introduction
The Person_GetlnfoAuthored method is called to get the metadata for the "Info Authored by Person" links on a person object from the agency from whence it came.
Usage Scenario
A Nervana skin will typically call the Person_GetlnfoAuthored method when it wants to navigate to the "Info Authored by Person" link from a person object, or to display a preview window with time-critical or recent information that the person authored.
Prototype
SCODE
Person_GetInfoAuthored (
[in] BSTR PersonObjec SRML,
[in] BOOL SemanticQuery,
[in] LONG QueryMask,
[out] BSTR *pQueryRequestGuid ) ;
Person GetlnfoAnnotated
Introduction
The Person_GetlnfoAnnotated method is called to get the metadata for the "Info Annotated by Person" links on a person object from the agency from whence it came.
Usage Scenario
A Nervana skin will typically call the Person_GetlnfoAnnotated method when it wants to navigate to the "Info Annotated by Person" link from a person object, or to display a preview window with time-critical or recent information that the person annotated. Prototype
SCODE
Person_GetInf oAnnotated (
[in] BSTR PersonObj ectSRML,
[in] LONG QueryMask,
[out] BSTR *pQueryReguestGuid ) ;
Person GetAnnotationsPosted
Introduction
The Person_GetAnnotationsPosted method is called to get the metadata for the "Annotations Posted by Person" links on a person object from the agency from whence it came.
Usage Scenario
A Nervana skin will typically call the Person_GetAnnotationsPosted method when it wants to navigate to the "Annotations Posted by Person" link from a person object, or to display a preview window with time-critical or recent annotations that the person posted.
Pro o ype
SCODE Person_GetAnnotationsPosted (
[in] BSTR PersonObj ectSRML,
[in] LONG QueryMask,
[out] BSTR *pQueryRequestGuid
Person SendEmailTo
Introduction
The Person_SendEmailTo method is called to send email to a person or customer object. The method opens the email client and initializes it with the email address of the person or customer object.
Usage Scenario
A Nervana skin will typically call the Person_SendEmailTo method when the user clicks on the "Send Email" menu option - off a popup menu on a person or customer object. Prototype
SCODE Person_SendΞmailTo ( [in] BSTR ObjectSRML )
Figure imgf000606_0001
Event OnBeforeQuery
Introduction
The OnBeforeQuery event is fired before the control issues a query to resources consistent with the current semantic request.
Usage Scenario
A Nervana client application (for instance, the semantic browser) or a Nervana skin will sink this event if it wants to cancel a query or cache state before the query is issued.
Prototype
VOID
OnBeforeQuery (
[in], GUID QuerylD,
[in] BSTR QueryBuffer,
[in] DWORD QueryMask,
[in] DWORD Flags,
[out] BOOL ♦Cancel ) ; Event: OnQueryBegin
Introduction
The OnQueryBegin event is fired when the control issues the first query to a resource consistent with the current semantic request.
Usage Scenario
A Nervana client application (for instance, the semantic browser) or a Nervana skin will sink this event if it wants to cache state or display status information when the query is in progress.
VOID
OnQueryBegin ( [in] GUID ObjectlD )
rafe
Figure imgf000607_0001
Introduction
The OnQueryComplete event is fired before the control issues a query to resources consistent with the current semantic request.
Usage Scenario
A Nervana client application (for instance, the semantic browser) or a Nervana skin will sink this event if it wants to cancel a query or cache state before the query is issued.
Prototype
VOID
OnQueryComplete ( [in] GUID Query ID ) ; Event: OnQueryResultsAvailable
Introduction
The OnQueryResultsAvailable event is fired when there are available results of an asynchronous method call. The event indicates the request GUID, via which the caller can uniquely identify the specific method call that generated the response.
Usage Scenario
A Nervana client application (for instance, the semantic browser) or a Nervana skin will sink this event to get responses to method calls on the control.
Prototype
VOID OnQueryResultsAvailable (
[in] GUID QueryID,
[in] SCODE QueryResuit,
[in] BSTR Results,
[in] DWORD NumResults,
[in] DWORD QueryMask,
[in] VARIANT ResultsParam ) ;
Appendix A
Query Mask Values
ttdefine QM_RESULTS 0x01 #define QM_RESULTCOUNT 0x02 ttdefine QM_NEWRESULTS 0x04 #define QM_NEWRESULTCOUNT 0x08 ttdefine QM DEFAULT ( QM RESULTS ) Example :
Person_GetInf oAuthored ( PersonObj ectSRML, QM_RESULTS | QM_RESULTCOUNT, &QueryRequestGuid ) ;
Smart Request Watch Specification
Smart Request Watch Specification for the Information Nervous System
By: Nosa Omoigui
Figure imgf000611_0001
Smart Request Watch refers to a feature of the Information Nervous System that allows users of the semantic browser (the Information Agent or the Librarian) to monitor (or "watch") smart requests in parallel. This is a very important feature in that it enhances productivity by allowing users to track several requests at the same time.
The feature is implemented in the client-side semantic runtime, the semantic browser, and skins that allow a configurable way of watching smart requests (via a mechanism similar to "Picture-ln-Picture" (PIP) functionality in television sets). The following software components are used:
1. The Request Watch List (RWL)
2. Request Watch Groups
3. The Notification Manager (NM)
4. Watch Group Monitors (WLM)
5. The Watch Pane
6. The Watch Window
Figure imgf000612_0001
Overview
The Request Watch List is a list of smart requests (or smart agents) that the client runtime manages. This list essentially contains the smart requests the user wishes to monitor. The Request Watch List contains a list of entries, the Request Watch List Entry (RWLE) with the following data structure:
Figure imgf000612_0002
Figure 1: Table showing the Request Watch List Entry Data Structure
The Request Watch List (RWL) contains an array or vector of RWLE structures. The Request Watch List Manager manages the RWL. The semantic browser provides a user interface that allows the user to add smart requests to the RWL - the Ul talks to the RWLM to add and remove RWLEs to/from the RWL. The RWL is stored (and persisted) centrally by the client-side semantic runtime (either as an XML file-based representation or in a store like the Windows registry). Request Watch Groups
The RWL can also be populated by means of Request Watch Groups (RWGs). A Request Watch Group provides a means for the user to monitor a collection of smart requests. It also provides a simple way for users to have the semantic browser automatically populate the RWL based on configurable criteria. There are two types of RWGs: Auto Request Watch Groups and the Manual Request Watch Group. Auto Request Watch Groups are groups that are dynamically populated by the semantic browser depending on the selected profile, the profile of the currently displayed request, etc. The Custom Request Watch Group allows the user to manually populate a group of smart requests (regular smart requests or blenders) to monitor as a collection.
Users will be able to add the following types of Auto-RWGs (for one or more configurable profiles, including "All Profiles")1:
1. Breaking News - this tells the semantic browser to automatically add a Breaking News smart request to the RWL (for the selected profile(s)).
2. Headlines - this tells the semantic browser to automatically add a Headlines smart request to the RWL (for the selected profile(s))
3. Newsmakers - this tells the semantic browser to automatically add a Newsmakers smart request to the RWL (for the selected profιle(s))
4. Categorized Breaking News - this tells the semantic browser to automatically add Categorized Breaking News smart requests to the RWL (for the contextual profile). The semantic browser will dynamically add smart requests with category filters corresponding to each subcategory of the currently displayed smart request (and for the contextual or current profile) - if the currently displayed smart request has categories. For example, if the smart request "Breaking News" about Technology is currently being displayed in a he semantic browser instance, and if the category 'Technology" has 5 sub-categories (e.g., Wireless, Semiconductors, Nanotechnology, Software, and Electronics), the following smart requests will be dynamically added to the RWL when the current smart request is loaded:
• Breaking News about Technology. Wireless [<Contextual Profile Name>]
• Breaking News about Technology.Semiconductors [<Contextual Profile Name>]
• Breaking News about Technology.Nanotechnology [<Contextual Profile Name>]
• - Breaking News about Technology.Software [<Contextual Profile Name>]
• Breaking News about Technology.Electronics [<Contextual Profile Name>] -
Also, the RWLEs for these entries will be initialized with the RequestViewInstancelD of the current semantic browser instance. If the user navigates to a new smart request, the categorized Breaking News for the previously loaded smart request will be removed from the RWL and a new list of categorized Breaking News will be added for the new smart request (if it has any categories) - and initialized with a new RequestViewInstancelD corresponding to the new smart request view. This creates a smart user experience wherein relevant categorized breaking news (for subcategories) will be dynamically displayed based on the currently displayed request. The user will then be able to monitor Categorized Breaking News smart requests as a watch group or collection.
5. Categorized Headlines - this tells the semantic browser to automatically add Categorized Headlines smart requests to the RWL (for the contextual profile). This is similar to Categorized Breaking News, except that Headlines are used in this case. The user will then be able to monitor Categorized Headlines smart requests as a watch group or collection.
'' Note that this is only a preliminary list and that other types of RW s will be added in the future. 6. Categorized Newsmakers - this tells the semantic browser to automatically add Categorized Newsmakers smart requests to the RWL (for the contextual profile). This is similar to Categorized Breaking News, except that Newsmakers are used in this case. The user will then be able to monitor Categorized Newsmakers smart requests as a watch group or collection.
7. My Favorite Requests - this tells the semantic browser to automatically add all favorite smart requests to the RWL (for the selected profile(s)). This allows the user to watch or monitor all his/her favorite smart requests as a group.
8. My Favorite Breaking News - this tells the semantic browser to automatically add all favorite breaking news smart requests to the RWL (for the selected profile(s)). This allows the user to watch or monitor all his/her favorite breaking news smart requests as a group.
9. My Favorite Headlines - this tells the semantic browser to automatically add all favorite headlines smart requests to the RWL (for the selected profile(s)). This allows the user to watch or monitor all his/her favorite headlines smart requests as a group.
10. My Favorite Newsmakers - this tells the semantic browser to automatically add all favorite >. newsmakers smart requests to the RWL (for the selected profile(s)). This allows the user to watch or monitor all his/her favorite newsmakers smart requests as a group.
Request Watch Group SVIariager User
Figure 2 below shows the "Smart Request Watch" dialog box in the semantic browser. The top half of the dialog is used to add auto-watch groups. The user can select auto-watch group types and profile types ("All Profiles," "Contextual Profile," and the actual profile names) and add them to the auto-watch- group list. The user can also remove auto-watch-groups. The bottom half of the dialog box is used to add/remove smart requests to/from the manual watch group.
Figure imgf000615_0001
Figure 2: The "Smart Request Watch" Dialog Box
Figure imgf000616_0001
The Notification Manager (NM) is a componenf of the semantic runtime client that monitors smart requests in the RWL. The NM has a thread that periodically invokes each smart request in the RWL (via the client semantic query processor) and updates the RWLE with the "results count" and the "last update time." In the preferred embodiment the NM should invoke the smart requests every 5-30 seconds. The NM can smartly adjust the periodicity of request checks depending on the size of the RWL (in order to minimize bandwidth usage and the scalability impact on the Web service).
For time-sensitive smart requests (like Breaking News, Headlines, and Newsmakers), the NM should invoke the smart request without any additional time filter. However, for non time-sensitive requests (like for information as opposed to context types or for non time-sensitive context templates like Favorites and Recommendations), the NM should invoke the query for the smart request with a time filter (e.g., the last 10 minutes).
Watch Group Monitors
The semantic runtime client manages what the inventor calls Watch Group Monitors (WGM). For each watch group the user has added to the watch group list, the client creates a watch group monitor. A watch group monitor tracks the number of new results in each request in its watch group. The watch group monitor creates a queue for the RWLEs in the watch group that have new results. The WGM manages the queue in order to maximize the freshness of the results. The WGM periodically polls the NM to see whether there are new results for each request in its watch group. If there are, it adds the request to the queue depending on the 'last result time' of the request. It does this in order to prioritize requests with the freshest results first. The cu ently displayed visual style (skin) running in the Presenter would then call the semantic runtime OCX to dequeue the requests in the WGM queue. This way, the request watch user interface will be consistent with the existence of new results and the freshness of the results. Once there are no more new results in the currently displayed request, the smart style will dequeue the next request from the WGM queue.
Rftnjsm
The Watch Pane (WP) refers to a panel that gets displayed in the Presenter (alongside the main results pane) and which holds visual representations of the user's watch groups. The WP allows the user to glance at each watch group to see whether there are new results in its requests. The WP also allows the user to change the current view with which each watch group's real-time status gets displayed. The following views are currently defined: Tiled View- this displays the title of the watch group along with the total
Figure imgf000617_0001
The WP also allows the user to watch a watch group. The user will do this by selecting one of the watch groups in the WP and dragging it into the main results pane (or by a similar technique). This forms a Watch Window (WW). This is similar to TV's picture-in-picture functionality, except that in this case semantic requests and results as opposed to television channels are being "watched." The VWV can be displayed in any of the aforementioned views. When the WW is in Deep View however, the WW's view controls are displayed. The following controls are currently defined:
• Pinning Requests - this allows the user to pin a particular request in the watch group. The WW will keep displaying the new results for only the pinned requests (in a cycle) and will not advance to other requests in the watch group for as long as the current request remains pinned. o Swapping Requests - this allows the user to swap the currently displayed request with the main request being shown in the semantic browser. The smart style will invoke a method on the OCX to create a temporary request with the swapped request (hashed by its SQML buffer) and then navigate to that request while also informing the Presenter to now display the main request in its place (in the WW).
• Stop, Play, Seek, FF, RW, Speedup - these allow the user to stop, play, seek, fast- • forward, rewind or speedup the "watch group request stream." For instance, a fast- forward will advance to several requests ahead of the currently displayed one.
• Results controls - this allows the user to control the results in each request in the watch group. Essentially, the results are a stream within a stream and this will also allow the user to control the results in the current request in the current watch group.
• Auto-Display Mode - this will automatically hide the WW when there are no results to display and fade it in when there are new results. This way, the user can maximize the utility of his/her real estate on the screen knowing that watch windows will fade in when there are new semantic results. This feature also allows the user to manage his/her attention during information interaction in a personal and semantic way.
• Docking, Closing, Minimizing, Maximizing - these features, as the names imply, allows the user to dock, close, minimize or maximize watch windows.
Figure 3 below illustrates a watch window:
Figure imgf000618_0001
Figure 3: An Illustration of the Watch Window
Smart Sty les Specification
Smart Styles Speculation for the Information Nervous System
By: Nosa Omoigui
Irπ ffffi. TO }~
Overview
Smart Styles
• A color theme and animation theme applied to a style theme yields a smart style
• Smart - meaning it is adaptive to the mood of its request, context panes, preview mode, handheld mode, live mode, slideshow mode, Screensaver mode, blender/collection mode, accessibility, and user settings recognition (see below)
Style Classes o Subtle - for task-oriented productivity o Moderate - for task-oriented productivity with some presentation effects
• Exciting - exciting effects (good for both primary and secondary machines, and for inactive Nervana windows - e.g., Nervana client windows in the background or docked on the taskbar) o Super-exciting (great for smart screensavers with productivity - e.g., secondary machines
- when the user is using his/her primary machine)
• Sci-Fi (for Matrix fans, great for smart screensavers without specific need for productivity
- e.g., when the user is away from his/her desk)
Style Themes
• Variable, unlimited - created by us and perhaps third party skin authors Color Themes
• Variable, unlimited - created by us and perhaps third party skin authors Animation Themes
• Variable, unlimited - created by us and perhaps third party skin authors Implicit and Dynamic Smart Style Properties
• Mood - the smart style must convey the mood of the request (i.e., the request is a parameter passed to the smart style). This will involve semantic images, semantic motions, visualizations, etc. that convey the properties of the smart request (the context template or information type, the categories, whether there are filters (e.g., local documents), the information types of those filters, etc.) • Context panes - e.g., deep info pane (per object), dockable preview panes, dockable contextual PIP watch groups/panes, etc.
• Preview Mode - each smart style must be able to display its results for preview (in a small window)
• Handheld Mode - each smart style must be able to display its results optimized for a handheld device
• Live mode - each smart style must have a "live" mode during which it would display realtime semantic visualizations (per object). This can be toggled on or off (e.g. if the user does not want real-time semantic visualizations, or to save bandwidth that results from real-time Web service calls per object)
• Slideshow mode - each smart style must be able to "play" the results of the request - like a live stream
• Screensaver mode - each smart style must be able to play "play" the results of the request as a screensaver. This is a variant of slideshow mode, except in fullscreen/theater mode. o Blender/collection mode - each smart style must change its Ul appropriately if the request it is displaying is a blender/collection
• Accessibility - each smart style must support accessibility (TBD)
• User settings recognition - the Nervana Librarian will allow the user to indicate whether he/she is a beginner, a moderate user, or a power-user, and his/her job function (R&D, sales, marketing, executive, etc.). Each smart style should take this into account where appropriate.
Each smart style is responsible for:
• Visualizing the Mood of the Current Request (including semantic images, motion, chrome, etc. - CONSISTENT with the semantics oFthe request)
• Visualizing a Change in the number of Items in the Current Request
• Visualizing the Mood of each object (intrinsically)
• Visualizing the Mood of each object's context (headlines, breaking news, experts, etc.) o Binary/Absolute (is there breaking news, OR NOT? how many experts are there? how many headlines?) o On a gradient (how breaking is breaking news?, how critical are the headlines? what is the level of expertise for the experts?, etc.) o Consistent with what should deserve the user's ATTENTION
• Visualizing a change in each object's context (there is new breaking news, there are new annotations, etc.)
• Visualizing the RELATIVE criticality of each object being displayed (different sized view ports, different fonts, different chrome, etc.) Visualizing a request navigation and "loading" status (interstitials that INTRODUCE the mood of the new request being loaded)
Visualizing individual PIP windows (animated with an animation control)
Visualizing the addition of a new PIP window (to a PIP window palette)
Resizing/Moviπg/Docking PIP Windows
Visualizing preview windows (for context palettes, context visualization Ul on each object, timelines, etc.)
Sound consistent with the visualization of mood and notifications (across the board)
Figure 1 below shows a screenshot with the 'Smart Styles' dialog. This allows the user to browse smart styles by pivoting across style classes, style themes, color themes, and animation themes. A preview window shows the user a preview of the currently selected smart style.
Figure imgf000623_0001
Figure 1: Smart Styles Dialog Box Semantic Query Markup Language SQMIL Specification
Client-side Semantic Query Document Specification for the Information Nervous
System
By: Nosa Omoigui
Overview
Semantic Query Markup Language (SQML) Overview
Introduction
The Nervana Semantic DHTML Behavior is an Internet Explorer DHTML Behavior that From the client's perspective, every thing it understands is a query document. The client opens 'query documents,' just like how a word processor opens 'textual and compound documents.' The Nervana client is primarily responsible for processing a Nervana semantic query document and rendering the results. A Nervana semantic query document is expressed and stored in form of the Nervana Semantic Query Markup Language (SQML). This is akin to a "semantic file format."
The SQML semantic file format consists of the following:
Head - The 'head' tag, like in the case of HTML, includes tags that describe the document. o Title - The title of the document. o Comments - The comments of the document. o UserName -The usemame of the document creator o SystemNamβ - The systemname of the device on which the document was created o Subject - The subject of the document o Creator - The creator of the document o Company - The company in which the document was created
ReqυestType - This indicates the type of request. It can be "smart request" (indicating requests to one or more information community web services) or "dumb request"
(indicating requests to one or more local or network resources) o ObjectType - This fully qualifies the type of objects returned by the query. o URI - The location of the document o CreationTime - The creation time of the document o LastModifiedTime - The last modified time of the document o LastAccessedTime - The last accessed time of the document o Attributes - The attributes of the document. For now, there are no defined attributes. o RevisionNumber-The revision number of the document o Language - The language of the document o Version - this indicates the version of the query. This allows the web service's semantic query processor to return results that are versioned. For instance, one version of the browser can use V1 of a query, and another version can use V2. This allows the web service to provide backwards compatibility both at the resource level (e.g., for agents) and at the link level. Targets - This indicates the names and the URLs of the information community web services that the query document targets. o Type - this indicates the type of targets. This can be "targetentries," in which case the tag includes sub-tags indicating the actual web service targets, or "allsubscribedtargets," in which case the query processor uses all subscribed information communities. Categories - This indicates the list of category URLs that the query document refers to. Each "category" entry contains a name attribute and a URI attribute that indicates the URL of the Knowledge Domain Server (KDS) from which the category came. o Type - this indicates the type of categories. This can be either "categoryentries," in which case the sub-tag refers to the list of category entries, "allcategories," in which case all categories are requested from the information community web services, or "myfavoritecategories ," in which case the query processor gets the user's favorite categories and then generates compiled SQML that contains these categories (this compiled SQML is then sent to the server(s)). Query -This is the. parent tag for all the main query entries of the query document Resource - The reference to the 'dumb' resource being queried. Examples include file paths, URLs, cache entry identifiers, etc. These will be mapped to actual resource managers components by the interpreter. o Type - The type of resource reference, qualified with the namespace. Examples of defined resource reference types are: nervana:url (this indicates that the resource reference is a well-formed standard Internet URL, or a custom Nervana URL like 'agent://..."), nervana:filepath (this indicates that the resource reference is a path to a file or directory on the file-system), and nervana:namespaceref (this indicates that the resource comes from the client semantic namespace). o Uri - This indicates the universal resource identifier of the resource. In the case of paths and Internet URLs, this indicates the URL itself. In the case of namespace entries, this indicates the GUID identifier of the entry, o Mid - This indicates the metadata identifier, which is used by the SQML interpreter to map the resource to the metadata section of the document. The metadata id is mapped to the same identifier within the metadata section, o Args - This indicates the arguments of the resource identifier. Links - this indicates the reference to the semantic links (for "targets" only) o Type - this indicates the type of links. This can be "linkentries," indicating the links are explicit entries, o LinkEntries - this indicates the details of a link entry.
» Predicate - this indicates the type of predicate for the link. For instance, the predicate "nervana:relevantto" indicates that the query is "return all objects from the resource R that are relevant to the object O," where R and O and the specified resource and object, respectively. Other examples of predicates include nervanaxeportsto, nervaπa:teammateof, nervana:from, nervana:to, nervanaxc, nervana:bcc, nervana:attacheclto, nervana:sentby, nervana:sentto, nervana:postedon, nervana:containstext, etc.
■ Type - this indicates the type of object reference indicates in the 'Link' tag. Examples include standard XML data types like xml:string, xm integer, Nervana equivalents of same, custom Nervana types like nervana:datetimeref (which could refer to object references like 'today' and 'tomorrow'), and any standard Internet URL (HTTP, FTP, etc.) or Nervana URL (objects://, etc.) that refers to an object that Nervana can process as a semantic XML object
Metadata - this contains the references to the metadata entries. o MetadataEntry - this indicates the details of a metadata entry
■ Mid - this indicates the metadata identifier (GUID) - Value - this indicates the metadata itself Example: Documents (Information or1 Context-Based)
<?xml version= " 1 . 0 " encodiπg= "utf -8 " ?> <sqml>
<head requesttype="smart request" objecttype="context\headlines" uri="c:\foo' s\bar.pdf" creationtime="foo" lastmodifiedtime="foo" lastaccessedtime="foo" attributes="0" revisionnumber="0" language="foo" version="foo" /> <title>foo<:/title> <comments>foo</commen s> <username>foo</usemame> <systemname>foo</systemname> <subject>foo</subject> > <creator>foo</creator> <company>foo</company> <targets>
<target name="Marketing" ref ype="uri" ref="kis : //marketing/default . wsdl " /> <target name="Research" reftype="uri" ref = "kisp : //research/defaul . sdl " l> </targets> <cat.egories> -
<category name="reuters\pharmaceuticals\biotechnology" ref ype="uri" ref="kdsp : //reuters . com/categories .wsdl?id=45 "
/>
<category name="reuters\ρharmaceuticals\life_sciences" ref ype="uri" ref="kdsp : //reuters . co /categories .wsdl?id=57 "
/> </categories>
/>
<resources>
<resource name="foo" type="informa ion\documents\general document" reftype= "nervana:filepath" ref="file ://c: \bar.doc" mid="7886e4a0-55d9-45ac-a084-97adc6fffd0f" args="" / >
<resource name="f oo" type= " inf ormation\all information" reftype= "nervana : uri " ref =" file : //c : \bar . doc" mid=,I01fc64a3-c068-4339-bc97-17e5ff37e93f" args=""
/> <resource name="foo" type="information\all information" reftype="nervana : folderpath" ref="file://c:\" mid="f8cc39c3-e4f0-4a29-be2a-d2faf3βeb3a0" , args="includesubfolders=true"
/>
<resource name="foo" type="information\documents\general document" reftype="nervana : ri" ref="http : //www.bar. com/doc .htm" mid="fBcc39c3-e4f0-4a29-be2a-d2faf36eb3a0" args=""
<reεource name="foo" type="information\documents\general document" reftype="nervana :uri" ref="ftp : //gate . com/doc . x " mid="f8cc39c3-e4f0-4a29-be2a-d2faf36eb3a0" args=""
/>
<resource name="foo" type="in ormation\documents\general document" reftype="nervana : filepath" ref="file://\\servers\server\file.pdf " mid="lb870a25-4e98-45d8-a444-f0283a495357" args=""
/>
<resource name="foo" tyρe="information\documents\text document" reftyρe="nervana : ext" ref="" mid="7886e4a0-55d9-45ac-a084-97adc6fffd0f" args=
/>
<resource name="foo" type="information\documents\general document" reftype="nervana: cacheentry" ref="ef9c90ea-282d-46d6-b355-ac8a4fc2f3e5" mid="" args="" /> <reεource name="foo" type="information\email\email message" reftype="nervana:uri" ref="reguest : //email . all@ibm. com" mid="" args=""
/>
<resource name="foo" type="information\email\email annotation" reftype="nervana :uri" ref="objects ://rad. com/agency. asp" mid="" args="" /> <resource name="foo" type="information\documents\general document" reftype="nervana :uri" ref="objects ://rad. com/agency. asp" mid="" args=""
/> <resource name="foo" type="information\documents\general document" reftype="nervana :url" ref="objects ://rad. co /agency. asp" mid="" args=""
/> •^resource name="foo" type="information\documents\general document" ref type= "nervana : uri" ref="request : //documents . ll®intel . com" mid="" args^
/>
</resources> <links>
<link operator="and" predicates "nervan : relatedto " name="foo" type="information\documents\general document" ref type=" nervana': f ilepath" ref="file : //c : \foo. doc" mid="7886e4a0-55d9-45ac-a084-97adc6fffd0f"
/>
<link operator="and" predicate="nervana: contains" name="foo" type="information\documents\general document" reftype="nervan : text" ref="" mid="46ea76cb-1383-4885-af6f-0e0fcδa66896" args=""
/> <link operators»an " predicate="nervan :postedon" name="foo" tyρe="types\datetime" reftype="nervana: datetimeref" ref="" mid="3fa64c3c-4754-4380-91b5-521299036c62" args=""
/>
<link operators " and " predicate="nervan : relatedto" name= "foo" type="information\documents\general document" reftype="nervana : ri" ref="kisp : //98@in. com/m. asp" mid="c2649c39-alc3-4ca8-ae8d-c85c04372e9a" argss""
/> <link operators"and" predicates"nervana ;isof riority" name "foo" type " types\priority" reftype="nervana :priority" ref="" mid="69bbc048-98c8-4f76-8edf-5a00ce91cl83" args=" " /> </links> <metadata>
<metadataentry mid="7886e4a0-55d9-45ac-a084-97adc6fffd0f" reftype "uri" ref="file://c:\foo\bar.pdf" /> <value>
<document>
<title>εcenario modelling</title> <type>text</type> <format>application/pdf</format> <filepath>c-.\foo\bar.pdf</filepath> <shσrtfilename>bar.pdf</shortfilenanie> <creationtime>foo</creationtime> <lastmodifiedtime>foo</lastmodifiedtime> <lastaccessedtime>foo</lastaccessedtime> <attributes>0</attributes> <size>0</size> <subject>foo</subject> <creator>foo</creator> <manager>foo</manager> <companv> oo</company> <category>foo</category>
<keywords>foo</keywords>
<comments>foo</comments>
<hlinkbase>foo</hlinkbase>
<teτnplate>foo</template>
<lastsavedby>foo</lastsavedby>
<revisionnumber>0</revisionnumber>
<totaleditingtime>foo</totaleditingtime>
<numpages>0</numpages>
<numparagraphs>0</nu paragraphs>
<numlines>0</numlines>
<numwords>0</numwords>
<numcharacters>0</numcharacters>
<numcharacterswithspaces>0</numcharacterswithspaces>
<numbytes>0</numbytes> <1 nguage>foo</language> <version>foo</version> <abs ract>foo</abstract> </document
</value> /> <metadataentry mid="bfcbl2b4-70bb-473a-847c-ebffel87828f" reftype "uri" ref="file://c: \foo\bar.pdf" />
<value>
<email>
<title>scenario modelling</title> :type->text</type>
<format>application/pdf</format?
<filepath>c: \foo\bar.pdf</filepath>
<shortfilename>bar.pdf</shortfilename>
<creationtime>foo</creationtime>
<lastmodifiedtime>foo</lastmodifiedtime>
<lastaccessedtime>foo</lastaccessedtime>
<attributes>0</attributes>
<size>0</size>
<subject>foo</subject>
<creator>foo</creator>
<tnanager>foo</manager>
<company>foo</company>
<category>foo</category>
<keywords>foo</keywords>
<comments>foo</comments>
<hlinkbase>foo</hlinkbase>
<template>foo</template?
<lastsavedby>foo</lastsavedby>
<revisionnumber>0</revisionnumber>
<totaleditingtime>foo</totaleditingtime>
<πumpages>0</numpages>
<numparagraphs>0</numparagraphs>
<numlines>0</numlines>
<numwords>0</numwords>
<numcharacters>0</numcharacters>
<numcharacterswithspaces>0</numcharacterswithspaces> <numbytes>0</numbytes> <language>foo</language? <version>foo</version> <abs ract>foo</abstract? </email? </value?
/>
</metadata? </sq l?
Figure imgf000632_0001
SQML is generated in several ways:
• By creating a smart request
• By creating a local request o By creating an entity o By opening one or more local documents in the semantic browser
• By the client (dynamically) - in response to a drag and drop, smart copy and paste, intrinsic alert, context panel/link invocation, etc.
Figure imgf000632_0002
SQML that gets created on the client might not be ready (in real-time) for remote consumption - by the server's XML web service or at another machine site. This is especially the case when the SQML refers to local context like documents, entities, or smart requests (that are identified by unique identifiers in the semantic envinronment)1. In the preferred embodiment, the client always creates SQML that is ready for remote consumption. It does this by caching the metadata for all references in the metadata section of the document. This is critical because in some cases, the resource or object to which the reference points might no longer exist when the query is invoked. For instance, a user might drag and drop a document from the Internet to a smart request in order to generate a new relational request. The client extracts the metadata (including the summary) from the link and inserts the metadata into the SQML. Because the resolution of the query uses only the metadata, the query is ready for consumption once the metadata is inserted into the SQML document. However, the link that the object refers to might not exist the day after the user found it. In such a case, even if the user invokes the relational request after the link might have ceased to exist, the request will still work because the metadata would already have been cached in the SQML.
The client SQML parser performs lazy updating of metadata in the SQML. When the request is invoked, it attempts to update the metadata of all parameters (resources, etc.) in the SQML to handle the case where the objects might have changed since they were used to create the relational request. If the object does not exist, the client uses the metadata it already has. Else it updates it and uses the updated metadata. That way, even if the object has been deleted, the user experience does not get interrupted until the user actually tries to open the object from whence the metadata came.
Blenders (or collections) contain references to smart requests Es titiies Specification
Entities Specification for the Information Nervous System
By: Nosa Omoigui
Figure imgf000634_0001
Overview
Entities are a very powerful feature of the Information Nervous System. Entities allow the user to create a contextual definition that maps to how they work on a regular basis. Examples of entities include:
Action Items
2. Companies
3. Competitors
4. Customers
5. Meetings
6. Organizations
7. Partners
8. Products
9. Projects
10. Topics
There are also industry-specific entities. For instance, in pharmaceuticals, entities could include drugs, drug interaction issues, patents, FDA clinical trials, etc. Essentially, an entity is a semantic envelope that is a smart contextual object. An entity can be dragged and dropped like any other smart object. However, an entity is represented by SQML and not SRML (i.e., it is a query-object because it has much richer semantics). An entity can be included as a parameter to a smart request.
The user creates entities based on his/her tasks. Entities contain the following information:
1. Name/Description - a friendly descriptive name for the entity
2. The categories of the entity - based on a standard cross-industry taxonomy
3. Contextual resources - these could include keywords, local documents, Internet documents, or smart objects (such as people).
An entity can be opened in the semantic browser, can be used as a pivot for navigation, as a parameter for a smart request (e.g., Headlines on My Project), can be dragged and dropped, can be copied and pasted, can be used with the smart lens, can be visualized with a smart style, can be used as the basis for an intrinsic alert, can be saved as a .ENT document, can be emailed, shared, etc. In other words, an entity is a first-class smart object.
The semantic runtime client dynamically creates SQML by appending the rich metadata of the entity to the subject of the relational request to create a new rich SQML that refers to the entity. Portfolios
Portfolios are a special type of entity that contains a collection of entities. To minimize complexity, a portfolio cannot contain portfolios. A portfolio allows the user to manage a group of entities as one unit. A portfolio is a first-class entity and as such has all the aforementioned features of an entity. When a portfolio is used as a parameter in a smart request, the OR qualifier is applied (by default) to its containing entities. In other words, if Portfolio P contains entities E1 and E2, a smart request titled 'Headlines on P' will be processed as 'Headlines on E1 or E2.' The user can change this setting on individual smart requests (to an AND qualifier).
Sample Scenarios
1. A pharmaceuticals 'patent' entity could include the categories of the patent, relevant keywords, and relevant documents.
2. A CIA agent could create a 'terrorist' entity to track terrorists. This could include categories on terrorism, suspicious wire transfers, suspicious arms sales, classified documents, keywords, and terrorism experts in the information community.
3. Find All Breaking News on Yesterday's Meeting
4. Find Headlines on any of my competitors (this is done by creating the competitor entities, and then creating a smart request with the entities as parameters using the OR qualifier with each predicate).
5. Find Experts on my investment portfolio companies (create the individual entities, create a portfolio continining these entities and then create a smart request that has the "Experts' context template and that uses the portfolio as an argument).
Illustration
The figure below shows entity views displayed in the semantic browser (on the left).
Figure imgf000636_0001
nm
Eκtensibl€i Cltent-Side User Profiles Specϊfføation
Extensible Client-Side User Profiles Specification for the Information Nervous
System
ϊy: Nosa Omoigui
ΓWHΓTE™
Overview
Extensible client-side user profiles allow the user of a semantic browser to have different state for different job roles, identities, personas, work styles, etc. For instance, the user might create a profile for Work' and one for 'Home.' Many investment analysts track companies across a variety of industries. With the semantic browser, they would create profiles for each industry they track. Consultants move from project to project (and from industry to industry) and might want to save requests and entities created with each project. Profiles will be used to handle this scenario as well.
Profiles contain the following user state:
•• . Name/Description - the descriptive name of the profile
« Identity Information - the user name (currently tagged with the user's email address) and password o Areas of Interest or Favorite Categories - this is used to suggest information communities (agencies) to the user (by comparing against information communities with identical or similar categories) and as a default query filter for requests created with the profile
• Smart styles - the smart styles to be used by default for requests and entities created with the profile.
• Default Flag - this indicates whether the profile is the default profile. The default profile is initiated by default when the user wishes to create requests and entities, browse information communities, etc. Unless the user explicitly selects a different profile, the default profile gets used.
Profiles can be created, deleted, modified, and renamed.
Figure imgf000639_0001
Information Community Browsing and Subscription vSpecification for the Information
Nervous System
By: Nosa Omoigui
rTOτTrτ?g fϊτπ
Overview
The Nervana semantic browser will allow the user to subscribe and unsubscribe to/from information communities (agencies) for a given profile. These information communities will be readily available to the user underneath the profile entry in the semantic environment. In addition, these information communities will be queried by default for intrinsic alerts, context panels, and etc. whenever results are displayed for any request created using the same profile.
The semantic environment includes state indicating the subscribed information communities for each profile. The client-side semantic query processor (SQP) uses this information for dynamic requests that start from results for requests of a given profile (the SQP will ask the semantic runtime client for the information communities for the profile and then issue XML Web Service calls to those information communities as appropriate).
Figure 1 below shows the user interface for the information community subscription and un- subscription. The dialog box has combo boxes allowing the user to filter by profile, to view all, new1, subscribed, suggested, and un-subscribed communities, by industry and area of interest, by keywords, by publishing point (all publishing points, the local area network, the enterprise directory, and the global information community directory), and by creation time (anytime, today, yesterday, this week, and last week). The semantic runtime client queries the publishing point endpoint listeners (for each publishing point) using the filters. It then gathers the results and displays them in the results pane. The user is also able to view the categories of each information community in the results pane via a combo box.
' 'New1 is implemented by checking if there are new information communities that have been published since the last time the system checked for a given profile - each profile contains state for the 'last checked time' the publishing point listeners were queried.
Figure imgf000641_0001
Figure 1: Information Communities Dialog Box IP F fr67 -
Figure imgf000642_0001
Figure imgf000643_0001
Figure imgf000644_0001
Figure imgf000645_0001
644
Figure imgf000646_0001
Figure imgf000647_0001
Figure imgf000648_0001
Figure imgf000649_0001
Figure imgf000650_0001
Figure imgf000651_0001
Figure imgf000652_0001
Figure imgf000653_0001
Figure imgf000654_0001
Figure imgf000655_0001
Figure imgf000656_0001
Figure imgf000657_0001
Figure imgf000658_0001
Figure imgf000659_0001
Figure imgf000660_0001
Figure imgf000661_0001
Figure imgf000662_0001
Figure imgf000663_0001
Figure imgf000664_0001
Figure imgf000665_0002
Figure imgf000665_0003
Figure imgf000665_0001
Figure imgf000666_0001
665
Figure imgf000667_0001
666
Figure imgf000668_0001
Figure imgf000669_0001
Figure imgf000670_0001
Figure imgf000671_0001
Figure imgf000672_0001
Figure imgf000673_0001
ϊιffl«-iSai¥!irøff--aαiώi':- »i: -UJ«1 ό- -.r'! _ :»-IJ| ~'i -,'liι X~α~:fτ""-'Il_rD'!'Q'"5 "
Figure imgf000674_0001
- S*«
Figure imgf000675_0001
Figure imgf000676_0001
Figure imgf000677_0001
Figure imgf000678_0001

Claims

CLAIMSThe embodiments of the invention in which an exclusive property or privilege is claimed are defined as follows:
1. A system for knowledge retrieval, management, delivery and presentation, comprising: a server programmable to maintain semantic information; a client providing a user interface for a user to communicate with the server; and wherein the processor of the server operates to perform the steps of: securing information from information sources; semantic ascertaining one or more semantic properties of the information; and responding to user queries based upon one or more of the semantic properties.
2. The system of claim 1, wherein the first server further comprises structure or methodology directed to providing at least one of the following: a Semantic Network, a Semantic Data Gatherer, a Semantic Network Consistency Checker, an Inference Engine, a Semantic Query Processor, a Natural Language Parser, an Email Knowledge Agent, or a Knowledge Domain Manager.
3. The system of claim 1, wherein: the information comprises objects or events; and the semantic properties of the objects or events are represented by active agents for semantically linking to the semantics and properties of the queries.
4. A method for knowledge retrieval, management, delivery and presentation for use with a server system programmed to add, maintain and host domain specific information that is used to classify and categorize semantic information, comprising: securing information from information sources; semantically linking the information from the information sources; maintaining the semantic attributes of the semantically linked information; delivering requested semantic information based upon user queries; and presenting semantic information according to customizable user preferences.
PCT/US2004/004674 2001-06-22 2004-02-17 Semantic knowledge retrieval management and presentation WO2004075466A2 (en)

Priority Applications (10)

Application Number Priority Date Filing Date Title
AU2004213986A AU2004213986A1 (en) 2003-02-14 2004-02-17 Semantic knowledge retrieval management and presentation
EA200501304A EA200501304A1 (en) 2003-02-14 2004-02-17 SYSTEM FOR SEARCH, MANAGEMENT, COLLECTION, DETECTION, DELIVERY AND REPRESENTATION OF SEMANTIC KNOWLEDGE
MXPA05008670A MXPA05008670A (en) 2003-02-14 2004-02-17 Semantic knowledge retrieval management and presentation.
BR0407451-3A BRPI0407451A (en) 2003-02-14 2004-02-17 System and method for knowledge retrieval, semantics, management and presentation
JP2006503648A JP2006522388A (en) 2003-02-14 2004-02-17 Systems and methods for acquiring, managing, capturing, sharing, discovering, communicating and presenting semantic knowledge
CA002555280A CA2555280A1 (en) 2003-02-14 2004-02-17 Semantic knowledge retrieval management and presentation
EP04711863A EP1599811A4 (en) 2003-02-14 2004-02-17 System and method for semantic knowledge retrieval, management, capture, sharing, discovery, delivery and presentation
US11/931,793 US20080147788A1 (en) 2001-06-22 2007-10-31 Information nervous system
US12/358,224 US20100070448A1 (en) 2002-06-24 2009-01-22 System and method for knowledge retrieval, management, delivery and presentation
US13/168,785 US20120191716A1 (en) 2002-06-24 2011-06-24 System and method for knowledge retrieval, management, delivery and presentation

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US44773603P 2003-02-14 2003-02-14
US60/447,736 2003-02-14

Publications (2)

Publication Number Publication Date
WO2004075466A2 true WO2004075466A2 (en) 2004-09-02
WO2004075466A3 WO2004075466A3 (en) 2004-10-28

Family

ID=32908491

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2004/004674 WO2004075466A2 (en) 2001-06-22 2004-02-17 Semantic knowledge retrieval management and presentation

Country Status (10)

Country Link
EP (1) EP1599811A4 (en)
JP (1) JP2006522388A (en)
KR (1) KR20060004909A (en)
CN (1) CN1853180A (en)
AU (1) AU2004213986A1 (en)
BR (1) BRPI0407451A (en)
CA (1) CA2555280A1 (en)
EA (1) EA200501304A1 (en)
MX (1) MXPA05008670A (en)
WO (1) WO2004075466A2 (en)

Cited By (43)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008016499A1 (en) * 2006-07-31 2008-02-07 Microsoft Corporation Presenting information related to topics extracted from event classes
US7546295B2 (en) 2005-12-27 2009-06-09 Baynote, Inc. Method and apparatus for determining expertise based upon observed usage patterns
WO2010025390A2 (en) * 2008-08-29 2010-03-04 The Administrators Of The Tulane Educational Fund Copyright status determination system and method
US7698270B2 (en) 2004-12-29 2010-04-13 Baynote, Inc. Method and apparatus for identifying, extracting, capturing, and leveraging expertise and knowledge
WO2010089248A1 (en) 2009-02-03 2010-08-12 International Business Machines Corporation Method and system for semantic searching
WO2011008862A2 (en) * 2009-07-14 2011-01-20 Zoomii, Inc. Markup language-based authoring and runtime environment for interactive content platform
US7921067B2 (en) 2006-09-04 2011-04-05 Sony Deutschland Gmbh Method and device for mood detection
US7930263B2 (en) 2007-01-12 2011-04-19 Health Information Flow, Inc. Knowledge utilization
WO2011160204A1 (en) * 2010-06-22 2011-12-29 Primal Fusion Inc. Methods and apparatus for searching of content using semantic synthesis
WO2011163384A2 (en) * 2010-06-22 2011-12-29 Eyewonder Inc. Dynamic creative creation & delivery
US8095523B2 (en) 2004-12-29 2012-01-10 Baynote, Inc. Method and apparatus for context-based content recommendation
US8204856B2 (en) 2007-03-15 2012-06-19 Google Inc. Database replication
US8495147B1 (en) 2006-07-13 2013-07-23 Avaya Inc. Threading of mixed media
US8495001B2 (en) 2008-08-29 2013-07-23 Primal Fusion Inc. Systems and methods for semantic concept definition and semantic concept relationship synthesis utilizing existing domain definitions
US8510302B2 (en) 2006-08-31 2013-08-13 Primal Fusion Inc. System, method, and computer program for a consumer defined information architecture
US8638319B2 (en) 2007-05-29 2014-01-28 Livescribe Inc. Customer authoring tools for creating user-generated content for smart pen applications
US8676732B2 (en) 2008-05-01 2014-03-18 Primal Fusion Inc. Methods and apparatus for providing information of interest to one or more users
US8676722B2 (en) 2008-05-01 2014-03-18 Primal Fusion Inc. Method, system, and computer program for user-driven dynamic generation of semantic networks and media synthesis
US8849860B2 (en) 2005-03-30 2014-09-30 Primal Fusion Inc. Systems and methods for applying statistical inference techniques to knowledge representations
US20150200875A1 (en) * 2013-01-16 2015-07-16 Boris Khvostichenko Double filtering of annotations in emails
US9092516B2 (en) 2011-06-20 2015-07-28 Primal Fusion Inc. Identifying information of interest based on user preferences
US9098502B1 (en) 2012-11-16 2015-08-04 Google Inc. Identifying documents for dissemination by an entity
US9104779B2 (en) 2005-03-30 2015-08-11 Primal Fusion Inc. Systems and methods for analyzing and synthesizing complex knowledge representations
US9177248B2 (en) 2005-03-30 2015-11-03 Primal Fusion Inc. Knowledge representation systems and methods incorporating customization
US9235806B2 (en) 2010-06-22 2016-01-12 Primal Fusion Inc. Methods and devices for customizing knowledge representation systems
US9262520B2 (en) 2009-11-10 2016-02-16 Primal Fusion Inc. System, method and computer program for creating and manipulating data structures using an interactive graphical interface
US9292855B2 (en) 2009-09-08 2016-03-22 Primal Fusion Inc. Synthesizing messaging using context provided by consumers
US9378203B2 (en) 2008-05-01 2016-06-28 Primal Fusion Inc. Methods and apparatus for providing information of interest to one or more users
US9524506B2 (en) 2011-10-21 2016-12-20 Bigmachines, Inc. Methods and apparatus for maintaining business rules in a configuration system
US9836765B2 (en) 2014-05-19 2017-12-05 Kibo Software, Inc. System and method for context-aware recommendation through user activity change detection
US10002335B2 (en) 2011-01-06 2018-06-19 Cardinal Logistics Management Corporation Dynamic workflow for remote devices
US10002325B2 (en) 2005-03-30 2018-06-19 Primal Fusion Inc. Knowledge representation systems and methods incorporating inference rules
US10223637B1 (en) 2013-05-30 2019-03-05 Google Llc Predicting accuracy of submitted data
US10248669B2 (en) 2010-06-22 2019-04-02 Primal Fusion Inc. Methods and devices for customizing knowledge representation systems
US10372815B2 (en) 2013-07-12 2019-08-06 Microsoft Technology Licensing, Llc Interactive concept editing in computer-human interactive learning
WO2021041998A3 (en) * 2019-08-29 2021-04-08 Fraudmarc Inc. Low-latency, outbound message monitoring, control, and authentication
TWI737857B (en) * 2016-11-18 2021-09-01 日商倍樂生思泰服務股份有限公司 Service supporting system, service supporting method, and computer program
WO2021260684A1 (en) * 2020-06-21 2021-12-30 Avivi Eliahu Kadoori System and method for detection and auto-validation of key data in any non-handwritten document
US11294977B2 (en) 2011-06-20 2022-04-05 Primal Fusion Inc. Techniques for presenting content to a user based on the user's preferences
US11483413B1 (en) * 2019-06-04 2022-10-25 Thomas Layne Bascom Federation broker system and method for coordinating discovery, interoperability, connections and correspondence among networked resources
WO2022272188A1 (en) * 2021-06-22 2022-12-29 Ravindra Pratap Singh Devices and systems that measure, quantify, compare, package, and capture human content in databases
WO2023278682A1 (en) * 2021-06-30 2023-01-05 Earley Information Science, Inc. Multi-bot digital content retrieval and generation systems
WO2023278639A1 (en) * 2021-06-30 2023-01-05 Earley Information Science, Inc. Digital data processing systems and methods for multi-domain digital content retrieval and generation with dead-end prevention

Families Citing this family (57)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100882582B1 (en) * 2006-12-20 2009-02-12 한국과학기술정보연구원 System and method for research information service based on semantic web
EP2122487A4 (en) * 2007-01-16 2011-02-02 Timmins Software Corp Systems and methods for analyzing information technology systems using collaborative intelligence
EP2129120A4 (en) * 2007-01-22 2010-02-03 Sony Corp Information processing device and method, and program
KR100913441B1 (en) * 2007-08-20 2009-09-03 한국과학기술원 Method for Searching Semantic Resource Using Semantic Space Mapping of Resource
KR100918503B1 (en) * 2008-01-21 2009-09-24 윤재민 Method and System for Providing Web Page Information Using Website Link Information
US9589381B2 (en) 2008-06-12 2017-03-07 Microsoft Technology Licensing, Llc Copying of animation effects from a source object to at least one target object
CA2639438A1 (en) 2008-09-08 2010-03-08 Semanti Inc. Semantically associated computer search index, and uses therefore
KR101072147B1 (en) * 2009-02-10 2011-10-10 경북대학교 산학협력단 Method and system for transforming blog posting to Ontology-based information
US20110178860A1 (en) * 2010-01-15 2011-07-21 Imrey G Christopher System and method for resolving transactions employing goal seeking attributes
US20110184945A1 (en) * 2010-01-22 2011-07-28 Qualcomm Incorporated Location aware recommendation engine
KR101146510B1 (en) * 2010-05-20 2012-05-25 소프트포럼 주식회사 System for encrypting synchronization database and method therefor
CN102737052A (en) * 2011-04-12 2012-10-17 国际商业机器公司 Method and system for processing input
CN102185917A (en) * 2011-04-29 2011-09-14 深圳市五巨科技有限公司 Method and system for adaptation between server and mobile terminal, and server adaptation device
CN102609819A (en) * 2012-02-16 2012-07-25 华为软件技术有限公司 Information exhibiting system
KR101445218B1 (en) * 2012-05-23 2014-09-30 이청종 Method for collaborative profit share for contents
JP5962256B2 (en) * 2012-06-29 2016-08-03 カシオ計算機株式会社 Input support apparatus and input support program
US9270667B2 (en) * 2012-11-01 2016-02-23 Microsoft Technology Licensing, Llc Utilizing X.509 authentication for single sign-on between disparate servers
KR101503463B1 (en) * 2013-09-25 2015-03-19 목포대학교산학협력단 Internal monitoring of vessels using the weighted average
CN103729112B (en) * 2014-01-10 2018-03-06 百度在线网络技术（北京）有限公司 The method and apparatus of display information
CN105095320B (en) 2014-05-23 2019-04-19 邓寅生 The mark of document based on relationship stack combinations, association, the system searched for and showed
WO2016049582A1 (en) * 2014-09-25 2016-03-31 Oracle International Corporation System and method for determination of partition identifiers in a multitenant application server environment
CN104639987B (en) * 2015-01-29 2018-12-14 联发科技（新加坡）私人有限公司 Controlled terminal and its information providing method
CN106326300A (en) * 2015-07-02 2017-01-11 富士通株式会社 Information processing method and information processing device
CN105141668B (en) * 2015-07-31 2018-09-25 中冶南方工程技术有限公司 A kind of method of data synchronization based on distributed multi agent
US10042619B2 (en) 2015-08-25 2018-08-07 Cognizant Technology Solutions India Pvt. Ltd. System and method for efficiently managing enterprise architecture using resource description framework
CN106886543B (en) * 2015-12-16 2020-01-17 清华大学 Knowledge graph representation learning method and system combined with entity description
CN106960271A (en) * 2016-02-29 2017-07-18 艾威梯科技(北京)有限公司 One kind cooperates and method of quality control and system
US11403418B2 (en) * 2018-08-30 2022-08-02 Netskope, Inc. Enriching document metadata using contextual information
CN105740471A (en) * 2016-03-14 2016-07-06 燕山大学 Intelligent method for dynamically querying paper collection states
US10404835B2 (en) * 2016-03-17 2019-09-03 Google Llc Hybrid client-server data provision
US10083451B2 (en) 2016-07-08 2018-09-25 Asapp, Inc. Using semantic processing for customer support
WO2018009231A1 (en) 2016-07-08 2018-01-11 Asapp, Inc. Automatically responding to a request of a user
KR101869618B1 (en) * 2016-12-29 2018-06-20 (주) 쓰리웨어 Personalized news contents providing system
KR101904643B1 (en) 2017-02-09 2018-10-05 한국기술교육대학교 산학협력단 Article generation method using the decision tree
JP6667876B2 (en) * 2017-04-26 2020-03-18 サイレックス・テクノロジー株式会社 Base station, base station system, and communication method
KR102457568B1 (en) * 2017-09-29 2022-10-21 삼성전자주식회사 Electronic device for providing image associated with input information and operating method thereof
CN107862081B (en) * 2017-11-29 2021-07-16 四川无声信息技术有限公司 Network information source searching method and device and server
CN109299400A (en) * 2018-09-06 2019-02-01 北京奇艺世纪科技有限公司 A kind of viewpoint abstracting method, device and equipment
CN109256029B (en) * 2018-09-12 2021-09-03 广州小鹏汽车科技有限公司 Automatic setting method and device for location attribute
CN109492208B (en) * 2018-10-12 2023-06-23 天津字节跳动科技有限公司 Document editing method and device, equipment and storage medium thereof
CN111049723B (en) * 2018-10-15 2022-11-08 广州虎牙信息科技有限公司 Message pushing method, message management system, server and computer storage medium
US11630824B2 (en) * 2018-10-16 2023-04-18 Shimadzu Corporation Document search method and document search system
CN109858036B (en) * 2019-02-26 2023-07-28 科大讯飞股份有限公司 Method and device for dividing documents
CN110048936B (en) * 2019-04-18 2021-09-10 宁波青年优品信息科技有限公司 Method for judging junk mail by semantic associated words
CN110222017B (en) * 2019-05-13 2021-09-21 北京百度网讯科技有限公司 Real-time data processing method, device and equipment and computer readable storage medium
CN110347839B (en) * 2019-07-18 2021-07-16 湖南数定智能科技有限公司 Text classification method based on generative multi-task learning model
CN110727760B (en) * 2019-09-08 2023-11-07 天津大学 Method for carrying out distributed regular path query on large-scale knowledge graph
CN110781685B (en) * 2019-10-18 2022-08-19 四川长虹电器股份有限公司 Method for automatically marking correctness of semantic analysis result based on user feedback
CN110728389B (en) * 2019-10-21 2023-11-10 中国民航信息网络股份有限公司 Air ticket order processing method, device and system crossing reservation system
CN111209028B (en) * 2020-01-07 2021-10-01 京东数字科技控股有限公司 Data processing method and device, electronic equipment and storage medium
CN111392296B (en) * 2020-04-14 2021-03-09 重庆市环卫集团有限公司 A wisdom is received and is transported system for rubbish transfer station
US11720346B2 (en) 2020-10-02 2023-08-08 International Business Machines Corporation Semantic code retrieval using graph matching
KR102542394B1 (en) 2020-12-09 2023-06-13 주식회사 이앤지테크 Customized travel scheduling virtual reality platform system and Device method of the Same
CN113312656B (en) * 2021-07-29 2022-04-15 阿里云计算有限公司 Data rotation method, device, equipment and system
CN113657121B (en) * 2021-09-03 2023-04-07 四川大学 Log variable semantic annotation method
CN114398442B (en) * 2022-01-25 2023-09-19 中国电子科技集团公司第十研究所 Information processing system based on data driving
CN115357581B (en) * 2022-08-19 2023-05-05 筑智建科技(重庆)有限公司 Distributed storage method for massive BIM data

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030126136A1 (en) 2001-06-22 2003-07-03 Nosa Omoigui System and method for knowledge retrieval, management, delivery and presentation

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6240407B1 (en) * 1998-04-29 2001-05-29 International Business Machines Corp. Method and apparatus for creating an index in a database system
US6418448B1 (en) * 1999-12-06 2002-07-09 Shyam Sundar Sarkar Method and apparatus for processing markup language specifications for data and metadata used inside multiple related internet documents to navigate, query and manipulate information from a plurality of object relational databases over the web
US6311194B1 (en) * 2000-03-15 2001-10-30 Taalee, Inc. System and method for creating a semantic web and its applications in browsing, searching, profiling, personalization and advertising

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030126136A1 (en) 2001-06-22 2003-07-03 Nosa Omoigui System and method for knowledge retrieval, management, delivery and presentation

Cited By (76)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7698270B2 (en) 2004-12-29 2010-04-13 Baynote, Inc. Method and apparatus for identifying, extracting, capturing, and leveraging expertise and knowledge
US8095523B2 (en) 2004-12-29 2012-01-10 Baynote, Inc. Method and apparatus for context-based content recommendation
US8601023B2 (en) 2004-12-29 2013-12-03 Baynote, Inc. Method and apparatus for identifying, extracting, capturing, and leveraging expertise and knowledge
US7702690B2 (en) 2004-12-29 2010-04-20 Baynote, Inc. Method and apparatus for suggesting/disambiguation query terms based upon usage patterns observed
US9904729B2 (en) 2005-03-30 2018-02-27 Primal Fusion Inc. System, method, and computer program for a consumer defined information architecture
US10002325B2 (en) 2005-03-30 2018-06-19 Primal Fusion Inc. Knowledge representation systems and methods incorporating inference rules
US8849860B2 (en) 2005-03-30 2014-09-30 Primal Fusion Inc. Systems and methods for applying statistical inference techniques to knowledge representations
US9177248B2 (en) 2005-03-30 2015-11-03 Primal Fusion Inc. Knowledge representation systems and methods incorporating customization
US9104779B2 (en) 2005-03-30 2015-08-11 Primal Fusion Inc. Systems and methods for analyzing and synthesizing complex knowledge representations
US9934465B2 (en) 2005-03-30 2018-04-03 Primal Fusion Inc. Systems and methods for analyzing and synthesizing complex knowledge representations
US7856446B2 (en) 2005-12-27 2010-12-21 Baynote, Inc. Method and apparatus for determining usefulness of a digital asset
US7693836B2 (en) 2005-12-27 2010-04-06 Baynote, Inc. Method and apparatus for determining peer groups based upon observed usage patterns
US7580930B2 (en) 2005-12-27 2009-08-25 Baynote, Inc. Method and apparatus for predicting destinations in a navigation context based upon observed usage patterns
US7546295B2 (en) 2005-12-27 2009-06-09 Baynote, Inc. Method and apparatus for determining expertise based upon observed usage patterns
US8495147B1 (en) 2006-07-13 2013-07-23 Avaya Inc. Threading of mixed media
WO2008016499A1 (en) * 2006-07-31 2008-02-07 Microsoft Corporation Presenting information related to topics extracted from event classes
US8510302B2 (en) 2006-08-31 2013-08-13 Primal Fusion Inc. System, method, and computer program for a consumer defined information architecture
US7921067B2 (en) 2006-09-04 2011-04-05 Sony Deutschland Gmbh Method and device for mood detection
US7930263B2 (en) 2007-01-12 2011-04-19 Health Information Flow, Inc. Knowledge utilization
US8204856B2 (en) 2007-03-15 2012-06-19 Google Inc. Database replication
US8842100B2 (en) 2007-05-29 2014-09-23 Livescribe Inc. Customer authoring tools for creating user-generated content for smart pen applications
US8638319B2 (en) 2007-05-29 2014-01-28 Livescribe Inc. Customer authoring tools for creating user-generated content for smart pen applications
US9792550B2 (en) 2008-05-01 2017-10-17 Primal Fusion Inc. Methods and apparatus for providing information of interest to one or more users
US8676722B2 (en) 2008-05-01 2014-03-18 Primal Fusion Inc. Method, system, and computer program for user-driven dynamic generation of semantic networks and media synthesis
US9378203B2 (en) 2008-05-01 2016-06-28 Primal Fusion Inc. Methods and apparatus for providing information of interest to one or more users
US8676732B2 (en) 2008-05-01 2014-03-18 Primal Fusion Inc. Methods and apparatus for providing information of interest to one or more users
US11182440B2 (en) 2008-05-01 2021-11-23 Primal Fusion Inc. Methods and apparatus for searching of content using semantic synthesis
US9361365B2 (en) 2008-05-01 2016-06-07 Primal Fusion Inc. Methods and apparatus for searching of content using semantic synthesis
US11868903B2 (en) 2008-05-01 2024-01-09 Primal Fusion Inc. Method, system, and computer program for user-driven dynamic generation of semantic networks and media synthesis
US10803107B2 (en) 2008-08-29 2020-10-13 Primal Fusion Inc. Systems and methods for semantic concept definition and semantic concept relationship synthesis utilizing existing domain definitions
US8943016B2 (en) 2008-08-29 2015-01-27 Primal Fusion Inc. Systems and methods for semantic concept definition and semantic concept relationship synthesis utilizing existing domain definitions
US8495001B2 (en) 2008-08-29 2013-07-23 Primal Fusion Inc. Systems and methods for semantic concept definition and semantic concept relationship synthesis utilizing existing domain definitions
WO2010025390A3 (en) * 2008-08-29 2010-06-17 The Administrators Of The Tulane Educational Fund Copyright status determination system and method
US9595004B2 (en) 2008-08-29 2017-03-14 Primal Fusion Inc. Systems and methods for semantic concept definition and semantic concept relationship synthesis utilizing existing domain definitions
CN102224504A (en) * 2008-08-29 2011-10-19 图兰恩教育基金管理人 Copyright status determination system and method
WO2010025390A2 (en) * 2008-08-29 2010-03-04 The Administrators Of The Tulane Educational Fund Copyright status determination system and method
WO2010089248A1 (en) 2009-02-03 2010-08-12 International Business Machines Corporation Method and system for semantic searching
WO2011008862A2 (en) * 2009-07-14 2011-01-20 Zoomii, Inc. Markup language-based authoring and runtime environment for interactive content platform
WO2011008862A3 (en) * 2009-07-14 2011-05-05 Zoomii, Inc. Markup language-based authoring and runtime environment for interactive content platform
US9292855B2 (en) 2009-09-08 2016-03-22 Primal Fusion Inc. Synthesizing messaging using context provided by consumers
US10181137B2 (en) 2009-09-08 2019-01-15 Primal Fusion Inc. Synthesizing messaging using context provided by consumers
US9262520B2 (en) 2009-11-10 2016-02-16 Primal Fusion Inc. System, method and computer program for creating and manipulating data structures using an interactive graphical interface
US10146843B2 (en) 2009-11-10 2018-12-04 Primal Fusion Inc. System, method and computer program for creating and manipulating data structures using an interactive graphical interface
WO2011163384A2 (en) * 2010-06-22 2011-12-29 Eyewonder Inc. Dynamic creative creation & delivery
US11474979B2 (en) 2010-06-22 2022-10-18 Primal Fusion Inc. Methods and devices for customizing knowledge representation systems
US9235806B2 (en) 2010-06-22 2016-01-12 Primal Fusion Inc. Methods and devices for customizing knowledge representation systems
US9576241B2 (en) 2010-06-22 2017-02-21 Primal Fusion Inc. Methods and devices for customizing knowledge representation systems
WO2011163384A3 (en) * 2010-06-22 2012-05-03 Eyewonder Inc. Dynamic creative creation & delivery
US10474647B2 (en) 2010-06-22 2019-11-12 Primal Fusion Inc. Methods and devices for customizing knowledge representation systems
US10248669B2 (en) 2010-06-22 2019-04-02 Primal Fusion Inc. Methods and devices for customizing knowledge representation systems
US10140619B2 (en) 2010-06-22 2018-11-27 Sizmek Technologies, Inc. Dynamic creative creation and delivery
WO2011160204A1 (en) * 2010-06-22 2011-12-29 Primal Fusion Inc. Methods and apparatus for searching of content using semantic synthesis
US10002335B2 (en) 2011-01-06 2018-06-19 Cardinal Logistics Management Corporation Dynamic workflow for remote devices
US11294977B2 (en) 2011-06-20 2022-04-05 Primal Fusion Inc. Techniques for presenting content to a user based on the user's preferences
US9092516B2 (en) 2011-06-20 2015-07-28 Primal Fusion Inc. Identifying information of interest based on user preferences
US9715552B2 (en) 2011-06-20 2017-07-25 Primal Fusion Inc. Techniques for presenting content to a user based on the user's preferences
US10409880B2 (en) 2011-06-20 2019-09-10 Primal Fusion Inc. Techniques for presenting content to a user based on the user's preferences
US9098575B2 (en) 2011-06-20 2015-08-04 Primal Fusion Inc. Preference-guided semantic processing
US9524506B2 (en) 2011-10-21 2016-12-20 Bigmachines, Inc. Methods and apparatus for maintaining business rules in a configuration system
US9098502B1 (en) 2012-11-16 2015-08-04 Google Inc. Identifying documents for dissemination by an entity
US10439969B2 (en) * 2013-01-16 2019-10-08 Google Llc Double filtering of annotations in emails
US20150200875A1 (en) * 2013-01-16 2015-07-16 Boris Khvostichenko Double filtering of annotations in emails
US11526773B1 (en) 2013-05-30 2022-12-13 Google Llc Predicting accuracy of submitted data
US10223637B1 (en) 2013-05-30 2019-03-05 Google Llc Predicting accuracy of submitted data
US10372815B2 (en) 2013-07-12 2019-08-06 Microsoft Technology Licensing, Llc Interactive concept editing in computer-human interactive learning
US11023677B2 (en) 2013-07-12 2021-06-01 Microsoft Technology Licensing, Llc Interactive feature selection for training a machine learning system and displaying discrepancies within the context of the document
US9836765B2 (en) 2014-05-19 2017-12-05 Kibo Software, Inc. System and method for context-aware recommendation through user activity change detection
TWI737857B (en) * 2016-11-18 2021-09-01 日商倍樂生思泰服務股份有限公司 Service supporting system, service supporting method, and computer program
US11483413B1 (en) * 2019-06-04 2022-10-25 Thomas Layne Bascom Federation broker system and method for coordinating discovery, interoperability, connections and correspondence among networked resources
US11063986B2 (en) 2019-08-29 2021-07-13 Fraudmarc Inc. Low-latency, outbound message monitoring, control, and authentication
US11805151B2 (en) 2019-08-29 2023-10-31 Fraudmarc Inc. Low-latency, outbound message monitoring, control, and authentication
WO2021041998A3 (en) * 2019-08-29 2021-04-08 Fraudmarc Inc. Low-latency, outbound message monitoring, control, and authentication
WO2021260684A1 (en) * 2020-06-21 2021-12-30 Avivi Eliahu Kadoori System and method for detection and auto-validation of key data in any non-handwritten document
WO2022272188A1 (en) * 2021-06-22 2022-12-29 Ravindra Pratap Singh Devices and systems that measure, quantify, compare, package, and capture human content in databases
WO2023278682A1 (en) * 2021-06-30 2023-01-05 Earley Information Science, Inc. Multi-bot digital content retrieval and generation systems
WO2023278639A1 (en) * 2021-06-30 2023-01-05 Earley Information Science, Inc. Digital data processing systems and methods for multi-domain digital content retrieval and generation with dead-end prevention

Also Published As

Publication number Publication date
CA2555280A1 (en) 2004-09-02
BRPI0407451A (en) 2006-02-07
EA200501304A1 (en) 2006-08-25
MXPA05008670A (en) 2005-11-17
EP1599811A4 (en) 2008-02-06
JP2006522388A (en) 2006-09-28
WO2004075466A3 (en) 2004-10-28
KR20060004909A (en) 2006-01-16
EP1599811A2 (en) 2005-11-30
CN1853180A (en) 2006-10-25
AU2004213986A1 (en) 2004-09-02

Similar Documents

Publication Publication Date Title
WO2004075466A2 (en) Semantic knowledge retrieval management and presentation
US20030126136A1 (en) System and method for knowledge retrieval, management, delivery and presentation
US20100070448A1 (en) System and method for knowledge retrieval, management, delivery and presentation
US20070081197A1 (en) System and method for semantic knowledge retrieval, management, capture, sharing, discovery, delivery and presentation
US7433876B2 (en) Semantic web portal and platform
Dzbor et al. Magpie–towards a semantic web browser
CA2808275C (en) Distributed computing services platform
US8700738B2 (en) Dynamic feed generation
US20070016563A1 (en) Information nervous system
US20080195483A1 (en) Widget management systems and advertising systems related thereto
US20080126178A1 (en) Surge-Based Online Advertising
US20080052343A1 (en) Usage-Based Prioritization
US20080052162A1 (en) Calendar-Based Advertising
AU2001268674A1 (en) Distributed computing services platform
KR100919606B1 (en) Distributed computing services platform
US20160077727A1 (en) Online Protocol Community
AU2002345906A1 (en) System and method for knowledge retrieval, management, delivery and presentation

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): BW GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
WWE Wipo information: entry into national phase

Ref document number: PA/a/2005/008670

Country of ref document: MX

WWE Wipo information: entry into national phase

Ref document number: 2006503648

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 1020057015062

Country of ref document: KR

WWE Wipo information: entry into national phase

Ref document number: 2004711863

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2004213986

Country of ref document: AU

Ref document number: 2255/CHENP/2005

Country of ref document: IN

Ref document number: 200501304

Country of ref document: EA

ENP Entry into the national phase

Ref document number: 2004213986

Country of ref document: AU

Date of ref document: 20040217

Kind code of ref document: A

WWP Wipo information: published in national office

Ref document number: 2004213986

Country of ref document: AU

WWE Wipo information: entry into national phase

Ref document number: 20048099806

Country of ref document: CN

WWP Wipo information: published in national office

Ref document number: 2004711863

Country of ref document: EP

WWP Wipo information: published in national office

Ref document number: 1020057015062

Country of ref document: KR

ENP Entry into the national phase

Ref document number: PI0407451

Country of ref document: BR

WWE Wipo information: entry into national phase

Ref document number: 2555280

Country of ref document: CA