W3C libwww Using

Request Callback functions

As we have seen in the previous chapters, the core part of the Library knows nothing about how to access, for example, a HTML document from a remote server. All this depends on what the application has registered. In many situations there are a number of things to do before a request is actually sent over the wire. For example we might already have the document in a cache or we might have some translation of the URL so that we don't go directly to the remote server. The latter case includes redirection of a request to go through a proxy server or a gateway. Likewise, when a request is terminated, we might want to keep a log of the request and the result, update history lists etc.

The Library does provide a large amount of such pre- and post processing modules. However, the exact amount used by an application depends on the purpose of the application. Simple script-like applications typically do not need any history mechanism etc. Therefore these modules are not a part of the core but instead they can be registered as all other preferences. The Net Manager provides functionality for registering a set of callback functions that can be called before and after a request has been executed. Of course, the result of a pre-processing might be that the request does not have to be executed at all in which case the request can be terminated before the protocol module is called to execute the request.

Generic Handling of Callbacks

The registration of callback functions is handled by the HTNet Manager and it is (of course) based on lists as we have seen so many times before. A callback function can be added to a list by using the following function: 
extern BOOL HTNetCall_add (HTList * list, HTNetCallback * cbf, int status);
The callback function has to be of type HTNetCallback which is defined as 
typedef int HTNetCallback (HTRequest * request, int result);
This means that a callback function is called with the current request object and the result of the request. Now, if the callback is registered as a pre callback then we obviously do not yet have a result and the functions is called with the code HT_OK. However, if it is a post callback function then the result code may take any of the following values:
HT_ERROR
An error occured
HT_INTERRUPTED
The request was interrupted
HT_LOADED
The document was loaded
HT_NO_DATA
OK, but no data
HT_RETRY
Retry request after at a later time
HT_PERM_REDIRECT
The request has been permanently redirected and we send back the new URL
HT_TEMP_REDIRECT
The request has been temporarily redirected and we send back the new URL
HT_NO_ACCESS
The request could not be fulfilled because it didn't contain sufficient credentials
When a callback function is registered, it may be registered with a status code for which it is to be called. This means that there may be different callback functions to handle error situations, redirections etc. The status code may also take any of the values above, or HT_ALL if it is to be called always.

A callback function may return any code it likes, but IF the return code is different than HT_OK, then the callback loop is stopped. If we are in the before loop and a function returns anything else than HT_OK then we immediately jump to the after loop passing the last return code from the before loop.

Likewise, a callback function can be removed from a list using the following function: 

extern BOOL HTNetCall_delete (HTList * list, HTNetCallback *cbf);
or if you simply want to remove all functions from a list then you can use 
extern BOOL HTNetCall_deleteAll (HTList * list);

Pre-Request Callbacks

 When a request has been issued there are a number of things that an application might want to do with the request before it actually goes on the wire, for example to talk to a remote HTTP server. Examples are checking if the object already is kept in a cache managed by the application, if the request should be redirected to a proxy or a gateway, or there is some other kind of translation of the URL taking place. The Library provides a variety of modules that handles many common place translations such as redirection of URLs and caching. The full list of modules are:

Rule File Management

An application can be setup by using a rule file, also known as a configuration file. This is for example the case with the W3C httpd and the W3C Line Mode Browser. The Rules module provides basic support for configuration file management and the application can use this is desired. The module is not referred to by the Library. Reading a rule file is implemented as a stream converter so that a rule file can come from anywhere, even across the network!

Proxies and Gateways

 Applications do not have to provide native support for all protocols, they can in many situations rely on the support of proxies and gateways to help doing the job. Proxy servers are often used to carry client requests through a firewall where they can provide services like corporate caching and other network optimizations. Both Proxy servers and gateways can serve as "protocol translators" which can convert a request in the main Web protocol, HTTP, to an equivalent request in another protocol, for example NNTP, FTP, or Gopher. In a later section we will see how to set up the Proxy servers and gateways using the Proxy module.

As a HTTP request looks different when it is directed to a proxy server than to a origin server, the HTTP module needs to know whether it is talking to a proxy for this particular request or not. You can specify in a request object whether a proxy is being used or not by using the following methods: 

extern void HTRequest_setProxing (HTRequest * request, BOOL proxying);
extern BOOL HTRequest_proxying (HTRequest * request);

Cache Manager

Caching is a required part of any efficient Internet access applications as it saves bandwidth and improves access performance significantly in almost all types of accesses. The Library supports two different types of cache: The memory cache and the file cache. The two types differ in several ways which reflects their two main purposes: The memory cache is for short term storage of graphic objects whereas the file cache is for intermediate term storage of data objects. Often it is desirable to have both a memory and a file version of a cached document, so the two types do not exclude each other. The HTCache module provides a basic cache that can be used by an application.

Register a list of BEFORE Callbacks

Until now, we have only described how to build a list of callback functions. We will now describe how to setup a list as either a pre processing set of callback function (the BEFORE loop, or a post processing set (AFTER loop). A set of callback functions can be registered to be called before the request is started by using the following function: 
extern BOOL HTNet_setBefore	(HTList * list);
In many cases you know when you register a callback function that this is a function that you always want to be called when either a request starts up or terminates. In the former case you can simply register the callback directly using the following function: 
extern BOOL HTNetCall_addbefore	(HTNetCallback *cbf, int status);

Post-Request Callbacks

When a request is terminated, the application often has to do some action as a result of the request (and of the result of the request). The Application part of the Library provides two following modules to handle logging and history management. You can register a POST request handler in the Net Manager as described in the User's Guide. The set of modules provided by the Library is:

Logging

Often it is required to log the requests issued to the Library. This can either be the case if the application is a server or it can also be useful in a client application. The HTLog Module provides a simple logging mechanism which can be enabled if needed.

History Management

Another type of logging is keeping track of which documents a user has visited when browsing along on the Web. The HTHistory module provides a basic set of functionality to keep track of multiple linear history lists.

Register a list of AFTER Callbacks

The registration of a set of callback functions to be called when a request has terminated is handled in very much the same way: 
extern BOOL HTNet_setAfter	(HTList * list);
extern BOOL HTNetCall_addBefore (HTNetCallback *cbf, int status);

Henrik Frystyk, libwww@w3.org, @(#) $Id: Net.html,v 1.9 1996/12/09 03:24:12 jigsaw Exp $