Annotation of libwww/Library/src/HTCache.html, revision 2.22
2.1 frystyk 1: <HTML>
2: <HEAD>
2.16 frystyk 3: <TITLE>W3C Sample Code Library libwww Persistent Cache Manager</TITLE>
2.1 frystyk 4: </HEAD>
5: <BODY>
2.10 frystyk 6: <H1>
7: Persistent Cache Manager
8: </H1>
2.1 frystyk 9: <PRE>
10: /*
11: ** (c) COPYRIGHT MIT 1995.
12: ** Please first read the full copyright statement in the file COPYRIGH.
13: */
14: </PRE>
2.10 frystyk 15: <P>
2.11 frystyk 16: The cache contains details of persistent files which contain the contents
2.10 frystyk 17: of remote documents. The existing cache manager is somewhat naive - especially
2.11 frystyk 18: in its garbage collection but it is just an example of how it can be
19: done.However, it is a fully HTTP/1.1 compliant cache manager. More advanced
20: implementations are welcome!
2.10 frystyk 21: <P>
22: This module is implemented by <A HREF="HTCache.c">HTCache.c</A>, and it is
2.18 frystyk 23: a part of the <A HREF="http://www.w3.org/Library/">W3C Sample Code Library</A>.
2.1 frystyk 24: <PRE>
25: #ifndef HTCACHE_H
26: #define HTCACHE_H
27:
2.10 frystyk 28: #include "WWWLib.h"
2.1 frystyk 29: </PRE>
2.10 frystyk 30: <H2>
2.11 frystyk 31: Initialize and Terminate the Persistent Cache
2.10 frystyk 32: </H2>
33: <P>
2.21 frystyk 34: The <CODE>cache_root</CODE> is the URI of the location of the persistent
35: cache. An example is "<CODE>file:/tmp/w3c-lib</CODE>". If
36: <CODE>cache_root</CODE> is <CODE>NULL</CODE> then determine a cache root
37: using the following algorithm:
38: <OL>
39: <LI>
40: Look for any environment variables (if supported) in the following order:
41: <CODE>WWW_CACHE</CODE>, <CODE>TMP</CODE>, and <CODE>TEMP</CODE>. If none
42: are set then then fall back on "<CODE>/tmp</CODE>".
43: <LI>
44: Append the folder name "<CODE>w3c-cache</CODE>" to the root identified above
45: </OL>
46: <P>
47: The <CODE>cache_root</CODE> location does not have to exist, it will be created
48: automatically if not. An empty string will make '/' the cache root.
49: <P>
50: The size is the total size in MBytes - the default size is 20M. The cache
51: can not be less than 5M.
52: <P>
53: We can only enable the cache if we are in <A HREF="HTLib.html#Secure">secure
54: mode</A> where we can not access the local file system. This is for example
55: the case if using an application as a telnet shell.
2.11 frystyk 56: <PRE>
57: extern BOOL HTCacheInit (const char * cache_root, int size);
2.10 frystyk 58: </PRE>
59: <P>
2.11 frystyk 60: After the cache has been terminated it can not be used anymore unless you
61: do another <CODE>HTCacheInit()</CODE> call.
62: <PRE>
63: extern BOOL HTCacheTerminate (void);
2.10 frystyk 64: </PRE>
2.11 frystyk 65: <H2>
66: Cache Mode Parameters
67: </H2>
2.10 frystyk 68: <P>
2.22 ! kahan 69: The persistent cache has a set of overall parameters that you can adjust
2.10 frystyk 70: <H3>
2.11 frystyk 71: Enable and Disable the Cache
2.10 frystyk 72: </H3>
73: <P>
2.11 frystyk 74: The cache can be temporarily suspended by using the enable/disable flag.
75: This does not prevent the cache from being enabled/disable at a later point
76: in time.
77: <PRE>
78: extern void HTCacheMode_setEnabled (BOOL mode);
79: extern BOOL HTCacheMode_enabled (void);
2.10 frystyk 80: </PRE>
2.22 ! kahan 81: <P>
! 82: The cache can be setup to whether cache password protected documents thru the
! 83: protected flag. By default this flag is turned off.
! 84: <PRE>
! 85: extern void HTCacheMode_setProtected (BOOL mode);
! 86: extern BOOL HTCacheMode_protected (void);
! 87: </PRE>
2.10 frystyk 88: <H3>
2.11 frystyk 89: What is the current Cache Root?
2.10 frystyk 90: </H3>
91: <P>
2.11 frystyk 92: Return the value of the cache root. The cache root can only be set through
2.21 frystyk 93: the <CODE>HTCacheInit()</CODE> function. The string returned MUST be freed
94: by the caller
2.11 frystyk 95: <PRE>
2.21 frystyk 96: extern char * HTCacheMode_getRoot (void);
2.10 frystyk 97: </PRE>
98: <H3>
2.11 frystyk 99: Total Cache Size
2.10 frystyk 100: </H3>
101: <P>
2.11 frystyk 102: We set the default cache size to 20M. We set the minimum size to 5M in order
103: not to get into weird problems while writing the cache. The size is indicated
104: in Mega bytes. The size is given in MBytes and is also returned in MBytes.
2.14 frystyk 105: We don't consider the metainformation as part of the total cache size which
106: is the the reason for why the min cache size should not be less than 5M.
2.11 frystyk 107: <PRE>
108: extern BOOL HTCacheMode_setMaxSize (int size);
109: extern int HTCacheMode_maxSize (void);
2.10 frystyk 110: </PRE>
111: <H3>
2.19 frystyk 112: Max Size of a Single Cache Entry
113: </H3>
114: <P>
2.20 frystyk 115: It is also possible to control the max size of a single cache entry so that
116: the cache doesn't get filled with a very few, very large cached entries.
117: The default max size for a single cached entry is 3M. The value indicated
118: must be in Mbytes, for example, a vaue of 3 would mean 3 MBytes.
2.19 frystyk 119: <PRE>
120: extern BOOL HTCacheMode_setMaxCacheEntrySize (int size);
121: extern int HTCacheMode_maxCacheEntrySize (void);
122: </PRE>
123: <H3>
2.10 frystyk 124: How do we handle Expiration of Cached Objects?
125: </H3>
126: <P>
127: There are various ways of handling <CODE>Expires</CODE> header when met in
2.11 frystyk 128: a <I>history list</I>. Either it can be ignored all together, the user can
129: be notified with a warning, or the document can be reloaded automatically.
130: This flag decides what action to be taken. The default action is
2.10 frystyk 131: <CODE>HT_EXPIRES_IGNORE</CODE>. In <CODE>HT_EXPIRES_NOTIFY</CODE> mode ,
132: we push a message on to the Error stack which is presented to the user.
2.4 frystyk 133: <PRE>
134: typedef enum _HTExpiresMode {
135: HT_EXPIRES_IGNORE = 0,
136: HT_EXPIRES_NOTIFY,
137: HT_EXPIRES_AUTO
138: } HTExpiresMode;
139:
2.11 frystyk 140: extern void HTCacheMode_setExpires (HTExpiresMode mode);
141: extern HTExpiresMode HTCacheMode_expires (void);
142: </PRE>
143: <H3>
144: Disconnected Operation
145: </H3>
146: <P>
147: The cache can be set to handle disconnected operation where it does not use
2.20 frystyk 148: the network to validate entries and do not attempt to load new documents.
2.11 frystyk 149: All requests that can not be fulfilled by the cache will be returned with
150: a <CODE>"504 Gateway Timeout"</CODE> response. There are two modes of how
2.21 frystyk 151: the cache can operate in disconnected mode:
2.20 frystyk 152: <DL>
153: <DT>
154: <EM>No network activity at all</EM>
155: <DD>
156: Here is uses its own persistent cache
157: <DT>
158: <EM>Forward all disconnected requests to a proxy cache</EM>
159: <DD>
160: Here it uses the HTTP/1.1 cache-control to indicate that the proxy should
161: operate in disconnected mode. This mode only really makes sense when you
162: are using a proxy, of course.
163: </DL>
2.11 frystyk 164: <PRE>
165: typedef enum _HTDisconnectedMode {
166: HT_DISCONNECT_NONE = 0,
167: HT_DISCONNECT_NORMAL = 1,
168: HT_DISCONNECT_EXTERNAL = 2
169: } HTDisconnectedMode;
170:
171: extern void HTCacheMode_setDisconnected (HTDisconnectedMode mode);
172: extern HTDisconnectedMode HTCacheMode_disconnected (void);
173: extern BOOL HTCacheMode_isDisconnected (HTReload mode);
2.1 frystyk 174: </PRE>
2.10 frystyk 175: <H2>
2.12 frystyk 176: The Cache Index
177: </H2>
178: <P>
179: The persistent cache keeps an index of its current entries so that garbage
180: collection and lookup becomes more efficient. This index is stored automatically
181: at regular intervals so that we don't get out of sync. Also, it is automatically
182: loaded at startup and saved at closedown of the cache.
183: <H3>
184: Reading the Cache Index
185: </H3>
186: <P>
187: Read the saved set of cached entries from disk. we only allow the index ro
188: be read when there is no entries in memory. That way we can ensure consistancy.
189: <PRE>
190: extern BOOL HTCacheIndex_read (const char * cache_root);
191: </PRE>
192: <H3>
193: Write the Cache Index
194: </H3>
195: <P>
196: Walk through the list of cached objects and save them to disk. We override
197: any existing version but that is normally OK as we have already read its
198: contents.
199: <PRE>
200: extern BOOL HTCacheIndex_write (const char * cache_root);
201: </PRE>
202: <H2>
2.11 frystyk 203: The HTCache Object
2.10 frystyk 204: </H2>
205: <P>
2.11 frystyk 206: The cache object is what we store about a cached objet in memory.
207: <PRE>
208: typedef struct _HTCache HTCache;
209: </PRE>
210: <H3>
2.12 frystyk 211: Create and Update a Cache Object
2.11 frystyk 212: </H3>
213: <P>
2.10 frystyk 214: Filling the cache is done as all other transportation of bulk data in libwww
215: using <A HREF="HTStream.html">streams</A>. The cache object creater is a
216: stream which in many cases sits on a <A HREF="HTTee.html">T stream</A> so
217: that we get the original feed and at the same time can parse the contents.
2.14 frystyk 218: <P>
219: In some situations, we want to append data to an already exiting cache entry.
220: This is the case when a use has interrupted a download and we are stuck with
221: a subpart of the document. If the user later on whishes to download the object
222: again we can issue a range request and continue from where we were. This
223: will in many situations save a lot of bandwidth.
2.11 frystyk 224: <PRE>
2.14 frystyk 225: extern HTConverter HTCacheWriter, HTCacheAppend;
2.11 frystyk 226: </PRE>
2.12 frystyk 227: <P>
228: This function writes the metainformation along with the data object stored
229: by the HTCacheWriter stream above. If no headers are available then the meta
230: file is empty
231: <PRE>
2.14 frystyk 232: extern BOOL HTCache_writeMeta (HTCache * cache, HTRequest * request,
233: HTResponse * response);
2.12 frystyk 234: </PRE>
235: <P>
236: In case we received a "<CODE>304 Not Modified</CODE>" response then we do
237: not have to tough the body but must merge the metainformation with the previous
238: version. Therefore we need a special metainformation update function.
239: <PRE>
2.14 frystyk 240: extern BOOL HTCache_updateMeta (HTCache * cache, HTRequest * request,
241: HTResponse * response);
2.12 frystyk 242: </PRE>
2.11 frystyk 243: <H3>
2.18 frystyk 244: Check Cached Entry
245: </H3>
246: <P>
247: After we get a response back, we should check whether we can still cache
248: an entry and/or we should add an entry for a resource that has just been
249: created so that we can remember the etag and other things. The latter allows
250: us to guarantee that we don't loose data due to the lost update problem.
251: <PRE>
252: extern HTCache * HTCache_touch (HTRequest * request, HTResponse * response,
253: HTParentAnchor * anchor);
254: </PRE>
255: <P>
256: <H3>
2.11 frystyk 257: Load a Cached Object
258: </H3>
259: <P>
260: Loading a cached object is also done as all other loads in libwww by using
261: a <A HREF="HTProt.html">protocol load module</A>. For the moment, this load
262: function handles the persistent cache as if it was on local file but in fact
263: it could be anywhere.
264: <PRE>
2.15 frystyk 265: extern HTProtCallback HTLoadCache;
2.11 frystyk 266: </PRE>
267: <H3>
268: Delete a Cache Object
269: </H3>
270: <P>
271: Remove a HTCache object from memory and from disk. You must explicitly remove
272: a lock before this operation can succeed
273: <PRE>
274: extern BOOL HTCache_remove (HTCache * cache);
275: </PRE>
276: <H3>
2.13 frystyk 277: Delete All Cache Objects in Memory
2.11 frystyk 278: </H3>
279: <P>
280: Destroys all cache entried in memory but does not write anything to disk.
281: Use the index methods above for doing that. We do not delete the disk contents.
282: <PRE>
283: extern BOOL HTCache_deleteAll (void);
2.10 frystyk 284: </PRE>
285: <H3>
2.13 frystyk 286: Delete all Cache Object and File Entries
287: </H3>
288: <P>
289: Destroys all cache entried in memory <B>and</B> on disk. This call basically
290: resets the cache to the inital state but it does not terminate the cache.
291: That is, you don't have to reinitialize the cache before you can use it again.
292: <PRE>
293: extern BOOL HTCache_flushAll (void);
294: </PRE>
295: <H3>
2.11 frystyk 296: Find a Cached Object
2.10 frystyk 297: </H3>
298: <P>
299: Verifies if a cache object exists for this URL and if so returns a URL for
300: the cached object. It does not verify whether the object is valid or not,
301: for example it might have expired. Use the cache validation methods for checking
302: this.
2.11 frystyk 303: <PRE>
304: extern HTCache * HTCache_find (HTParentAnchor * anchor);
305: </PRE>
306: <H3>
307: Verify if an Object is Fresh
308: </H3>
309: <P>
310: This function checks whether a document has expired or not. The check is
311: based on the metainformation passed in the anchor object The function returns
312: the level of validation needed for getting a fresh version. We also check
313: the cache control directives in the request to see if they change the freshness
314: discission.
315: <PRE>
316: extern HTReload HTCache_isFresh (HTCache * me, HTRequest * request);
317: </PRE>
318: <H3>
319: Register a Cache Hit
320: </H3>
321: <P>
322: As a cache hit may occur several places, we have a public function where
323: we can declare a download to be a true cache hit. The number of hits a cache
324: object has affects its status when we are doing garbage collection.
325: <PRE>
326: extern BOOL HTCache_addHit (HTCache * cache);
327: </PRE>
328: <H3>
329: Find the Location of a Cached Object
330: </H3>
331: <P>
332: Is we have a valid entry in the cache then we also need a location where
333: we can get it. Hopefully, we may be able to access it thourgh one of our
334: protocol modules, for example the <A HREF="WWWFile.html">local file module</A>.
335: The name returned is in URL syntax and must be freed by the caller
336: <PRE>
337: extern char * HTCache_name (HTCache * cache);
338: </PRE>
339: <H3>
340: Locking a Cache Object
341: </H3>
342: <P>
343: While we are creating a new cache object or while we are validating an existing
344: one, we must have a lock on the entry so that not other requests can get
345: to it in the mean while. A lock can be broken if the same request tries to
346: create the cache entry again. This means that we have tried to validate the
347: cache entry but we got a new shipment of bytes back from the origin server
348: or an intermediary proxy.
349: <PRE>
350: extern BOOL HTCache_getLock (HTCache * cache, HTRequest * request);
351: extern BOOL HTCache_breakLock (HTCache * cache, HTRequest * request);
352: extern BOOL HTCache_hasLock (HTCache * cache);
353: extern BOOL HTCache_releaseLock (HTCache * cache);
2.1 frystyk 354: </PRE>
355: <PRE>
356: #endif
357: </PRE>
2.10 frystyk 358: <P>
359: <HR>
2.9 frystyk 360: <ADDRESS>
2.22 ! kahan 361: @(#) $Id: HTCache.html,v 2.21 1999/02/03 16:35:11 frystyk Exp $
2.9 frystyk 362: </ADDRESS>
2.10 frystyk 363: </BODY></HTML>
Webmaster
![[BACK]](/cvsweb/icons/back.gif){kind=icon}
Return to HTCache.html CVS log ![[TXT]](/cvsweb/icons/text.gif){kind=icon}
![[DIR]](/cvsweb/icons/dir.gif){kind=icon}
Up to [Public] / libwww / Library / src