Annotation of libwww/Library/src/HTCache.html, revision 2.25
2.1 frystyk 1: <HTML>
2: <HEAD>
2.16 frystyk 3: <TITLE>W3C Sample Code Library libwww Persistent Cache Manager</TITLE>
2.1 frystyk 4: </HEAD>
5: <BODY>
2.10 frystyk 6: <H1>
7: Persistent Cache Manager
8: </H1>
2.1 frystyk 9: <PRE>
10: /*
11: ** (c) COPYRIGHT MIT 1995.
12: ** Please first read the full copyright statement in the file COPYRIGH.
13: */
14: </PRE>
2.10 frystyk 15: <P>
2.11 frystyk 16: The cache contains details of persistent files which contain the contents
2.10 frystyk 17: of remote documents. The existing cache manager is somewhat naive - especially
2.11 frystyk 18: in its garbage collection but it is just an example of how it can be
19: done.However, it is a fully HTTP/1.1 compliant cache manager. More advanced
20: implementations are welcome!
2.10 frystyk 21: <P>
22: This module is implemented by <A HREF="HTCache.c">HTCache.c</A>, and it is
2.18 frystyk 23: a part of the <A HREF="http://www.w3.org/Library/">W3C Sample Code Library</A>.
2.1 frystyk 24: <PRE>
25: #ifndef HTCACHE_H
26: #define HTCACHE_H
27:
2.10 frystyk 28: #include "WWWLib.h"
2.1 frystyk 29: </PRE>
2.10 frystyk 30: <H2>
2.11 frystyk 31: Initialize and Terminate the Persistent Cache
2.10 frystyk 32: </H2>
33: <P>
2.21 frystyk 34: The <CODE>cache_root</CODE> is the URI of the location of the persistent
35: cache. An example is "<CODE>file:/tmp/w3c-lib</CODE>". If
36: <CODE>cache_root</CODE> is <CODE>NULL</CODE> then determine a cache root
37: using the following algorithm:
38: <OL>
39: <LI>
40: Look for any environment variables (if supported) in the following order:
41: <CODE>WWW_CACHE</CODE>, <CODE>TMP</CODE>, and <CODE>TEMP</CODE>. If none
42: are set then then fall back on "<CODE>/tmp</CODE>".
43: <LI>
44: Append the folder name "<CODE>w3c-cache</CODE>" to the root identified above
45: </OL>
46: <P>
47: The <CODE>cache_root</CODE> location does not have to exist, it will be created
48: automatically if not. An empty string will make '/' the cache root.
49: <P>
50: The size is the total size in MBytes - the default size is 20M. The cache
51: can not be less than 5M.
52: <P>
53: We can only enable the cache if we are in <A HREF="HTLib.html#Secure">secure
54: mode</A> where we can not access the local file system. This is for example
55: the case if using an application as a telnet shell.
2.11 frystyk 56: <PRE>
57: extern BOOL HTCacheInit (const char * cache_root, int size);
2.10 frystyk 58: </PRE>
59: <P>
2.11 frystyk 60: After the cache has been terminated it can not be used anymore unless you
61: do another <CODE>HTCacheInit()</CODE> call.
62: <PRE>
63: extern BOOL HTCacheTerminate (void);
2.10 frystyk 64: </PRE>
2.11 frystyk 65: <H2>
66: Cache Mode Parameters
67: </H2>
2.10 frystyk 68: <P>
2.22 kahan 69: The persistent cache has a set of overall parameters that you can adjust
2.10 frystyk 70: <H3>
2.11 frystyk 71: Enable and Disable the Cache
2.10 frystyk 72: </H3>
73: <P>
2.11 frystyk 74: The cache can be temporarily suspended by using the enable/disable flag.
75: This does not prevent the cache from being enabled/disable at a later point
76: in time.
77: <PRE>
78: extern void HTCacheMode_setEnabled (BOOL mode);
79: extern BOOL HTCacheMode_enabled (void);
2.10 frystyk 80: </PRE>
2.22 kahan 81: <P>
82: The cache can be setup to whether cache password protected documents thru the
83: protected flag. By default this flag is turned off.
84: <PRE>
85: extern void HTCacheMode_setProtected (BOOL mode);
86: extern BOOL HTCacheMode_protected (void);
87: </PRE>
2.10 frystyk 88: <H3>
2.11 frystyk 89: What is the current Cache Root?
2.10 frystyk 90: </H3>
91: <P>
2.11 frystyk 92: Return the value of the cache root. The cache root can only be set through
2.21 frystyk 93: the <CODE>HTCacheInit()</CODE> function. The string returned MUST be freed
94: by the caller
2.11 frystyk 95: <PRE>
2.21 frystyk 96: extern char * HTCacheMode_getRoot (void);
2.10 frystyk 97: </PRE>
98: <H3>
2.11 frystyk 99: Total Cache Size
2.10 frystyk 100: </H3>
101: <P>
2.11 frystyk 102: We set the default cache size to 20M. We set the minimum size to 5M in order
103: not to get into weird problems while writing the cache. The size is indicated
104: in Mega bytes. The size is given in MBytes and is also returned in MBytes.
2.14 frystyk 105: We don't consider the metainformation as part of the total cache size which
106: is the the reason for why the min cache size should not be less than 5M.
2.11 frystyk 107: <PRE>
108: extern BOOL HTCacheMode_setMaxSize (int size);
109: extern int HTCacheMode_maxSize (void);
2.10 frystyk 110: </PRE>
111: <H3>
2.19 frystyk 112: Max Size of a Single Cache Entry
113: </H3>
114: <P>
2.20 frystyk 115: It is also possible to control the max size of a single cache entry so that
116: the cache doesn't get filled with a very few, very large cached entries.
117: The default max size for a single cached entry is 3M. The value indicated
118: must be in Mbytes, for example, a vaue of 3 would mean 3 MBytes.
2.19 frystyk 119: <PRE>
120: extern BOOL HTCacheMode_setMaxCacheEntrySize (int size);
121: extern int HTCacheMode_maxCacheEntrySize (void);
122: </PRE>
123: <H3>
2.23 kahan 124: Default expiration time of cache entries
125: </H3>
126: <P>
127: If a response does not arrive with an expiration time and does not
128: explicitly forbid its being cached, use the default expiration time. The
129: time is given in seconds (e.g., 3,600 is one hour).
130: <PRE>
131: extern void HTCacheMode_setDefaultExpiration (const int exp_time);
132: extern int HTCacheMode_DefaultExpiration (void);
133: </PRE>
134: <H3>
2.10 frystyk 135: How do we handle Expiration of Cached Objects?
136: </H3>
137: <P>
138: There are various ways of handling <CODE>Expires</CODE> header when met in
2.11 frystyk 139: a <I>history list</I>. Either it can be ignored all together, the user can
140: be notified with a warning, or the document can be reloaded automatically.
141: This flag decides what action to be taken. The default action is
2.10 frystyk 142: <CODE>HT_EXPIRES_IGNORE</CODE>. In <CODE>HT_EXPIRES_NOTIFY</CODE> mode ,
143: we push a message on to the Error stack which is presented to the user.
2.4 frystyk 144: <PRE>
145: typedef enum _HTExpiresMode {
146: HT_EXPIRES_IGNORE = 0,
147: HT_EXPIRES_NOTIFY,
148: HT_EXPIRES_AUTO
149: } HTExpiresMode;
150:
2.11 frystyk 151: extern void HTCacheMode_setExpires (HTExpiresMode mode);
152: extern HTExpiresMode HTCacheMode_expires (void);
153: </PRE>
154: <H3>
155: Disconnected Operation
156: </H3>
157: <P>
158: The cache can be set to handle disconnected operation where it does not use
2.20 frystyk 159: the network to validate entries and do not attempt to load new documents.
2.11 frystyk 160: All requests that can not be fulfilled by the cache will be returned with
161: a <CODE>"504 Gateway Timeout"</CODE> response. There are two modes of how
2.21 frystyk 162: the cache can operate in disconnected mode:
2.20 frystyk 163: <DL>
164: <DT>
165: <EM>No network activity at all</EM>
166: <DD>
167: Here is uses its own persistent cache
168: <DT>
169: <EM>Forward all disconnected requests to a proxy cache</EM>
170: <DD>
171: Here it uses the HTTP/1.1 cache-control to indicate that the proxy should
172: operate in disconnected mode. This mode only really makes sense when you
173: are using a proxy, of course.
174: </DL>
2.11 frystyk 175: <PRE>
176: typedef enum _HTDisconnectedMode {
177: HT_DISCONNECT_NONE = 0,
178: HT_DISCONNECT_NORMAL = 1,
179: HT_DISCONNECT_EXTERNAL = 2
180: } HTDisconnectedMode;
181:
182: extern void HTCacheMode_setDisconnected (HTDisconnectedMode mode);
183: extern HTDisconnectedMode HTCacheMode_disconnected (void);
184: extern BOOL HTCacheMode_isDisconnected (HTReload mode);
2.1 frystyk 185: </PRE>
2.10 frystyk 186: <H2>
2.12 frystyk 187: The Cache Index
188: </H2>
189: <P>
190: The persistent cache keeps an index of its current entries so that garbage
191: collection and lookup becomes more efficient. This index is stored automatically
192: at regular intervals so that we don't get out of sync. Also, it is automatically
193: loaded at startup and saved at closedown of the cache.
194: <H3>
195: Reading the Cache Index
196: </H3>
197: <P>
198: Read the saved set of cached entries from disk. we only allow the index ro
199: be read when there is no entries in memory. That way we can ensure consistancy.
200: <PRE>
201: extern BOOL HTCacheIndex_read (const char * cache_root);
202: </PRE>
203: <H3>
204: Write the Cache Index
205: </H3>
206: <P>
207: Walk through the list of cached objects and save them to disk. We override
208: any existing version but that is normally OK as we have already read its
209: contents.
210: <PRE>
211: extern BOOL HTCacheIndex_write (const char * cache_root);
212: </PRE>
213: <H2>
2.11 frystyk 214: The HTCache Object
2.10 frystyk 215: </H2>
216: <P>
2.11 frystyk 217: The cache object is what we store about a cached objet in memory.
218: <PRE>
219: typedef struct _HTCache HTCache;
220: </PRE>
221: <H3>
2.12 frystyk 222: Create and Update a Cache Object
2.11 frystyk 223: </H3>
224: <P>
2.10 frystyk 225: Filling the cache is done as all other transportation of bulk data in libwww
226: using <A HREF="HTStream.html">streams</A>. The cache object creater is a
227: stream which in many cases sits on a <A HREF="HTTee.html">T stream</A> so
228: that we get the original feed and at the same time can parse the contents.
2.14 frystyk 229: <P>
230: In some situations, we want to append data to an already exiting cache entry.
231: This is the case when a use has interrupted a download and we are stuck with
232: a subpart of the document. If the user later on whishes to download the object
233: again we can issue a range request and continue from where we were. This
234: will in many situations save a lot of bandwidth.
2.11 frystyk 235: <PRE>
2.14 frystyk 236: extern HTConverter HTCacheWriter, HTCacheAppend;
2.11 frystyk 237: </PRE>
2.12 frystyk 238: <P>
239: This function writes the metainformation along with the data object stored
240: by the HTCacheWriter stream above. If no headers are available then the meta
241: file is empty
242: <PRE>
2.14 frystyk 243: extern BOOL HTCache_writeMeta (HTCache * cache, HTRequest * request,
244: HTResponse * response);
2.12 frystyk 245: </PRE>
246: <P>
247: In case we received a "<CODE>304 Not Modified</CODE>" response then we do
248: not have to tough the body but must merge the metainformation with the previous
249: version. Therefore we need a special metainformation update function.
250: <PRE>
2.14 frystyk 251: extern BOOL HTCache_updateMeta (HTCache * cache, HTRequest * request,
252: HTResponse * response);
2.12 frystyk 253: </PRE>
2.25 ! kahan 254: <P>
! 255: Clear a cache entry
! 256: <PRE>
! 257: extern BOOL HTCache_resetMeta (HTCache * cache, HTRequest * request,
! 258: HTResponse * response);
! 259: </PRE>
2.11 frystyk 260: <H3>
2.18 frystyk 261: Check Cached Entry
262: </H3>
263: <P>
264: After we get a response back, we should check whether we can still cache
265: an entry and/or we should add an entry for a resource that has just been
266: created so that we can remember the etag and other things. The latter allows
267: us to guarantee that we don't loose data due to the lost update problem.
268: <PRE>
269: extern HTCache * HTCache_touch (HTRequest * request, HTResponse * response,
270: HTParentAnchor * anchor);
271: </PRE>
272: <P>
273: <H3>
2.11 frystyk 274: Load a Cached Object
275: </H3>
276: <P>
277: Loading a cached object is also done as all other loads in libwww by using
278: a <A HREF="HTProt.html">protocol load module</A>. For the moment, this load
279: function handles the persistent cache as if it was on local file but in fact
280: it could be anywhere.
281: <PRE>
2.15 frystyk 282: extern HTProtCallback HTLoadCache;
2.11 frystyk 283: </PRE>
284: <H3>
285: Delete a Cache Object
286: </H3>
287: <P>
288: Remove a HTCache object from memory and from disk. You must explicitly remove
289: a lock before this operation can succeed
290: <PRE>
291: extern BOOL HTCache_remove (HTCache * cache);
292: </PRE>
293: <H3>
2.13 frystyk 294: Delete All Cache Objects in Memory
2.11 frystyk 295: </H3>
296: <P>
297: Destroys all cache entried in memory but does not write anything to disk.
298: Use the index methods above for doing that. We do not delete the disk contents.
299: <PRE>
300: extern BOOL HTCache_deleteAll (void);
2.10 frystyk 301: </PRE>
302: <H3>
2.13 frystyk 303: Delete all Cache Object and File Entries
304: </H3>
305: <P>
306: Destroys all cache entried in memory <B>and</B> on disk. This call basically
307: resets the cache to the inital state but it does not terminate the cache.
308: That is, you don't have to reinitialize the cache before you can use it again.
309: <PRE>
310: extern BOOL HTCache_flushAll (void);
311: </PRE>
312: <H3>
2.11 frystyk 313: Find a Cached Object
2.10 frystyk 314: </H3>
315: <P>
316: Verifies if a cache object exists for this URL and if so returns a URL for
317: the cached object. It does not verify whether the object is valid or not,
318: for example it might have expired. Use the cache validation methods for checking
319: this.
2.11 frystyk 320: <PRE>
2.24 kahan 321: extern HTCache * HTCache_find (HTParentAnchor * anchor, char * default_name);
2.11 frystyk 322: </PRE>
323: <H3>
324: Verify if an Object is Fresh
325: </H3>
326: <P>
327: This function checks whether a document has expired or not. The check is
328: based on the metainformation passed in the anchor object The function returns
329: the level of validation needed for getting a fresh version. We also check
330: the cache control directives in the request to see if they change the freshness
331: discission.
332: <PRE>
333: extern HTReload HTCache_isFresh (HTCache * me, HTRequest * request);
334: </PRE>
335: <H3>
336: Register a Cache Hit
337: </H3>
338: <P>
339: As a cache hit may occur several places, we have a public function where
340: we can declare a download to be a true cache hit. The number of hits a cache
341: object has affects its status when we are doing garbage collection.
342: <PRE>
343: extern BOOL HTCache_addHit (HTCache * cache);
344: </PRE>
345: <H3>
346: Find the Location of a Cached Object
347: </H3>
348: <P>
349: Is we have a valid entry in the cache then we also need a location where
350: we can get it. Hopefully, we may be able to access it thourgh one of our
351: protocol modules, for example the <A HREF="WWWFile.html">local file module</A>.
352: The name returned is in URL syntax and must be freed by the caller
353: <PRE>
354: extern char * HTCache_name (HTCache * cache);
355: </PRE>
356: <H3>
357: Locking a Cache Object
358: </H3>
359: <P>
360: While we are creating a new cache object or while we are validating an existing
361: one, we must have a lock on the entry so that not other requests can get
362: to it in the mean while. A lock can be broken if the same request tries to
363: create the cache entry again. This means that we have tried to validate the
364: cache entry but we got a new shipment of bytes back from the origin server
365: or an intermediary proxy.
366: <PRE>
367: extern BOOL HTCache_getLock (HTCache * cache, HTRequest * request);
368: extern BOOL HTCache_breakLock (HTCache * cache, HTRequest * request);
369: extern BOOL HTCache_hasLock (HTCache * cache);
370: extern BOOL HTCache_releaseLock (HTCache * cache);
2.1 frystyk 371: </PRE>
372: <PRE>
373: #endif
374: </PRE>
2.10 frystyk 375: <P>
376: <HR>
2.9 frystyk 377: <ADDRESS>
2.25 ! kahan 378: @(#) $Id: HTCache.html,v 2.24 2000/06/19 13:43:33 kahan Exp $
2.9 frystyk 379: </ADDRESS>
2.10 frystyk 380: </BODY></HTML>
Webmaster
![[BACK]](/cvsweb/icons/back.gif){kind=icon}
Return to HTCache.html CVS log ![[TXT]](/cvsweb/icons/text.gif){kind=icon}
![[DIR]](/cvsweb/icons/dir.gif){kind=icon}
Up to [Public] / libwww / Library / src