[1] R. Fielding, J. Gettys, J. C. Mogul, H. Frystyk, T. Berners-Lee, Hypertext
Transfer Protocol -- HTTP/1.1, HTTP Working Group, Internet Draft,
v11-spec-07, 12 August 1996.
[2] J. C. Mogul, "The Case for Persistent-Connection HTTP," Proc. ACM
SIGCOMM '95, Oct., pp. 299-313.
Goals of HTTP/1.1
Support wide diversity of client/server/network configurations:
Many consumers of HTTP: PDAs, CD-ROM, ...
Many network links: low-power radio to PDAs, transoceanic links, ...
Many network architectures: caches for transoceanic links, hierarchical
caches, corporate gateways, ...
Terms (from [1])
Message
Basic unit of HTTP communication
Consists of structured sequence of octets
Transmitted via a connection.
Resource
A network data object or service that can be identified by a URI.
Entity
Information transferred as payload of request or response
Example: an HTML document
Content negotiation
Mechanism for selecting appropriate representation when servicing request.
Representation of entities in any response can be negotiated (including
error responses).
Server on which a given resource resides or is to be created.
Proxy
Intermediary program which acts as both a server and a client for
the purpose of making requests on behalf of other clients.
Requests are serviced internally or by passing them on, with possible translation,
to other servers.
A proxy must implement both client and server requirements of HTTP 1.1
spec.
Intermediaries
HTTP/1.0 was devised for a simple case when a user agent (client) requests
a document from an origin server:
request chain ------------------------>
UA -------------------v------------------- O
<----------------------- response chain
HTTP/1.1 recognizes that one or more intermediaries (e.g., a proxy)
may exist:
request chain -------------------------------------->
UA -----v----- A -----v----- B -----v----- C -----v----- O
<------------------------------------- response chain
If a cache is used on any intermediary, the request/response chain is shortened:
request chain -------------------------------------->
UA -----v----- A -----v----- B - - - - - - C - - - - - - O
<------------------------------------- response chain
Some Compatibility Problems
HTTP clients, applications, and servers have some subtle incompatibilities
that the HTTP/1.1 spec lists:
Multiple data/time stamp formats (the first below is prefered in HTTP/1.1):
Sun, 06 Nov 1994 08:49:37 GMT ; RFC 822, updated by RFC 1123
Sunday, 06-Nov-94 08:49:37 GMT ; RFC 850, obsoleted by RFC 1036
Sun Nov 6 08:49:37 1994 ; ANSI C's asctime() format
Use of different time zones for date/time comparisons (GMT only must be
used)
Rules to determine when two URIs are equivalanet (e.g., server name comparison
is case insensitive, certain characters [e.g., "~"] are illegal)
Character encodings:
Character sets in countries differ.
HTTP/1.1 defines a character encoding as "a method usedwith one or more
tables to convert a sequence of octets into a sequence of characters."
Content coding:
"Content coding values indicate an encoding transformation that has been
or can be applied to an entity" (e.g., compression).
Specified in Accept-Encoding and Content-Encoding header
fields.
Initially only gzip (RFC 1952), Unix compress (Lempel-Ziv-Welz encoding),
and deflate (RFC 1950, 1951) are registered with the Internet Assigned
Numbers Authority (IANA).
Tags
Product Tokens
User-Agent and Server fields contain product tokens,
which allow communicating applications to identify themselves by software
name and version.
Most fields using product tokens also allow sub-products, separated by
whitespace.
By convention, the products are listed in order of their significance for
identifying the application.
Language tags specify a natural language, not a computer language, of messages.
Example tags include en, en-US, en-cockney, i-cherokee, x-pig-latin.
Entity Tags
HTTP/1.0 was designed without thought of caching. So date/time stamps have
been used by caches to identify when an entity is inconsistent with the
document on the origin server.
In contrast, HTTP/1.1 uses entity tags
to compare two or more entities from the same requested resource.
HTTP/1.1 also adds new header fields: ETag,
If-Match, If-None-Match, and If-Range.
Other Tags
Other tags include quality values (how
much degradation in quality is tolerated).
New Request Types
HTTP/1.0 uses GET, HEAD, and POST.
HTTP/1.1 adds several new ones:
PUT x
Requests that x be stored under the
supplied URI - thus allowing a client to write a file to a server!
DELETE x
Remove x. So now a client can delete
a file on a server!
TRACE
Used to invoke a remote, application-layer loop-back of request message.
TRACE allows client to see what is being received at other end of a request
chain for testing or diagnosis.
The value of the Via header field traces of request chain.
Use of Max-Forwards header field allows client to limit request
chain length. Useful for testing a chain of proxies forwarding messages
in an infinite loop.
Final recipient is either origin server or first proxy or gateway to receive
a Max-Forwards value of zero in the request.
OPTIONS
A request for information about the communication options available
on the request/response chain identified by the Request-URI.
There is also a provision to extend this list.
Return codes of 405 (Method Not Allowed) and 501 (Not Implemented) are
added for servers to respond. GET and HEAD must be supported by all servers;
the rest are optional.
Upon receipt of unauthorized request for URI, server MAY respond with challenge:
WWW-Authenticate: Basic realm="WallyWorld"
where "WallyWorld" is string assigned by server to identify protection
space of the Request-URI.
Client sends userid and password, separated by ":", using base64 encoding.
Example: User agent sends userid "Aladdin", password
"open sesame":
Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==
Caching
Terms
Explicit expiration time
Time at which origin server intends that an
entity should no longer be returned by a cache without further validation.
Heuristic expiration time
Expiration time assigned by cache when no
explicit expiration time is available.
Age
Age of response is time since it was sent
by, or successfully validated with, origin server.
Overview
"The goal of caching in HTTP/1.1 is to eliminate the need to send requests
in many cases, and to eliminate the need to send full responses in many
other cases."
Not sending requests (by expiration)
eliminates round trip delays, because responses are only sent to reply
to requests.
Not sending full responses (by validation)
saves network bandwidth, because the full document is not sent over
the network.
Client receives exactly the same response
from cache as from origin server.
(except for hop-by-hop headers)
Unlike HTTP/1.0:
HTTP/1.1 maintains semantic transparency unless
user-agent or origin server says to relax.
Warning is sent to the user if relaxed.
Freshness
In HTTP/1.1 clients, servers, and caches specify the document freshness
they require, to permit relaxing transparency.
Cache responds to user request in one of four ways:
Returned doc was checked for equivalence with origin server's copy.
Returned doc is "fresh enough": meets least restrictive freshness requirement
of client, server, and cache.
Warning that freshness demand is being violated (using the Warning
response header and maybe an icon of a rotting fish in the Web browser).
Not Modified, Proxy Redirect, or error response returned.
For example, if cache cannot communicate with origin server, then cache
must follow cases 2, 3, or 4.
Expiration Model
Recall that expiration is used to avoid sending requests.
Expiration by client:
Client's request may specify maximum age it's willing to accept of an unvalidated
response
Specifying value of zero forces cache(s) to
revalidate all responses.
Client may also specify that it will accept stale responses, up to some
maximum amount of staleness. This may be necessary to support disconnected
operation, or high availability in the face of poor connectivity.
Clients use Max-Age directive of
Cache-Control header.
Expiration by server:
Server could tell proxy that it must not cache a document
(e.g., for copyrighted material, or pay-per-view documents).
Server could also assign expiration time in past to force caches to revalidate
every request.
Servers use either Expires header
or Max-Age directive of Cache-Control
header.
Heuristic Expiration
Caches work fastest if they don't check origin server to decide if cached
copy is up to date.
HTTP/1.1 authors hope that in the future servers will assign explicit expiration
times to documents.
Otherwise caches assign heuristic expiration
times, based on algorithms not specified by HTTP/1.1 (e.g., 10% of the
interval since the Last-Modified time of a document).
Expiration Calculation
A response in a cache has expired if
freshness_lifetime > current_age
where
freshness_lifetime is either
the max-age directive from a response,
or if not present expires_value - current_date,
or if not present, a heuristic.
current_age
is a formula that uses the age header value, which is the sender's
estimate of the amount of time since the response was generated or last
revalidated by the origin server. Age is corrected to take into account
propagation delay between multiple levels of caches.
Validators
Recall that
Cache uses expiration to avoid sending
a request to an origin server.
Origin server uses validation to avoid
sending full responses to requests.
Cache checks with origin server to see if its cached entry is still
usable, in which case full response is not sent.
HTTP/1.0 uses two forms of validating when an entity in a cache is equivalent
to an entity in an origin server: strong and
weak.
One can think of a strong validator as
one that changes whenever the bits of an entity changes, while a weak
validator changes whenever the meaning of an entity changes.
Weak Validator
Example: Last-Modified header field
Strong Validator
Example: an integer that is incremented in stable storage every time an
entity is changed
Strong validator is specified by ETag
entity-header field.
Is used when servers
don't store modification times
HTTP's one-second resolution date value does not distinguish document versions,
or
to avoid "certain paradoxes that may arise form the use of modification
tags."
To perform conditional document retrieval from origin server, a cache specifies
in its request boolean validating conditions (e.g., perform the GET if
and only if validator matches, perform GET if and only if no validators
match).
HTTP Methods that Write to URLs
HTTP/1.1 specifies write-through semantics
for any HTTP methods that update a resource on an origin server (i.e.,
any method but GET and HEAD). Thus the origin server is contacted on every
write operation.
"The alternative (known as "write-back" or "copy-back" caching) is not
allowed in HTTP/1.1, due to the difficulty of providing consistent updates
and the problems arising from server, cache, or network failure prior to
write-back."
History Lists (the "Back" Button)
History lists or Back-buttons are meant to show exactly
what the user saw at a past time.
Thus the "back" button or history list in a Web browser should simply
redisplay a previous page, and should not try to show a semantically transparent
view of the current state of a resource.
(So when using a search engine, you'd never get a message to "repost"
when you go back to the page of items found by the search.)
What Can be Cached
The Cache-Control response directives
allow an origin server to override the default cachability of a response.
A shared cache might be a proxy, and a private cache a Web browser.
public
Response is cachable
private
All or part of the response message is intended for a single use and must
not be cached by a shared cache.
no-cache
All or part of the response must not be cached anywhere.
No-Store Directive
No-Store directive prohibits cache from storing
any part of request or response in non-volatile storage.
Thus there's no way that sensitive information in request or response
could be inadvertently released (e.g., on backup tapes).
"The purpose of this directive is to meet the stated requirements of
certain users and service authors who are concerned about accidental releases
of information via unanticipated accesses to cache data structures. While
the use of this directive may improve privacy in some cases, we caution
that it is NOT in any way a reliable or sufficient mechanism for ensuring
privacy. "
User-agent Forced Cache Reloads
A user agent can force a cache reload using the Cache-Control
request directive. There are two forms:
end-to-end
The "no-cache" Cache-Control directive forces
a reload
end-to-end revalidation
Forces each cache along the path to the origin server to revalidate its
own entry, if any, with the next cache or server.
Other Aspects of Caching
Caching in HTTP/1.1 is complex. The protocol spec discusses many more aspects
of caching.
Persistent Connections
Originally in the Web, a separate TCP connection was used to fetch each
URL. Analysis of HTTP performance [2] lead to the use of persistent HTTP
conenctions. The advantages (from [1]):
Saves CPU time and memory: By opening and closing fewer TCP connections, CPU time is saved,
and memory used for TCP protocol control blocks is also saved.
HTTP requests and responses can be pipelined
on a connection. Pipelining allows client to make multiple requests without waiting
for each respons
This allows single TCP connection to be used much more efficiently, with
much lower elapsed time.
Network congestion is reduced. Reduces number of packets caused by TCP opens, and by allowing
TCP sufficient time to determine the congestion state of the network.
HTTP can evolve more gracefully. Errors can be reported without the penalty of closing the TCP connection.
Clients using future versions of HTTP might optimistically try a new feature,
but if communicating with an older server, retry with old semantics after
an error is reported.
Persistent connections in HTTP/1.1, unlike earlier versions, are the default
behavior.
One complication of consistent connections is that either the client
or server or a proxy might time-out the connection if no data is sent for
a while. So implementations of HTTP must be able to reopen connections
whenever needed. Another case when computer software gets much more complex
to get a performance improvement.
Miscellaneous HTTP/1.1 Features
Chunks
A message can be transfered as a series of chunks, each with its
own size indicator, followed by a footer. "This allows dynamically-produced
content to be transferred along with the information necessary for the
recipient to verify that it has received the full message." In comparison,
script-generated responses in HTTP/1.0 normally have zero length, so the
user-agent verify if it got the whole message!
Chunks are also used in persistent connections, which required self-defined
message lengths (rather than defining length upon closure of a connection).
New Header Fields
For Either Request or Response
Cache-Control
Connection
Persistent TCP connection is used unless set to "CLOSE."
Upgrade
Allows clients and servers to negotiate an upgrade to the application protocol
used on the connection. Used with 101 response code.
Via
Records intermediaries visited by a request type of TRACE
For Requests Only
HOST
Used to supplement a URI that is not absolute:
GET /pub/WWW/TheProject.html HTTP/1.1
Host: www.w3.org