More headers

31st May 2024

Oh wow, it’s the 31st of May already and I wanted to publish to the blog before the month is out. Luckily I did the coding two weeks ago and already drafted the post, I should get a move on.

I wanted to round out the functionality of Tinn to make it a compliant web server. This essentially means supporting a few more headers, correctly closing the connection when appropriate and adding support for the HEAD method.

Host header

HTTP requests must include the host header. This is so web servers hosting multiple websites on a single IP & socket know what site to return. This is a requirement for HTTP/1.1 and the server should reject any request without the header with a response code of 400, aka Bad Request. Technically it's an optional header for HTTP/1.0 but most clients include it anyway and some servers reject requests without it.

The server doesn't actually need to do anything with the header, that is outside the spec, but it must validate it. So I added the check. Currently I don't need to use it for anything, but one day I will.

Connection header

The connection header in HTTP requests defines if the connection should be closed once the response is sent or kept open. The default behaviour if the header is not supplied is dependent on the HTTP version. For 1.0 the connection should close, for 1.1 it should remain open.

I added code to read the header or set its default value if not supplied.

if (request->connection.length==0) {
	if (token_is(request->version, "HTTP/1.0")) {
		request->connection = default_header("close");
	} else {
		request->connection = default_header("keep-alive");
	}
}

I then changed the response code to honour the setting once the response had been sent.

static bool send_response(struct pollfd* pfd, ClientState* state) {
	Response* response = state->response;

	ssize_t sent = response_send(response, pfd->fd);
	if (sent < 0) {
		ERROR("send error for %s (%d)", state->address, pfd->fd);
		return false;
	}
	if (response->stage != RESPONSE_DONE) {
		pfd->events = POLLOUT;
	} else {
		Request* request = state->request;
		if (token_is(request->connection, "close")) {
			return false;
		}
		request_reset(request);
		response_reset(response);
		pfd->events = POLLIN;
	}
	return true;
}

Testing this was a pain because all the common clients I use use HTTP/1.1 and either send a connection header set to keep-alive or no connection header at all. I was surprised to learn that both curl and wget don't set the connection header to close and instead close the connection themselves after receiving the response. I therefore had to break out the telnet client and craft some tests by hand.

HEAD method

A compliant server must support both GET and HEAD methods. The HEAD method generates a response that includes the same headers as a GET request but without the request body. I guess the idea is (or was) that clients could perform a HEAD request to get details about the required resource (like if it is a valid resource, when it was last modified, what size it is) before committing to a potentially costly download. The reality is I don't think any common clients take advantage of this option and as a result some professional servers don't implement support for the HEAD method despite being part of the standard.

I'm being a good developer and added support anyway. This required adding a new content source to my response module so content generators could record the content type and length without storing the actual content. I updated the static files module to record the file size (aka content length) for HEAD requests and skip the actual reading of the file. For the blog module, I still generate the response content as this is currently the easiest way to compute the content length.

Testing this was fairly easy as a -I option to curl will make a HEAD request.

> curl -I localhost:8080
HTTP/1.1 200 OK
Date: Fri, 31 May 2024 22:19:31 GMT
Server: Tinn
Content-Type: text/html; charset=utf-8
Content-Length: 242439
Cache-Control: no-cache
Last-Modified: Fri, 31 May 2024 22:17:44 GMT

Summary

With that, I am fairly confident the server is compliant with the standards, except for one thing. I did a search for all the MUST requirements in the spec, it shows up 212 times and I think I have all of the applicable ones covered except for parsing obsolete date formats in header fields. They are obsolete formats so I should be safe, still I’ll add that one to the backlog for another day.