GET, POST, safety, idempotency
About a year ago I wrote a short article asking when it’s okay to use GET to do POST’s job. Since then I’ve learnt a bit about web standards and web pragmatism, but also about the specifics: safety and idempotency.
Recently I found a blog devoted to well-designed URLs. I love well-designed URLs (enough to have made DecentURL.com, a web service that turns ugly URLs into decent ones). For instance, there’s no question about which of these URLs is better:
http://micropledge.com/projects/modwsgi http://micropledge.com/app.cgi?type=3&id=2401
Unfortunately, I think Mike Schinkel got a bit carried away in his somewhat ranty post about how SnipURL’s GET-based API is bad.
On the web, it’s not simply a case of “GET is always evil for requests that aren’t safe”. I’m afraid he’s ignoring the evidence that GET sometimes just works better. Paul Buchheit sums it up nicely:
There’s no question that POST is the “right” way and generally safer, but sometimes it’s annoying. Even though GET isn’t “supposed” to work, it often can be made to. Don’t believe me? Google does billions of dollars a year in GET based transactions in the form of CPC ad clicks (which can cost over $50/click).
Okay, perhaps I’m a touch biased. Perhaps I’m being a little defensive because DecentURL’s API works just like SnipURL’s. :-)
But apart from the pragmatics (“it works”), I believe Mr SnipURL and I are sticking to the standard. It’s not just about GET vs POST. There’s this other little point: the distinction between safe and idempotent.
Safe means a request doesn’t cause any side effects. A safe request just grabs data from a database and display it. Static pages, browsing source code, reading your email online — these are all “safe” requests.
Idempotent means that doing the request 10 times has the same effect as doing it once. An idempotent request might create something in a database the first time, but it won’t do it again. Or it’ll just return the reference to it the next time around. As a friend said to me:
From the browser’s perspective, there is no difference than if the response had always existed for all time prior to the first request. One can cache that response without any perceptible effect, for instance, and bots can request it again and again without damaging anything.
Idempotent is exactly what creating a DecentURL or a SnipURL is, and it’s why we’re allowed to use GET. You do it the first time, and the service creates a record in the database. But there’s no harm in GETting it again — the service simply grabs the existing database entry.
As the HTTP standard notes:
In particular, the convention has been established that the GET and HEAD methods SHOULD NOT have the significance of taking an action other than retrieval. These methods ought to be considered “safe”.
It’s a “should”, a rule of thumb for good reason. But what the standard does actually mandate is that GET must be idempotent (see 9.1.2).
And it seems like the makers of all the popular URL redirection services realise this (TinyURL, SnipURL, Metamark, notlong.com). All of those services either use GET normally, allow GET, or allow GET in the API call that creates a URL.
But what about the spider trap that Mike proposes? (See the PHP code in his blog entry.) Won’t it fill our databases? Won’t it suck our non-safe services into recursive oblivion? Well, apparently it doesn’t happen, or TinyURL and co would have big problems on their hands.
Also, if you create a decent robots.txt, you’ll stop spider-trap problems, at least from good spiders. And if it’s a spider with malicious intent, it could just as easily break a POST-only service as an allow-GETs service. Just like with XSRF, using POST isn’t a catch-all for baddies.
29 March 2008 by Ben 8 comments
8 comments and pings (oldest first)
By the by, a week or so ago I wrote a friendly critique in reply to Mike’s blog entry, saying much of what I’ve said above. But my comment hasn’t appeared yet (it says your comment is awaiting moderation). Maybe he’s still thinking about it, or maybe it was just too long … :-)
“Idempotent means that doing the request 10 times has the same effect as doing it once.”
No, you are wrong. Idempotence does not mean that at all, despite how common it is to make this mistake. You’re confusing yourself with referential transparency, which is something else (though, all idempotent functions are referentially transparent, not necessarily the other way around though).
I wrote about this quite some time ago. http://blog.tmorris.net/idempotence-versus-referential-transparency/
Hi Tony … I don’t know enough about functional programming to comment on “idempotency” in that context, but here I’m using it in the context of the HTTP standard, where it defines idempotency by saying: Methods can also have the property of “idempotence” in that the side-effects of N > 0 identical requests is the same as for a single request.
I could be wrong, and I did give a fairly loose definition, but that sounds pretty much like how I defined it above. Remember, I’m talking about it only in the context of the HTTP spec. Or am I missing something else?
A service can’t assume any intent from the sender of a GET request, except the wish to get information. It’s not that you can’t have side-effects on the server side, just that the client can’t be held responsible, because he never asked for them. If I see a sponsored link on a webpage, it doesn’t tell me (the client) : “Click here to take some money from the advertiser’s Google Ads account”, it tells me “Click here to GET some information about …”, no more than this. The difference lies in the client’s intent. I repeat : whatever behaviour you map to a GET request, never assume the client wants any more than obtaining information. You can’t change the semantics of GET, even if you warn about it having a different semantics for your particular service (your warning should be and will be ignored), because everyone already has a common understanding of GET, defined in RFC2616, and all implementations assume it’s a SAFE method and behave accordingly. Otherwise, you’re breaking the web. Idempotency won’t rescue you if you forget the semantics of SAFE.
You should use POST because:
- it is never cached (way better than trashing cache with random URLs or sending hundreds of HTTP headers begging for no-cache)
- it is never automatically re-sent (proxy won’t try sending it twice if your server is slow)
- it is never pre-fetched (browsers and accelerating proxies might pre-fetch GET even for logged-in users)
- robots.txt is useless against spambots and email harvesters, so your GET links are still exposed
Tony: There’s a particular context in which the word “idempotent” is used in the context of HTTP requests. To map it to the mathematical meaning you’re using, you jump through the normal hoop to model a stateful action as a pure function: consider an HTTP request to be a function f : World -> World, where World is the entire state of the world. The request does something, which turns one state of the world (before the request) into a new one (after the request).
Seen in that way, an idempotent function is precisely one where performing it n times has the same effect as performing it once. This is the normal meaning of “idempotent” in HTTP and web contexts. In this light, the assertion that idempotence implies referential transparency is also false. (And, indeed, I can’t see any way to make it true; if you aren’t restricting yourself to things that look like functions — and are therefore referentially transparent — in the first place, then talking about idempotence in the naive way just doesn’t make sense.)
Nice post. For more info refer this link http://techno-terminal.blogspot.in/2015/08/why-post-method-is-non-idempotent-and.html
[…] If so, you probably don’t understand the right and wrong times to use each properly, the keys are safety and idempotency: It’s not just about GET vs POST. There’s this other little point: the distinction between safe […]