191 weeks agoHow to use the same S3 account with many users

Sometimes you want to set up an Amazon S3 account and have some users read and write to it while not giving your S3 credentials away. Why would you want this:

  • Maybe you are a contractor who manages his clients’ backups, you pay S3 fees yourself and charge your clients at whatever rate you have negotiated. If you have more than one client you face a problem of isolating everyone’s data.
  • You might want some data to be writable only by you and readable by some subset of users (who do not neccesarily have an S3 account), and you want to be able to revoke the rights to read it at any moment

These are the basic scenarios, but you can imagine that there are others that are much more intricate. Some people suggest that Amazon should build a more sophisticated access control system into S3. I disagree, if they will try to satisfy everyone’s need for fine-grained access control they will have to build a very complicated system just for that. A better solution is to separate the permission control and in fact it can be done to some extent with what we have right now.

To implement it we need a compatible software and a permission control server. Note that this server is not a proxy, its purpose is only to decide if certain request should be permitted or denied. That is, on every action client software submits a summary of an action it’s about to take, for example “to read a certain file in a bucket”, permission control server (let’s call it PCS) then uses whatever logic it needs to make a decision and if the permission is granted returns a “signed URL” to execute that action. Signed URLs in S3 have an expiration time, so one doesn’t need to worry — once that action was permitted client has only a limited time for executing the action, if it were to repeat it, a different signed URL has to be issued.

The beauty of this setup is that we have no restrictions on the complexity of access control logic and we can use any authentification scheme we want, we can use LDAP auth for ex. or anything else for that matter, this wouldn’t be possible if the access control was implemeted entirely by Amazon S3 means. We don’t need the PCS to be powerful (however we do need it to be online at all times) — it only makes decisions on access control and is not part of the actual data transfer.

I know some people who use S3 Backup in beta are interested in this kind of functionality, so someday I will add that and accompany it with some basic PHP scripts you could use to implement your own compatible Permission Control Servers. If you feel curious (you should be if you want to use Amazon S3 storage for corporate backup or private file sharing or if you want to make money reselling the backup services), so if you are — contact me and explain in a few paragraphs how are you planning to use this kind of setup and we’ll see what can be done.