Looks very useful. But it seems the hard part is building the manifest. If I've got a bucket with millions of objects of different types, I have to have a separate function that walks all the objects and decides whether to put them into the manifest. Say I have an S3 with images and video, and each media file has an associated metadata.json file -- I only want to process the metadata.json to stick info into a search engine. Would be nice if I could specify a *pattern* and let Batch walk all the objects and only run against the keys matching the regex pattern, like /{image|video}/.+/metadata.json or something.

Chris Shenton Agree that it's the tricky part.

I was thinking about adding a command to the SLS plugin that would help you build a manifest. Basically specify a bucket and pattern as you mentioned, and it would crawl the bucket and add the items to the manifest.

Only downside is that it could take quite a while if you had millions of objects...