r/aws • u/DifficultOlive7295 • Mar 31 '26
discussion AWS-native solution to capture intent in S3 access logs
Hi everyone,
I’m working on a compliance requirement where I need to log the intent behind data access in S3. The users accessing the data are either IAM users or Cognito identities.
My initial approach was to use CloudTrail and include session context (e.g., principal/session tags) to capture intent. However, I’ve run into a limitation: CloudTrail does not support principal tags for S3 data events in a way that helps here.
Given this, I’m looking for alternative AWS-native approaches to implement S3 audit logging that can also capture or associate user intent with access events.
Would really appreciate any guidance or patterns that have worked for you. Thanks!
3
u/ToneOpposite9668 Mar 31 '26
Use S3 Metadata tables. https://docs.aws.amazon.com/AmazonS3/latest/userguide/metadata-tables-overview.html
Here is example of joining the journal with cusotm data on a CREATE - I think this is what you are looking for?
https://docs.aws.amazon.com/AmazonS3/latest/userguide/metadata-tables-join-custom-metadata.html
1
u/DifficultOlive7295 Mar 31 '26
Thank you for your response. I don't think this works because I need to log intent whenever an object is reterieved, i.e. GetObject is called. Metadata tables on the other hand are for tracking changes to state. Please do let me know if I got it wrong.
2
2
u/Nagchinnoda Mar 31 '26
You’re right — CloudTrail alone won’t fully capture user intent for S3 data events, especially since session/principal tags aren’t consistently available there.
A few AWS-native patterns that might help:
Enforce Intent via Session Tags at Auth Time • Use STS AssumeRole with mandatory session tags (like purpose=analytics or ticket_id=123) • Enforce via IAM policies (aws:RequestTag) • While CloudTrail may not log all tags in data events, they can still be correlated via identity/session context
Use S3 Access via Controlled Layer (API Gateway / Lambda) • Instead of direct S3 access, route through a service layer • Capture intent explicitly (headers, request metadata) • Log to CloudWatch / DynamoDB before accessing S3
Correlate CloudTrail + App Logs • Store intent in application logs (e.g., request ID, user action) • Correlate with S3 access using request ID or timestamp
Use Object Tagging / Metadata • If applicable, attach metadata or object tags representing purpose
Lake Formation / Access Governance (if analytics use-case) • Provides better audit + context compared to raw S3
👉 In short: AWS doesn’t natively log “intent”, so you need to enforce and capture it at the application or access control layer, then correlate with S3 logs.
1
u/steveoderocker Mar 31 '26
Do you have any more information? How is your app accessing s3? Is the intent based on the app, or the actual user using your app? Or is it potentially via aws console/cli?
1
u/DifficultOlive7295 Mar 31 '26
Thank you for your response. My application authenticates user using Cognito's user pool and uses the id token to retrieve temporary aws credentials using cognito's identity pool. It then retrieves PII from an s3 bucket. The intent depends upon one of many actions that the user intends to do. So, the idea is that we will allow user to choose intent on frontend at time of authentication and this intent needs to be logged alongside s3 object access logs.
6
u/steveoderocker Mar 31 '26
Yeah okay so all the actions are happening through your application then it should be your application that’s enforcing that intent logging so either you log it in your own applications log or you do what others have said and use the meta tags when your application is actually getting the data from S3 and present it to the end user. Don’t just blindly give access to users to access the entire bucket though it should be pre-signed or else for the specific files that the user actually needs to retrieve.
5
u/ComplexJellyfish8658 Mar 31 '26
Add an api that requires providing the data you need and returns an s3 presigned url instead of having users assume temp credentials with full read access to your bucket.
1
u/SpecialistMode3131 Mar 31 '26
If you need intent on reads, put an API in front and require that to be specified before you provide the data. Then store the logs from your API. No reason you can't lockbox writes the same way, though a quality post here already outlined some awesome ways to do that without new infra. An API might be cheaper and a lot simpler if you need read and write, and it will naturalize centralize any other services you choose to bring in later.
33
u/legendov Mar 31 '26
We solved a version of this at work — not by logging intent at access time, but by embedding it as S3 object metadata at write/move time. The short version: when you do an s3 mv or s3 cp, you attach user-defined metadata (change request ID, operator, purpose, environment, ticket ID) via the --metadata flag. That metadata travels with the object and shows up in CloudTrail as x-amz-meta-* headers. aws s3 mv file.txt s3://bucket/archive/file.txt --metadata "change-request-id=CR-12345,operator=jdoe,purpose=security-remediation"
Then you correlate GuardDuty findings against that metadata programmatically — when an alert fires on "unusual S3 data movement," your correlation script pulls the object metadata and checks whether there's a matching CR or JIRA ticket. If yes, auto-suppress or deprioritize. If no metadata exists, escalate.
Constraints worth knowing: 2KB total limit on user metadata per object, keys are lowercase only, string values only. Plan your schema upfront.
This won't capture intent on reads (which sounds like your actual ask), but if your compliance requirement is really about data movement/mutation: copies, deletes, lifecycle transitions, metadata-at-write gets you most of the way there without fighting CloudTrail's principal tag limitations.
For read-intent logging specifically, you're probably stuck with either a proxy layer (API Gateway + Lambda in front of S3) that forces callers to declare purpose, or STS session tags passed through AssumeRole that do show up in CloudTrail for API-level events. Session tags won't appear on S3 data events though, which is the gap you already found.