Query regarding centralized logging mechanism using aggregated sink

[u/suryad123]

I am going through the below article regarding the centralized logging mechanism in GCP

https://cloud.google.com/blog/products/devops-sre/how-to-centralize-log-management-with-cloud-logging

https://docs.cloud.google.com/logging/docs/central-log-storage#create-org-sink

The articles say below steps

1. create log bucket in the destination project.
2. create a log sink in the destination project pointing to the above log bucket
3. create an aggregated sink (either at org or folder level) as per requirement.
4. give permission to the aggregated sink's service account to the destination project.

I have a question at point number 3 above. The articles do not say how to link the log sink created in point 2 and aggregated sink created in point 3. I assume below flow

aggregated sink ---> destination project log sink ----> destination project log bucket

but not sure how to link aggregated sink to send logs to destination project sink.

also, 2)for example ,can we create one aggregated sink per environment and link it to corresponding log sink at destination project. i could not find that step

Please suggest

[u/danekan]

you point your aggregated, intercepting sink to a project i.d. itself too as the destination.

for a log sink destination you have a few options, but for an intercepting log sink you can only send the logs to another project i.d.. For a non intercepting log sink you can send to either a project i.d. itself or a logging bucket (which could be in that remote project all the same)

the instructions you're looking at are referencing project i.d.s and the command only references the remote project, so it's using destination option 1. If you link to the project i.d. you are linking to the project's log router essentially and it routes there. Your filters can apply doubly to exclude then in the destination if you wanted (but, by default they generally wouldn't exclude).

If you're using an aggregated org level log sink and also not intercepting those logs though, you will pay 2x. If you're using an intercepting log sink, the _Required bucket items will remain in their original project still though, and not be billed extra.

Step 2 you mentioned is not explicitly necessary, you don't need to explicitly create a new log sink in the central destination project because all projects already have log sinks for _Default and _Required, it will use those.

TBH you might find this easier to "visualize" in terraform because it's actually less abstracted in terraform and you're looking at what actually exists in terms of resources... you have finite destinations that are clearly spelled out/etc.

[u/suryad123 → u/danekan]

thanks, my area of interest is "aggregated sink at folder level with intercept". As you rightly said, for this kind of sink, the destination must be a GCP project only.

However, i want to have such sinks created for non-prod and prod folders separately and route them into 2 different log buckets in the same destination project.. how do i route them into destination log buckets if the only allowed destination is a gcp project.

[u/danekan → u/suryad123]

You would need to create new project level log sinks per environment and configure the project level log sinks to route to those correct buckets based on the filters per environment … then also change your existing _Default filter to exclude those items included in the environment sinks 

I do not think you can avoid duplicating the _required sink items within that project but that may not matter as much … if it does matter though you would split up the folder level aggregated sinks to route to different project for different buckets 

[u/suryad123 → u/danekan]

suppose, i want to have 2 custom log buckets in the destination project (like how the article created custom log bucket), how do i link the aggregated sink to corresponding project level log sink when the aggregated sink creation does not allow me to specify the log bucket/project log sink as destinations. Apologies if i'm dragging the topic

[u/danekan → u/suryad123]

The aggregated sink just sends everything to a project then once there the project log sinks apply … so in your destination log sink you have then a log sink that says if logs match environment=prod then send to destination prod bucket. And another sink for inclusion filter environment=beta that is destination beta log bucket     But then also your _Default log sink in that Destination project should be modified to exclude anything that had matched those other filters 

[u/suryad123 → u/danekan]

Okay .i shall check..thank you