Skip to content

Commit a07ee8d

Browse files
Shakir Ahmed IbrahimShakir Ahmed Ibrahim
Shakir Ahmed Ibrahim
authored and
Shakir Ahmed Ibrahim
committed
added blogs from dev.to
1 parent 819b7fb commit a07ee8d

7 files changed

+2034
-0
lines changed

‎_posts/2022-08-29-aws-cloudtrail-log-file-validation.md‎

Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
---
2+
canonical_url: https://dev.to/aws-builders/aws-cloudtrail-log-file-validation-1ehl
3+
categories: aws, cloudtrail, logging, s3
4+
date: 2022-08-29
5+
tags: aws, cloudtrail, logging, s3
6+
title: AWS CloudTrail log file validation
7+
---
8+
9+
This post first appeared on [dev.to](https://dev.to/aws-builders/aws-cloudtrail-log-file-validation-1ehl)
10+
11+
## Introduction
12+
CloudTrail lets us log all API calls in our AWS cloud. In this post, we shall see how to create a CloudTrail, see where the logs are stored
13+
in S3, delete log, digest files and perform log file validation.
14+
15+
## Create a CloudTrail
16+
Search for CloudTrail on the AWS console and create a trail.
17+
![Create cloud trail](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5ojcn1fxyhmdz2bvzvzi.png)
18+
19+
## S3 bucket
20+
A bucket should be automatically created and associated with the CloudTrail. A folder with the name CloudTrail should appear on the bucket
21+
where all the cloud trail logs should get saved.
22+
![S3 bucket for cloud trail](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yw5zei5v3ouuie7f0mbn.png)
23+
24+
## Generate logs
25+
Now let's do an activity and see if it gets logged. Create a lambda function with name helloWorld and all other settings as default. You can
26+
do any other activity on AWS cloud as well, instead of creating a function.
27+
![Lambda function](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ntkzldnsrkx2xenqdsb9.png)
28+
29+
We should see some files on S3 for this activity.
30+
![Log files on S3](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x5oayy9diyj1w6kui5kf.png)
31+
32+
## Delete log file
33+
I am deleting one of the log files.
34+
![Deleting log file](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zle3jv7vq8m0eedykz19.png)
35+
36+
Log files are not suppose to be modified/deleted, as they can hold important auditing information, so now we need to find if our log files
37+
are modified or deleted(as in this case).
38+
39+
We try to validate now from the [AWS CLI](https://aws.amazon.com/cli/), it should say the digest file doesn't exit.
40+
```
41+
$ aws cloudtrail validate-logs --trail-arn arn:aws:cloudtrail:ap-south-1:<accoount-id>:trail/management-events --start-time 2022-08-29
42+
Validating log files for trail arn:aws:cloudtrail:ap-south-1:<account-id>:trail/management-events between 2022-08-29T00:00:00Z and
43+
2022-08-29T06:26:38Z
44+
45+
Results requested for 2022-08-29T00:00:00Z to 2022-08-29T06:26:38Z
46+
No digests found
47+
```
48+
49+
This is because we have not enabled log file validation for the cloud trail.
50+
51+
## Enable Log file validation
52+
We can enable log file validation, by editing the cloud trail.
53+
![Enable log file validation](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1s7p6e0r18dpgjs11as3.png)
54+
55+
56+
## Digest
57+
As the log file validation is enabled, we should see a new folder CloudTrail-Digest in S3.
58+
![CloudTrail-Digest folder in S3](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ixq3invj5kug2y1m5hfj.png)
59+
60+
And digest files should get added each hour.
61+
![Digest file in S3](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/36mc4wdwlsucq7a12n26.png)
62+
63+
64+
## Validate
65+
As we enabled Log file validation, we can now check the integirty of the logs.
66+
```
67+
$ aws cloudtrail validate-logs --trail-arn arn:aws:cloudtrail:ap-south-1:<account-id>:trail/management-events --start-time 2022-08-29
68+
Validating log files for trail arn:aws:cloudtrail:ap-south-1:<account-id>:trail/management-events between 2022-08-29T00:00:00Z and
69+
2022-08-29T07:00:20Z
70+
71+
Results requested for 2022-08-29T00:00:00Z to 2022-08-29T07:00:20Z
72+
Results found for 2022-08-29T05:55:08Z to 2022-08-29T06:55:08Z:
73+
74+
1/1 digest files valid
75+
```
76+
Though we deleted a log file earlier, it shows the digest is valid, because we did not enable log file validation then.
77+
78+
After an hour, we should see two digest files.
79+
![Digest files](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vsot6bvy7kzsq632poii.png)
80+
81+
The log file validation seems good for now.
82+
```
83+
$ aws cloudtrail validate-logs --trail-arn arn:aws:cloudtrail:ap-south-1:<account-id>:trail/management-events --start-time 2022-08-29
84+
Validating log files for trail arn:aws:cloudtrail:ap-south-1:<account-id>:trail/management-events between 2022-08-29T00:00:00Z and
85+
2022-08-29T08:17:57Z
86+
87+
Results requested for 2022-08-29T00:00:00Z to 2022-08-29T08:17:57Z
88+
Results found for 2022-08-29T05:55:08Z to 2022-08-29T07:55:08Z:
89+
90+
2/2 digest files valid
91+
10/10 log files valid
92+
```
93+
94+
## Delete log file with validation check
95+
We can try deleting log file that was created after enabling log file validation.
96+
![Delete another log file](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rmkawn7l1s5hw91o8vmb.png)
97+
98+
As expected the log file validations fails for one file. However the digests are still valid.
99+
```
100+
$ aws cloudtrail validate-logs --trail-arn arn:aws:cloudtrail:ap-south-1:<account-id>:trail/management-events --start-time 2022-08-29
101+
Validating log files for trail arn:aws:cloudtrail:ap-south-1:<account-id>:trail/management-events between 2022-08-29T00:00:00Z and
102+
2022-08-29T08:22:42Z
103+
104+
Log file
105+
s3://aws-cloudtrail-logs-<account-id>-4a8dcb98/AWSLogs/<account-id>/CloudTrail/ap-south-1/2022/08/29/<account-id>_CloudTrail_ap-south-1_20220829T0755Z_7rDSVFC6Icgi9Z8V.json.gz
106+
INVALID: not found
107+
108+
Results requested for 2022-08-29T00:00:00Z to 2022-08-29T08:22:42Z
109+
Results found for 2022-08-29T05:55:08Z to 2022-08-29T07:55:08Z:
110+
111+
2/2 digest files valid
112+
9/10 log files valid, 1/10 log files INVALID
113+
```
114+
It also clearly says the validation failed because it can't find a file that we deleted.
115+
116+
## Delete digest
117+
This time we can try deleting a digest file.
118+
![Delete digest](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vzzw970xszdwllgpqqdp.png)
119+
120+
Hence digest validation should also fail.
121+
```
122+
$ aws cloudtrail validate-logs --trail-arn arn:aws:cloudtrail:ap-south-1:<account-id>:trail/management-events --start-time 2022-08-29
123+
Validating log files for trail arn:aws:cloudtrail:ap-south-1:<account-id>:trail/management-events between 2022-08-29T00:00:00Z and
124+
2022-08-29T10:09:35Z
125+
126+
Digest file
127+
s3://aws-cloudtrail-logs-<account-id>-4a8dcb98/AWSLogs/<account-id>/CloudTrail-Digest/ap-south-1/2022/08/29/<account-id>_CloudTrail-Digest_ap-south-1_management-events_ap-south-1_20220829T085508Z.json.gz
128+
INVALID: not found
129+
130+
Log file
131+
s3://aws-cloudtrail-logs-<account-id>-4a8dcb98/AWSLogs/<account-id>/CloudTrail/ap-south-1/2022/08/29/<account-id>_CloudTrail_ap-south-1_20220829T0755Z_7rDSVFC6Icgi9Z8V.json.gz
132+
INVALID: not found
133+
134+
Results requested for 2022-08-29T00:00:00Z to 2022-08-29T10:09:35Z
135+
Results found for 2022-08-29T05:55:08Z to 2022-08-29T09:55:08Z:
136+
137+
3/4 digest files valid, 1/4 digest files INVALID
138+
20/21 log files valid, 1/21 log files INVALID
139+
```
140+
141+
Note that we can enable versioning on S3 buckets to restore files.
142+
143+
## Summary
144+
So we saw how the log file validation feature in CloudTrail helps us find if there were any manual modifications to the log files or digest
145+
files. Thank you for reading !!!

0 commit comments

Comments
 (0)