Why Do the Object Counts in S3 Buckets Differ Between the S3 Console and AWS CLI?

Emily
Jun 4
2 min read

When checking the object count in an S3 bucket with versioning enabled, you may notice a discrepancy between the number of objects displayed in the S3 console and the count returned by the AWS CLI. The S3 console shows the total number of objects, including non-current versions, while the AWS CLI's ls command returns only the count of the latest version of the objects.

Problem Overview

You checked the object count in your S3 bucket and found that the values differ between the S3 console and the AWS CLI. Here’s what you observed:

S3 Console

In the S3 console's metrics tab, the total number of objects is displayed as 7.

AWS CLI

When using the ls command to check the total number of objects, it shows 5.

$ aws s3 ls s3://test-20210913-ver --recursive --human-readable --summarize
2021-09-13 10:20:51  131.5 KiB S3①.png
2021-09-13 10:20:52  192.9 KiB S3②.png
2021-09-13 10:17:55  116.5 KiB S3③.png
2021-09-13 10:17:55  136.8 KiB S3④.png
2021-09-13 10:17:56  155.2 KiB S3⑤.png

Total Objects: 5
Total Size: 732.8 KiB

Why Are the Results Different?

In this S3 bucket, versioning is enabled, and there are 5 current objects and 2 non-current objects stored.

While the S3 console displays the total number of objects, including non-current versions, the AWS CLI's ls command only returns the count of the latest version of the objects.

Note: The total number of objects displayed in the S3 console reflects the NumberOfObjects metric.

NumberOfObjects Metric

This metric represents the total number of objects stored in the bucket across all storage classes. To calculate this value, it counts all objects in the bucket (both current and non-current versions) and the total part count of all incomplete multipart uploads for the bucket.

What Happens with the sync Command?

When you use the sync command to copy objects from a versioned bucket to another bucket, only the latest version of the objects will be copied.

$ aws s3 sync s3://test-20210913-ver s3://copy-20210917-test
copy: s3://test-20210913-ver/S3①.png to s3://copy-20210917-test/S3①.png
copy: s3://test-20210913-ver/S3③.png to s3://copy-20210917-test/S3③.png
copy: s3://test-20210913-ver/S3②.png to s3://copy-20210917-test/S3②.png
copy: s3://test-20210913-ver/S3⑤.png to s3://copy-20210917-test/S3⑤.png
copy: s3://test-20210913-ver/S3④.png to s3://copy-20210917-test/S3④.png

After running the following command to list the objects in the destination bucket:

$ aws s3 ls s3://copy-20210917-test --recursive --human-readable --summarize
2021-09-17 09:59:48  131.5 KiB S3①.png
2021-09-17 09:59:48  192.9 KiB S3②.png
2021-09-17 09:59:48  116.5 KiB S3③.png
2021-09-17 09:59:48  136.8 KiB S3④.png
2021-09-17 09:59:48  155.2 KiB S3⑤.png

Total Objects: 5
Total Size: 732.8 KiB

References

By understanding the differences in how the S3 console and AWS CLI report object counts, you can better manage your S3 buckets and ensure accurate tracking of your stored objects. For more detailed information and additional context, you can refer to the original post here.