TechOverflow Logo
TechOverflow

如何使用 boto3 筛选给定 S3 目录中的对象

English Deutsch 简体中文

使用 boto3，你可以通过应用前缀筛选器按目录筛选给定 bucket 中的对象。

而不是使用以下命令遍历所有对象

s3_filter_example.py
for obj in my_bucket.objects.all():
    pass # ...

（完整示例请参见如何使用 boto3 遍历 Wasabi / S3 bucket 中的所有对象）

你可以使用以下命令应用前缀筛选器

s3_filter_example.py
for obj in my_bucket.objects.filter(Prefix="MyDirectory/"):
    print(obj)

不要忘记前缀参数的尾部 /！仅使用 filter(Prefix="MyDirectory") 而不带尾部斜杠也会匹配例如 MyDirectoryFileList.txt。

此完整示例打印 10k-Test-Objects 目录中每个对象的对象描述（来自我们关于如何使用 boto3 在 Wasabi / S3 中创建大量测试文件的文章）。

s3_filter_full_example.py
import boto3

# 创建到 Wasabi / S3 的连接
s3 = boto3.resource('s3',
    endpoint_url = 'https://s3.eu-central-1.wasabisys.com',
    aws_access_key_id = 'MY_ACCESS_KEY',
    aws_secret_access_key = 'MY_SECRET_KEY'
)

# 获取 bucket 对象
my_bucket = s3.Bucket('boto-test')

# 遍历 bucket 中的对象
for obj in my_bucket.objects.filter(Prefix="MyDirectory"):
    print(obj)

Don’t forget to fill in MY_ACCESS_KEY and MY_SECRET_KEY. Depending on what region and what S3-compatible service you use, you might need to use another endpoint URL instead of https://s3.eu-central-1.wasabisys.com.

示例输出：

s3_object_examples.txt
s3.ObjectSummary(bucket_name='boto-test', key='10k-Test-Objects/1.txt')
s3.ObjectSummary(bucket_name='boto-test', key='10k-Test-Objects/10.txt')
s3.ObjectSummary(bucket_name='boto-test', key='10k-Test-Objects/100.txt')
s3.ObjectSummary(bucket_name='boto-test', key='10k-Test-Objects/1000.txt')
s3.ObjectSummary(bucket_name='boto-test', key='10k-Test-Objects/10000.txt')
s3.ObjectSummary(bucket_name='boto-test', key='10k-Test-Objects/1001.txt')
s3.ObjectSummary(bucket_name='boto-test', key='10k-Test-Objects/1002.txt')
s3.ObjectSummary(bucket_name='boto-test', key='10k-Test-Objects/1003.txt')
s3.ObjectSummary(bucket_name='boto-test', key='10k-Test-Objects/1004.txt')
s3.ObjectSummary(bucket_name='boto-test', key='10k-Test-Objects/1005.txt')
s3.ObjectSummary(bucket_name='boto-test', key='10k-Test-Objects/1006.txt')
s3.ObjectSummary(bucket_name='boto-test', key='10k-Test-Objects/1007.txt')
s3.ObjectSummary(bucket_name='boto-test', key='10k-Test-Objects/1008.txt')
s3.ObjectSummary(bucket_name='boto-test', key='10k-Test-Objects/1009.txt')
s3.ObjectSummary(bucket_name='boto-test', key='10k-Test-Objects/101.txt')
[...]
Check out similar posts by category: Python, S3