AWS has announced the preview launch of queryable object metadata for Amazon S3 buckets. This feature allows users to query object metadata in S3 buckets using Iceberg-compatible tools such as Athena, Redshift, QuickSight, and Spark. This addresses the challenge of managing massive amounts of data on S3, as users can now search for objects based on specific criteria like key, size, and tags. With this feature, users can improve their data analytics, data processing, and AI training workflows. An interesting aspect of this feature is its ability to automatically capture rich metadata when S3 objects are added or modified. This metadata is stored in fully managed Apache Iceberg tables, allowing for efficient querying. This will be particularly useful for organizations that use S3 to store large datasets and need an efficient way to find specific objects within those datasets. This feature will simplify data discovery and improve overall efficiency.
Queryable Object Metadata for Amazon S3 Buckets (Preview)
AWS