The GSUTIL Wildcard rm delete objects with gsutils following a wildcard pattern. As part of our Google Cloud Platform Support Service, Bobcares provides answers to all of your questions.

GsUTIL Wildcard RM

Removing an object cannot be undone. However, with Google Cloud Storage, developers have remarkable flexibility and control over their data. Also, Google strictly regulates how deleted data is handled and removed. On the bucket(s), we can set up object versioning to safeguard ourself against errors. The gsutil rm command removes objects. Let’s look into the details:

1. The code gsutil rm gs://bucket/subdir/* can delete all objects in gs://bucket/subdir, but not in any of its sub-directories.

2. The code gsutil rm gs://bucket/subdir/** can delete all objects under gs://bucket/subdir or any of its subdirectories.

3. The following commands can remove gs://bucket/subdir and all objects and subdirectories under it:

For versioning-enabled buckets, the ** command will only remove the live version of each item in the subdirectory; in contrast, the -R option will also remove all object versions in the subdirectory.

4. Running gsutil rm -R on a bucket gsutil rm -R gs://bucket will delete all versions of all objects in the bucket, and then delete the bucket.

5. We can use code gsutil rm gs://bucket/** to delete all objects in the bucket, but not the bucket itself.

6. Use the gsutil -m option to execute a parallel (multi-threaded/multi-processing) remove if we have a lot of objects to get rid of: gsutil -m rm -R gs://my_bucket/subdir.

7. Commands like gsutil rm *.txt will not work as gsutil rm won’t delete files from the local file system.

8. In order to delete objects with gsutils following a wildcard pattern, we can try gsutil rm -r gs://bucket/folder/{WILDCARD-A}/folderA/{WILDCARD-B}/folderB applying the wildcard patterns.

We can use the gsutil -m option to allow multi-threading/multi-processing if we have a lot of objects to remove. Avoid using gsutil if we need to mass delete 100,000 or more objects because it takes a long time to finish. Alternatively, we can use Object Lifecycle Management, which can delete any number of objects, or the Google Cloud console, which can delete up to several million things.

9. When using wildcards in gsutil to delete a specific subdirectory, we must enclose the wildcards in single or double quotes.Before gsutil notices it (and complains that no local files match the wildcard), zsh is trying to enlarge the wildcard. Kindly attempt the following to stop zsh from doing so:
gsutil rm 'gs://bucket/**'.

[Want to learn more? Click here to reach us.]

Conclusion

To conclude, the article offers the a detailed explanation on use of GSUTIL RM command in Wildcards.