Let us take a closer look at the aws DynamoDB batch-write-item and how to perform a batch-write in DynamoDB with the support of our AWS support services at Bobcares.

What is BatchWriteItem?

BatchWriteItem inserts or deletes many items from one or more tables. A single call to Batch-Write-Item can send up to 16MB of data over the network. This can include up to 25 item insert or delete operations.

Individual items can be up to 400 KB when saved. However, an item’s representation can be more than 400KB when supplied in DynamoDB’s JSON format for the API call.

The PutItem and DeleteItem operations in BatchWriteItem are atomic, but BatchWriteItem as a whole is not. If any operation fails due to exceeding the table’s throughput or an internal processing fault, the unsuccessful operations are returned in the UnprocessedItems response parameter.

We can investigate and, if necessary, resubmit the requests. In a loop, we would typically call BatchWriteItem. Each cycle would look for items wait for processing and send a new BatchWriteItem request with those items until the processing of all items.

If the processing on items fails because there is insufficient provisioned throughput on all of the tables in the request, BatchWriteItem throws a ProvisionedThroughputExceededException.

If the aws DynamoDB returns any unprocessed items, the Batch-Write-Item operation on those things should be retried. However, it strongly suggests that we employ an exponential backoff algorithm.

Even if we immediately restart the batch process, the underlying read or write requests may still fail due to throttling on the individual tables.

Individual requests in the batch are substantially more likely to succeed if the batch operation is delayed using exponential backoff.

Batch-write-item and AWS

We may efficiently write or remove massive volumes of data, such as from Amazon EMR, or copy data from another database into AWS DynamoDB using Batch-Write-Item.

Batch-Write-Item does not act in the same manner as individual PutItem and DeleteItem calls in order to increase efficiency with these large-scale operations.

Individual put and delete requests, for example, cannot have conditions specified, and BatchWriteItem does not return removed items in the response.

Threads can write objects in parallel if we use a programming language that supports concurrency. The thread management logic must be included in the program. We must update or delete the given items one at a time in languages that do not enable threading.

In the AWS DynamoDB, the Batch-Write-Item performs the specified put and deletes operations in parallel in both cases, providing us the power of the thread pool technique without adding complexity to the application.

Although parallel processing improves latency, each put and delete request requires the same number of write capacity units regardless of whether it is executed in parallel or not. Delete operations on objects that do not exist using one write capacity unit.

Criteria for DynamoDB reject the entire batch-write operation

If we met any of the following conditions, DynamoDB rejects the entire batch write operation:

• The request for one or more tables given in the BatchWriteItem does not exist.
• The primary key characteristics given for an item in the request do not match those in the primary key schema of the related table.
• When attempting numerous operations on the same item in the single BatchWriteItem request. We cannot, for example, put and delete the same item in the same BatchWriteItem request.
• The request includes at least two items with the same hash and range keys.
• There are more than 25 requests in the batch.
• Any single item in a batch is more than 400 KB.
• The overall size of the request exceeds 16 MB.

Synopsis

batch-write-item
--request-items
[--return-consumed-capacity ]
[--return-item-collection-metrics ]
[--cli-input-json ]
[--generate-cli-skeleton ]
[--debug]
[--endpoint-url ]
[--no-verify-ssl]
[--no-paginate]
[--output ]
[--query ]
[--profile ]
[--region ]
[--version ]
[--color ]
[--no-sign-request]
[--ca-bundle ]
[--cli-read-timeout ]
[--cli-connect-timeout ]

AWS DynamoDB Batch Write

In AWS DynamoDB, a batch-write-item allows us to write numerous entries into several tables in a single API call. It uses the BatchWriteItem action to combine many write requests into a single API call.

This reduces the number of network calls and thereby improves application speed and latency. It is important to note, however, that while doing bulk writes, DynamoDB does not enable us to apply condition expressions on objects (“attribute not exists()”).

How to perform a Batch-Write in AWS DynamoDB?

To execute a batch-write, we can use the document client’s batch-Write method. Consider the following sample snippet.

Part: 1

const aws = require("aws-sdk");
const documentClient = new aws.DynamoDB.DocumentClient({ region: "us-east-1" });

const tableName = "dynobase-batch-operations"; const hashKey = "id";

const batchWrite = async () => {
await documentClient
.batchWrite({
RequestItems: {
[tableName]: [
{
PutRequest: {
Item: {
[hashKey]: "1",
name: "abc",
age: "19",
country: "USA",
},
},
},
{
PutRequest: {
Item: {
[hashKey]: "2",
name: "bcd",
age: "19",
country: "Canada",
},
},
},
{
PutRequest: {
Item: {
[hashKey]: "3",
name: "cdb",
age: "19",
country: "Norway",
},
},
},
],
},
}).promise();
};

batchWrite();
const aws = require("aws-sdk");
const documentClient = new aws.DynamoDB.DocumentClient({ region: "us-east-1" })

Part 2: to perform a batch-write-item in AWS DynamoDB

const tableName = "dynobase-batch-operations";

const hashKey = "id";

const batchWrite = async () => { await documentClient .batchWrite({ RequestItems: { [tableName]: [ { PutRequest: { Item: { [hashKey]: "1", name: "ABCD", age: "19", country: "USA", }, }, }, { PutRequest: { Item: { [hashKey]: "2", name: "bcd", age: "19", country: "USA", }, }, }, { PutRequest: { Item: { [hashKey]: "3", name: "cdb", age: "19", country: "Norway", }, }, }, ], }, }).promise(); }; batchWrite();

The snippet above employs the Document Client’s bulk write mechanism. The Document Client calls the BatchWriteItem action, which writes the three data items to a single table in a single batch operation.

If we want to write several data items to other tables, we may add more tables to the RequestItems and define Put procedures for each table’s data items.

[Need assistance with similar queries? We are here to help]

Conclusion

To conclude we have learned more about how to perform batch-write-item in AWS DynamoDB in a few simple steps with the assistance of our AWS support services.