From the archive
Amazon Bedrock opens managed batch inference
Bedrock adds asynchronous model jobs for large datasets, with results collected after the job completes.
Amazon Bedrock made batch inference generally available on August 21, 2024. The feature processed collections of model requests as a managed job rather than returning every answer interactively.
Work that does not need an immediate reply
AWS illustrated the release with summaries of call-center transcripts. Other datasets could follow the same prepare, submit, and collect workflow when results were not needed in real time.
The console and CreateModelInvocationJob API provided job-submission routes. Input and output data formed part of the job, allowing the results to feed later reporting or application workflows.
A separate route from chat requests
Batch inference added another way to use the models offered through Bedrock. It suited offline processing of a collection of records, while interactive applications still needed a request path that returned responses as the conversation progressed. The release separated those two processing needs within the same hosted platform.