scorebook.inference.bedrock

AWS Bedrock batch inference implementation for Scorebook.

This module provides utilities for running batch inference using AWS Bedrock's Model Invocation Jobs, supporting large-scale asynchronous processing. It handles API communication, request formatting, response processing, and S3 operations.

batch

async def batch(items: List[Any],
                model: Optional[str] = None,
                aws_region: Optional[str] = None,
                aws_profile: Optional[str] = None,
                bucket: Optional[str] = None,
                input_prefix: Optional[str] = None,
                output_prefix: Optional[str] = None,
                role_arn: Optional[str] = None,
                **hyperparameters: Any) -> List[Any]

Process multiple inference requests in batch using AWS Bedrock.

This asynchronous function handles batch processing of inference requests, optimizing for cost and throughput using AWS Bedrock's Model Invocation Jobs.

Arguments:

• items - List of preprocessed items to process.
• model - Bedrock model ID (e.g., 'us.anthropic.claude-3-5-sonnet-20241022-v2:0').
• aws_region - AWS region for Bedrock and S3.
• aws_profile - AWS profile name for authentication.
• bucket - S3 bucket name for input/output data.
• input_prefix - S3 prefix for input data.
• output_prefix - S3 prefix for output data.
• role_arn - IAM role ARN for Bedrock execution.
• hyperparameters - Additional parameters for the batch requests.

Returns:

A list of raw model responses.

s3_uri_to_bucket_and_prefix

def s3_uri_to_bucket_and_prefix(s3_uri: str) -> Tuple[str, str]

Parse S3 URI to bucket and prefix.