Skip to content
This repository was archived by the owner on Oct 12, 2023. It is now read-only.
This repository was archived by the owner on Oct 12, 2023. It is now read-only.

HTTPError: 403 Client Error In merge task foreach %dopar%  #370

@Pullarg

Description

@Pullarg

I have been using Azure Batch for a while and have come across a new error in the merge task even on the sample code. this appears to happen with any size of data while running the foreach %dopar%. has anyone else noticed this?

HTTPError: 403 Client Error: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. for url: https://_batchaccount._blob.core.windows.net/job12345?sv=2016-05-31&sr=c&st=2020-07-18T21%3A09%3A30Z&se=2020-07-21T21%3A09%3A30Z&sp=rwcl&sig=signature%3D&include=metadata&restype=container&comp=list&maxresults=1000

Reproduce Example
generateCredentialsConfig("credentials.json")

setCredentials("credentials.json")
generateClusterConfig("cluster.json")

cluster <- makeCluster("cluster.json")

Register your parallel backend
registerDoAzureParallel(cluster)

setVerbose(FALSE)
setAutoDeleteJob(FALSE)

mean_change = 1.001
volatility = 0.01
opening_price = 100

getClosingPrice <- function() {
days <- 1825 # ~ 5 years
movement <- rnorm(days, mean=mean_change, sd=volatility)
path <- cumprod(c(opening_price, movement))
closingPrice <- path[days]
return(closingPrice)
}

Optimize runtime. Chunking allows running multiple iterations on a single R instance.
opt <- list(chunkSize = 10)
start_p <- Sys.time()
closingPrices_p <- foreach(i = 1:1000, .combine='c', .options.azure = opt) %dopar% {
replicate(100000, getClosingPrice())
}
end_p <- Sys.time()

hist(closingPrices_s)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions