Azure Functions pricing and timeout

Question

I've just noticed recently, that Azure functions acquired a 5 min timeout on the dynamic pricing tier somewhere along the timeline. As I've been busy doing other things, this flew under my radar, until I noticed some long running functions not completing.

So I went digging, and found out that there are two pricing tiers - the dynamic and the app service based. The site is a bit vague on the whole concept, but as I understand it, this is how it stands:

Dynamic: Charged per usage time and memory allocation by the user. 5 minute timeout (so useless for one time long running operations now).

App service: Either a basic or standard tier VM, running full time, waiting for triggers. No timeout to speak of.

Now the first disappoints me, as I saw functions as a solution for my jobs that need to be fired once or twice per year, but then take a day or two to complete (comprehensive backup and data packaging for export).

The second confuses me - does this mean, that the stateless function now runs as a web app and I am to be charged for it as such? If this is the case, the whole concept of functions is now useless for my purposes, unless I implement a Cell processor, firing 80000 function instances upon a trigger to get the work done on time. If that's even possible.

Could someone please explain the model behind Functions pricing and what the best solution for my problem would be?

Thank you.

Answer 1

The Azure Batch Service is what you're after.

Basically, you can spin up the compute resources required, when required - and you only pay for the compute time that you use, the Batch Service itself (which provides all sorts of goodness such as error detection, retries, automatic scaling) does not incur an additional charge. (It has its own API, by the way - so you can completely automate it, if required)

You can install custom software on each VM in the pool before it starts performing work.

You can let the batch service scale automatically based on a customisable rule, or you can just create a pool of a fixed size and scale it manually.

https://azure.microsoft.com/en-gb/services/batch/

Update - according to comments on this GitHub issue, if your Function App is using an existing App Service Plan, then the timeout doesn't apply. Seemingly, the dynamic option or "Consumption Plan" timeout is to do with the timeout of underlying platform VM instances. So, that might be another route: pay for an app service plan, then you get to use Function App goodness without timeouts.

Answer 2

Functions are designed to handle short tasks (under 5 minutes). But there are workarounds. You can create an ARM Template to deploy a Function App with a Web App service Tier and delete it after the processing is done. You can use webjobs instead of functions (but you still have to pay for the Web App).

Answer 3

As someone on this thread said you should try looking at azure automation or something if your process is less frequently used and moreover takes a days to complete.

Without knowning exactly the scale (# of files / source and destination of file and backup) You could break the solution into smaller chunks (tasks => functions) and deploy the chunks in separate app plans

1. Master

   • This would be responsible for figuring out a roster of line items that need to be backed-up.
   • The master would simply raise a ticket in a queue or do a http ping to a worker.
   • This master function then can be deployed in a what is called "Application Service Plan" and can run for more than 5 minutes!!

2. Worker

   • This would be actually responsible for backup of a single unit of work.
   • This worker function can be deployed into a what is now called as a "Consumption Plan" previously known as "Dynamic Plan".
   • On a consumption plan (AFAIK) you could have 32 instance of the azure function running in parallel.
   • You can even clone the worker function app thereby take backup of 32/64/128 work units in parallel

It makes complete sense from a monetary point of view that on a Consumption Plan where you are charged by the about of Gb/s (memory and time of execution used) that the host prevents long running executions from happening.

Hope this helps

Answer 4

This thread deserves a bit more clarity. Firstly, If your function call takes some time to complete, you are going to want it to be async. Secondly, If you do make it async, it will not timeout after 5 mins.

Give it a shot :P

DateTime d = DateTime.Now;
Task t = new Task(() =>
{
    log.Info("Doing long task");
    for (int cnt = 0; cnt < 100; cnt++)
    {
        log.Info(DateTime.Now.Subtract(d).ToString());
        System.Threading.Thread.Sleep(10000);
    }
});
t.Start();

I am currently running them on a consumption plan for more than 5 mins, as per my post. There is a timeout, but it’s not on execution time obviously since I am running them longer than 5 mins. The time out is in the calling mechanism that executes the function. If you make it async, the caller returns and the function continues to execute. It is a common practice to put long running tasks on a separate thread/process than the calling thread/process. Keeping the orchestration layer of the function platform from blocking for long periods of time just makes sense. You should not block the calling layer for long running functions as it will seriously degrade the platform.