Skip to content

Queues, priorities and fair computing ressource distribution #1060

@scwgoire

Description

@scwgoire

Dear community, first of all thank you for making River, such a smart piece of engineering, that's lovely !

I'm having a fair resource access pattern I can't figure out how to fit within River's model. I think I need help.

Let's say I have a resource which all my customers can access one at a time. The customers submit job batches and each job needs to access the resource once. I would like all customers to fairly share the resource whatever the batch submission time was.

Imagine the following scenario:

  • customer A submits a 3 jobs batch (A1-A3)
  • customer B submits a 3 jobs batch (B1-B3) while A2 is being processed

I would like the execution pattern to be A1, A2, B1, A3, B2, B3 and not A1, A2, A3, B1, B2, B3.

The "clean" way to do this would be to use multiple queues, but in my case there isn't a fixed list of customers, so I believe it probably isn't the right path.

I also though about messing up a little with the priorities so that the jobs would have the following priorities:

  • A1: 1
  • A2: 2
  • A3, B1: 3
  • B2: 4
  • B3: 5
  • ... etc
    But I found out the priorities are bound to the 1-4 range. Plus there would be an overflow case to take special care for. So this doesn't seem an excellent option either.

Did I fail to find the River feature I need in the documentation ? What is the rationale for constraining priorities to such a few number set ?

Cheers!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions