Leverage as part of action space

[sadimoodi]

Hello @AminHP ,
I noticed that you are using a constant leverage of 100 during the construction of the MtSimulator object, as you know leverage could be variant on the order level, means every order could have a different leverage, did you think of making leverage as part of the action space?
I was thinking that allowing the agent to decide on the leverage (1x,5x,10x...etc) could allow it to make better decisions?

[AminHP]

Hi @sadimoodi,
Before I released this project, I implemented order-level leverage. But then I realized that MetaTrader does not allow it. So I removed it. Anyway, your idea sounds interesting and I think it is worth trying.

[sadimoodi]

Hello @AminHP ,
thank you for your answer, many of the other platforms allow specifying leverage on the order level, which i think is a HUGE advantage and would surely result in a more profitable agent.
Going towards this goal, I included another dimension inside the action space, (say leverage was allowed up to 10x), to be Discrete(10) as follows:

self.observation_space = spaces.Dict({
            'balance': spaces.Box(low=-np.inf, high=np.inf, shape=(1,)),
            'equity': spaces.Box(low=-np.inf, high=np.inf, shape=(1,)),
            'margin': spaces.Box(low=-np.inf, high=np.inf, shape=(1,)),
            'features': spaces.Box(low=-np.inf, high=np.inf, shape=self.features_shape),
            'leverage': spaces.Discrete(10),
            'orders': spaces.Box(
                low=-np.inf, high=np.inf,
                shape=(len(self.trading_symbols), self.symbol_max_orders, 3)
            )  # symbol, order_i -> [entry_price, volume, profit]
        })

[AminHP]

I don't understand why leverage should be part of the observation space. Every order has its own leverage. So, I think it should only be part of the action space. The effect of leverage is implicitly considered by equity and margin.

[sadimoodi]

Apologies, I was confused, this is not what i wanted to suggest, here is my action space:

 # spaces
        self.action_space = spaces.Box(
            low=-np.inf, high=np.inf,
            shape=(len(self.trading_symbols) * (self.symbol_max_orders + 3),)
        )  # symbol -> [close_order_i(logit), hold(logit), volume]

the third dimension here will be reserved for the leverage and i can deal with it the same way you deal with the volume (take abs then use np.clip to keep the values within a specific range([1-10])
observation space:

self.observation_space = spaces.Dict({
            'balance': spaces.Box(low=-np.inf, high=np.inf, shape=(1,)),
            'equity': spaces.Box(low=-np.inf, high=np.inf, shape=(1,)),
            'margin': spaces.Box(low=-np.inf, high=np.inf, shape=(1,)),
            'features': spaces.Box(low=-np.inf, high=np.inf, shape=self.features_shape),
            'orders': spaces.Box(
                low=-np.inf, high=np.inf,
                shape=(len(self.trading_symbols), self.symbol_max_orders, 4)
            )  # symbol, order_i -> [entry_price, volume, profit]
        })
The 4th dimension here is used to represent leverage on the order level (in addition to entry price, volume and profit).

I have my reservations on the above approach, i will try to come up with another one which includes using Discrete action space instead of Box

[andydoc]

Some prop firms implement different leverage for different asset classes eg 100 for Forex, 30 for Indices etc so it is probably right to implement it at the order level... Perhaps as a dict at the environment to look up when placing a trade, against the asset class...